I fit both criteria, and neither I nor my mentally ill friends would find it offensive or off-putting.

Of course, a quick poll also confirms we can't be bothered getting worked up about the casual use of "crazy", so we may not be representative.