I agree. People do not think that someone who is wearing some sort of support is doing so just to have support on their body. They are wearing the support for a need. I have had numerous people where I work (boiler engineer at a church as well as a computer tech), greet me with a pat on the back, a hug or just place their hand on my shoulder, and since I wear a sports bra almost constantly (not in the shower) for support, no one has said anything about it. They probably think it is just some sort of tee-shirt. At work I wear a semi-compressive sport bra but when I am out hiking I find a slightly padded lift sports bra to be very comfortable. The point is we are the ones with a need regardless of who we are and most people will understand.
A good example of this is when I told my mother about my gyno. I was a nervous wreck but she was totally supportive (pun-intended), and was curious as to how I took care of them. She was very understanding of the whole event. People care...at least most people.