I wear a 36A and fill it quite comfortably, i don't spill out of it and it's not completely full as in, certain movements space is created between breast and cup. I don't wear one hardly ever unless I go for a long run which I do weekly as I'm avid in it and enjoy the challenges of distance running. When my chest initially began growing some tissue I went to the Dr and he to felt around and discussed that it was breast tissue and asked if it made me uncomfortable or sore, at the time, no. But now that they have grown from slight tissue before, to now in the matter of a few years since then, for no known cause, to a full a cup, I find the bouncing when I run distracting and uncomfortable. So I wear sports bras and every now and a then I wear regular ones around the house.
Like your wife, my wife is not thrilled that I have them and far as she knows, as it's been a good year since she knew I wore one, and doesn't go through my drawers, probably thinks I threw them all out. I respect her dislike to my desire to wear a bra for comfort, and unless my breasts grow further in to full b/c range I can't really argue it's necessary since majority of women I know with smaller breasts openly admit to wearing bras cause a)they are women and b) it gives them shape even though their breasts are not shapely.
The bra is definitely definitive to women as a garment solely for women. As women are generally the ones with breasts sizable enough to require a garment for stability and comfort in the chest. However, if it is causing you pain then just wear one. I wear barely there bras and order a size up so if you are an a order a b if you get one cause they are snug but very comfortable and breathable. Also perhaps go back to the dr and just ask for a dr slip to suggest a bra. I feel if a woman sees that a medical professional advises it, they may dislike it, but they will respect it. Vs their husband going dr said i gotta wear a bra, i feel most women would just see that as suspicious. Cause yeah, women don't get it