Welcome to gyne.org, skillet89.
I think the confusion here is because there is a time when the gland is in an active swelling phase and a non-active phase. When it is in an active phase, THAT's when the anti-estrogen drugs you mentioned can help. These drugs not only help the pain associated with the early symptoms of gyne, but if given soon enough during the active phase, they can stop the tissue from growing further and possibly even make the swollen tissue reverse itself. Think of it as a rose that starts to bud, but then someone snips it off and it eventually withers and dies. If caught early enough in an active phase of gyne, these medicines can sometimes (not all the time) reverse the situation. This is why body builders use them immediately after a cycle.....to prevent any swelling of the gland, hopefully before it even starts.
After awhile, the "active" phase of gyne goes away, growth stops, and you're pretty much left with what you have. The longer the swollen glandular tissue has been there, the less chance you have of it going away with the use of the anti-estrogen medicines. If you wait too long, they really don't do much of anything to the tissue that's already there. So the key to these meds is to use them as soon as possible when you first notice changes or pain/tenderness going on in your breast. Typically, only endocrinologists will prescribe these meds. They are usually out of the realm of expertise of most family doctors and they usually will not prescribe them.
Letrozole is extremely expensive, as it does not come in generic form. Arimidex is another aromatase inhibitor (like Letrozole), but DOES come in generic form and is, therefore, much, much cheaper. Tamoxifen (or Nolvadex) is the medication usually used in early stage gynecomastia. It also comes in generic form.
All of these medicines are generally used in women being treated for breast cancer. Women usually take these types of medicines for several years, whereas, men usually only take them 2 or 3 months for gyne. You should be aware that they are not without some potentially serious side effects, so you should discuss this with your doctor if you are interested in trying them.