Hi Plarkin,
I'll attempt to address the 4 causes you listed:
Androgen Resistance Syndrome: this usually occurs when the testosterone receptor is not functioning properly. If this were the case, you'd have female external genitalia (i.e. no penis, testicles), but male internal genitalia (vas deferans, seminal vesicles, prostate, etc). Suffice it to say, I'm sure someone would've noticed this by now.
There is an alternative, incomplete androgen insensitivity, where it's not as extreme as the aforementioned. Again, there is likely to be ambiguous genitalia.
Enzyme Defects of Testosterone Production: Similar to the above, you'd exhibit the consequences of a mal-functioning testosterone pathway. The fact that your hormone tests are in the normal range rule this out as a possibility. You are producing testosterone, and therefore your enzymes are fine. This is also evident in the fact that you have facial hair. The thin facial hair can be attributed to other things such as your genetics, and ancestry.
Increased Peripheral Aromatase: This is definitely plausible, and moreso if you are overweight or obese. However, the problem would be manifest from the testosterone being converted to excess estrogen by the aromatase enzyme. In that case, the lab tests would have identified high estrogen levels.
Hypogonadotrophic Hypogonadism: This condition implies that there is low testosterone levels. However, the cause is not in the testes. Instead, either the hypothalamus or the pituitary are malfunctioning. They are not providing the testes with the adequate signal to produce testosterone. As a result, your testosterone levels would be low, and this would be found on the lab test. Further, I'm not sure if you have access to this information, but one (or more) of the following hormones would also be low: GnRH (gonadotropic releasing hormone, if the hypothalamus is the problem) or LH/FSH (leutinizing hormone/ follicle-stimulating hormone if the pituitary is the problem). A low testosterone level would be the first sign of problem in this case, and any endocrinologist would have immediately ordered an LH/FSH. I doubt 3 different endos missed this diagnosis.
This brings us back to the possibility of 'idiopathic' gynecomastia. Idiopathic simply means that the cause is unknown. In the vast majority of gyne, this is the case. It's probably due to the large variation in hormone levels at the onset of puberty. Your estrogen levels were probably out of whack, but are now in check. However, the breast tissue formed in that time out of excess estrogen.
Hope this message helps.