The earlier distinction between True and pseudo gynecomastia came from somewhere I am not familiar with.
You see, In most cases, Glandular tissue is surrounded by fat; or fat with glandular tissue growing through it. The result is the same either way.
The fatty part of the breast will increase and decrease as a person gains or loses weight, but will not completely do away.
The glandular portion of the breast tissue is the part most likely to shrink to the point that it becomes almost undetectable if there is not the hormonal support for it.
This happens to women as well. As her hormone levels fall off in menopause, the fibrous glandular tissue is prone to shrink. The Post-Menopausal woman is often left with sagging, fatty breasts which are somewhat less full than in her younger years.
Breast growth in men is actually quite common, and about 2/3 of men have some breast growth. Obviously, some lucky guys have none or very little.
Breast growth in puberty is quite normal and shrinkage is also quite normal. Many boys are told not to worry, that it will go away in a year or so. The truth is that the shrinkage may take several years but most often happens before the age of 20.
Adult onset Gynecomastia is less common and is almost always an unwanted side effect of some medication or drug; Though it can also be a symptom of many different diseases. If the cause is removed or treated quickly, The breast growth will usually shrink.
Some men are inclined to have fatty deposits that are trpically thought of as female. The breasts, hips, buttocks, and thighs are commonly thought of as feminine locations for fatty deposits but some of us have fat deposits there too.
If you have been seriously obese and then lost a significant amount of weight, your skin may have stretched and you'd be left with small saggy breasts. (This is very common also on elderly men)
Finally, there are those of us who inherit Gynecomastia as a family trait. For us, it will not go away.
As you see, the whole subject is very iffy. Each of us is different. If you really want a good answer to your question, The best response would likely come from an Endocrinologist.