That is true. Which why I never understood why we are told at any age, it will go away in a couple months, meanwhile in a couple years we find that isn't not only true, but it has multiplied in size. Muscle can tighten the tissue but it projects it as well as it builds under. Weight loss and dietary habits can significantly minimize adipose tissue in the glands and around it and as the fat loss takes place so does chest size to a degree. Problem is estrogen loves fat, breast tissue loves fat, so getting it to go away compared to other areas of the body isn't as quickly seen.
My own theory is as a teen, glandular growth is normally minimal and the growth is more fat tissue. As more testosterone begins to sculp the body the fat redistributes and the prominence goes away and only a small amount of glandular tissue remains but nothing as pronounced as the fat before. Hence how men can have breast cancer without having breasts. We all have varying amounts of glandular tissue. Some hardly. Some, quite a bit.
For those of us who have had breasts since puberty, something within us didn't get the message to stop developing glands and redistribute the fat. So our breasts continued to develop as girls do in the chest department. Some developing quite feminine chests and others only one breast or small breasts.
That's my theory.