It is unusual to see delayed onset of gynecomastia, where none existed before, without some cause, such as an endocrine issue, a medication, or some other external factor. A CT scan can be worth while if there's concern over a pituitary tumor, for example, but if this is causing your gynecomastia, one might expect to see an elevated prolactin level. The laboratory testing tends to be reasonably sensitive, where a CT scan is fairly non-specific and in the absence of something suspicious, not all that sensitive either.
Your comment about "serious supplements" raises my concern, primarily as I have seen a number of patients who have used sports supplements that should not have been suspect as causes of gynecomastia, but in the end were likely contributors. Two of the patients brought the products in, which they most closely associated with the development of their condition, and at least one of those products contained ingredients labeled as "proprietary blend." This was in an NO2 product. The other was a protein powder, and there were ingredients other than protein on the label, but none that I could identify specifically as potentially causative. Because of the variability of regulation in the supplement industry, there are instances of contamination of non-hormonal products with hormonally active substances, and similarly, products that are billed as one thing may have ingredients that do other things, including causing side effects. Perhaps if you are able to look back at any supplementation you were doing when the condition started, you might identify a culprit that you didn't think was a problem.
In the absence of any of those issues, a thorough evaluation that is normal three times over an extended period of time would suggest that it's safe to go ahead with surgery, though, obviously, there is no guarantee that you wouldn't experience a recurrence. Similarly, there is no way to say that recurrence is a sure thing, even if the issue remains unresolved. Caution is certainly appropriate, but there are reasons that "idiopathic" is a common diagnostic term in medicine.
Good luck,
Rick Silverman