Thanks for your closed PM @moobzie. It's perfectly clear where you're coming from, thank you for formalizing it. The problem lies in people who state their opinion and disallow others to, or suggest they do not, express theirs--regardless of the facts and truths they know and observe.
For me:
Gender = how people express their identity
Assigned Sex = what a doctor thinks they see when they pull us out
Biological Sex = what our chromosomes and our genes, etc. tell our bodies to do
Breasts on males, gynecomastia, are a part of the latter or caused by medical inducement (eg. side effects to medication). Those with idiopathic gynecomastia have some sort of hormonal/genetic predisposition that makes their body develop breasts. Those with simplified and, irrefutable, opinions on gender/sex will often get hung up here. Some may see breasts on a male as a sign of degeneracy. Some males may feel a sort of religious-based shame because of their breast tissue and have difficulty refuting this natural "affront" to their masculinity. Some have theirs removed to reinforce their gender presentation, just as some females augment their breasts to reinforce theirs. Regardless of their breast size, their choice in handling those breasts, if they act in a masculine manner they are men; no less men than any person without breast tissue. That's the heart of the "gender debate", whether people get to have a say in what their gender presentation is.
This is no different than people such as María José Martínez-Patiño, an athlete who was told was female at birth, raised as a girl and identified as a woman. But guess what? She had XY chromosomes. She also has androgen insensitivity disorder, which made her genetically male body develop in a female manner. She only found out because of our limited XX/XY views of sex at the time. These views limited her ability to compete. Her condition predisposed her to no advantage versus her competition, but was forced to withdraw "due to injury" or be publicly outed.
Beyond intersex people, there have been people on the gender-spectrum throughout history, people who have desired or been compelled to present to the world counter to what was assigned to them upon delivery. The rules of gender presentation ebb and flow constantly, too. Powerful men, rulers of empires, wore cosmetics in Egypt. All people, before trousers were cheap enough to make, wore some form of skirt. To think today's social norms are anything but flimsy guidelines set upon us by those that wish to separate us from our money, or have some desire in controlling society, is folly. Trans people are people, too and they've been here since the dawning of time!
I think what people gnash their teeth and wring their hands about are due to the fact that our concept of reality has been pretty limited. There are real concerns of fairness in sport, because we've operated in a limited binary view of ourselves, and there are bound to be outliers that challenge that. The answers are being worked on and it's ugly work, mostly because people are so ugly in trying to enforce their biases. There are answers, we simply must be good humans and find them.
Also, books, problematic because there are adult themes presented in some. My take is that voices of those outside the enforced gender standard have been silenced either passively or actively. The narratives that have come out are real, powerful, and true. These biographies and interpretations of author's history give people hope that they aren't alone, strange, or different. What if we, as predominantly males with gynecomastia, were able to read a book as youths about it and how it is common both in teenagers and in adults and it says nothing about our gender and enforces our normalcy. Would we have the shame we've carried with us our whole lives? Would we need a forum to congregate to and find community in for acceptance? It's the same thing for kids of color, those with non-binary sexual orientation/gender/sex, etc. They have been able to see themselves in a book and feel better about themselves, feel understood, or find understanding. The books aren't grooming them, they are affirming them, like what we find discussing ourselves here. Maybe the medium is rough around the edges, maybe these truths contain themes that are more adult than some children can handle or should be exposed to, but the goal of those that would remove these books from shelves aren't about finding better representative stories. They are solely dedicated to removing them. They think people that that exist outside of their limited worldview are degenerates, broken, ill; that they should be removed from view.
The real degenerates are those that force their children to have sex with prostitutes to show their questioning child that they are men (PragerU vidoc on detransition); that force their children into boot camps of shame and suicide to beat into them they are not who they think they are; that shame and disown their own children to the point of suicide.
I'm sorry if this upsets someone's apple cart, but I won't sit here and let someone tell others to "keep it to themselves" while they get to spread their nonsense unchallenged.