It seems to me that you are simplifying things a bit.
There is a shared experience of womanhood, which is comprised of many individual subjective experiences. Trans women certainly participate in that experience along with all women. One way to identify whether a person participates in this shared experience is to ask them.
But womanhood is also a set of physical “tendencies”, many of which are apparent because the human species is sexually dimorphic. Different individuals participate in those tendencies to different degrees. Many trans women do not participate in those tendencies until they start to physically transition with medical help. I’m not sure anybody ever fully participates in all of these tendencies.
But these tendencies do influence and help shape the shared experience. In particular they do seem to shape the shared experience of women’s sports.
I’m not entirely sure exactly how they shape it. Elite men athletes are better than elite women athletes. I’m not sure why, or if anybody knows exactly why. But i do think this is a reality that must be addressed.
For example, I remember reading that a particular sports organization decided that testosterone levels would be decisive in whether a person could compete in women’s sports. In that case, part of participating in women’s sports is training without the advantage of higher testosterone levels.
In this is true then some women are barred from women’s sports without drugs to lower their testosterone levels. This should not be an argument that they are not women. They can still largely participate in womanhood. But they are excluded from this aspect of it, which can cause a lot of hurt (that i’m sure i don’t fully appreciate.)
This leads to your argument that the needs of trans girls are the most urgent, and trump everything else. I have a lot of sympathy for this argument. But I was only a mediocre athlete. I enjoyed competitive sports but they never gave my life any meaning beyond the camaraderie involved. And I wonder if that keeps me from fully appreciating the cost to the meaning of women’s sports.
It may also be that testosterone should not be the decisive difference between men’s and women’s sports. Testosterone is just one advantage among many. Then we must argue over the meaning of sports. And to some degree we are creating that meaning through that argument, but it is creativity that must be constrained by reality in the sense that we cannot ignore these particular physical tendencies of women.
In summary, then, I do not think it is unreasonable for women athletes, particularly elite ones, to worry about fairness in the game. It is not automatically transphobic (although it can be). And it is a really hard issue.
A good analogy may be the debate over whether Oscar Pistorius could compete with his artificial legs. That debate was appropriate, but also required a great deal of sensitivity. The need for sensitivity here is even greater. Nobody tried to argue that Pistorius was not really human, or a man, because his bionic legs might give him an unfair advantage in running. Unfortunately, people are trying to argue that trans women are not really women because they might, on average, have a competitive advantage over other women.