The argument is made that gender and sex (interchanbeably) are not binary, but a spectrum, on the basis of 3 arguments:
The issue with the first is that the contention of sex not being a binary has manifested not due to the existence of intersex people, but due to claims being made to modify laws and services to accommodate needs of trans people who, in most cases, are wishing to be considered as the opposite gender. This arguments for these accommodations always state that gender is a spectrum and that the evidence is the presence of intersex people. This does not seem to be related to the needs of trans people, who, as stated, are mostly wishing to identify as the opposite gender and would like to use serives and amenities normally allocated to that opposite gender. In the cases where a trans person is claiming to be non-binary, it is not being done through scientific analysis, and it would be a gross assumption to believe that these non-binary trans people are intersex.
The issue with the second is that qualifying the system property of gender, or sex, as a spectrum on the basis of variation in endocrine expression would necessarily qualify all humans as not being completely male or female. By such a standard, could it not be argued that all humans are trans?
The argument on the structures of the brain is in its infancy and one would be hard pressed to be too apply it universally, as it is not likely that most people who claim to be trans would necessarily have the structural properties which are being suggested as leading to the behaviours proposed as being associated with transgenderism.