There's also the hypocritical nature of this argument, which is that trans people using the bathroom is somehow more of an invasion of privacy than people policing which bathroom they belong in. In fact, this will inevitably lead to (in fact, it has already has led to) non-trans people being harassed in bathrooms for looking like they might be trans.
https://www.cracked.com/blog/the-real-reason-trans-bathroom-panic/
https://www.cracked.com/blog/the-real-reason-trans-bathroom-panic/