Busy thread!
To answer more on the GRE question:
1) The type of school really matters for GRE performance. I'm not talking about just Ivy League vs community college here- it's also well documented that students from liberal arts colleges do not do well at the GRE, because that type of education is not conducive to test taking. Some departments are also small in those places and can't offer courses every year, so it's not unheard of to say not take quantum mechanics until your senior year... and because you have to take the test either end of junior year or beginning of senior year, you're fucked. (At this point, I should point out that of course women favor liberal arts colleges, and traditionally minority colleges also tend to have small physics departments.)
2) The test is really expensive- not sure how much it is now, but it was like $150 when I took it, and then you had to pay extra to send the scores to more than three schools, and can only be taken in specific locations three times a year. What's more, you can take it multiple times, and the only score that matters is your top one. I'm sure tons of students would do better if they could take it more times, but can't afford it or have scheduling constraints.
3) This is obviously an exam that requires you to do a ton of extra studying beyond usual coursework to do well on. To emphasize this, I never really had to memorize equations before in physics for exams, because it's pretty well-demonstrated there's no point when you can look them up, so they'd either be on the exam or you could make a cheat sheet to bring in (an index card or whatever). First thing you have to do when studying for the GRE? Literally memorize about 150 equations!
That took a lot of time! Then, because of the nature of the test (100 questions in 90 minutes), you literally can't work out all the problems as you normally would, so need to learn "tricks" like how to take limits to see where the answer will go, dimensional analysis, and all sorts of other stupid shit you're probably not going to use outside this artificial exam setting. College is already a super busy time, but at least I could carve out time to do it- it's not hard to imagine if you had a job with long hours or family to attend to you're just not going to do it.
4) To emphasize on the latter, it's been documented through research for example that in Latina families care of the family falls on women (I've also had Latina friends say this to me- of course you love your little siblings and want to help out, but guess who's expected to babysit in college still while your brother isn't?). Pretty obvious how that would affect something like point 3. I'm sure there are a ton of other things like that, some more subtle and some less so, that are too extensive to document here, but that's the first one I thought of.
To close, this might be anecdotal but Xander in the Netherlands (who's overseen over 100 astronomy PhDs, so think he knows what he's talking about more than anyone I know) has said that he has never met a student incapable of finishing a PhD. He's had students who needed to switch projects, or had family/life matters intervene, or other external things, but at the end of the day if you have the drive and ability to be accepted into a PhD program that person is good enough to finish. I will note though, the Netherlands does not require the Physics GRE.
But yes, I think your comparison to the insanity of residency is apt (which also is more likely to affect these minorities, of course). There's a lot of things in getting a PhD that are like that actually.
Yes, I have a life. It's quite different from yours.