Serving the Quantitative Finance Community

 
User avatar
outrun
Topic Author
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Is Hou Yifan right??

February 4th, 2017, 9:05 pm

In Just 5 Moves, Grandmaster Loses And Leaves Chess World Aghast

What would the pool gender ratio have to be for her pairing to be statistically significantly biased? 
 
User avatar
Paul
Posts: 7047
Joined: July 20th, 2001, 3:28 pm

Re: Is Hou Yifan right??

February 4th, 2017, 9:52 pm

Or a crazy/paranoid chess player...what are the chances of that?!
 
User avatar
outrun
Topic Author
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: Is Hou Yifan right??

February 4th, 2017, 9:57 pm

Hahahaha!
 
User avatar
outrun
Topic Author
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: Is Hou Yifan right??

February 4th, 2017, 11:29 pm

Suppose 70% if the participants was male (but could be more since in general less than 10% of master level players are female), then there was a 1% chance that she was randomly paired with3 or less males.
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Re: Is Hou Yifan right??

February 5th, 2017, 1:04 am

There's two issues that make these kinds of biases much more likely than it seems:

1. Although the chances that a particular female chess player would be randomly paired with so many female opponents drawn from a male-dominated pool is extremely low, the chance that at least one female chess player faces this kind of "bias" is quite high. If fact, if such odd biases did not occur, you'd know the system was rigged. That is, the distribution of realized gender ratios should have tails and some players inevitably end up in them.

2. If the pairing is randomized but biased toward opponents of similar scoring, then it's the gender ratio in the player's score band that matters. This has two bias-enhancing effects: if the ratio shows a trend, the player's expected gender ratio might be much different from the overall ; and 2) stochastic effects guarantee that some scorebands will be lumpy WRT gender.
 
User avatar
ppauper
Posts: 11729
Joined: November 15th, 2001, 1:29 pm

Re: Is Hou Yifan right??

February 5th, 2017, 7:15 am

1) are the pairings all done ahead of time, or do the results of earlier rounds affect later pairings? 

2) Why is she upset? she doesn't like playing women? I presume the female players were in the tournament on merit and I don't see the difference between playing a male and a female chess player with the same ability.
 
User avatar
outrun
Topic Author
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: Is Hou Yifan right??

February 5th, 2017, 11:50 am

Some data:
255 players in this tournament
the top 100 woman in the world

the ids  (rating ranked, in this tournament) of woman that are in the world top 100 AND were playing in this tournament:
22 Yifan
38 Wenjun
47 Muzychuk Anna
51 Muzychuk Mariya
54 Lagno Kateryna
56 Gunina Valentina
58 Stefanova Antoaneta
66 Batsiashvili Nino
76 Paehtz Elisabeth
80 Javakhishvili Lela
83 Khademalsharieh Sarasadat
85 Zhukova Natalia
86 Zatonskih Anna
88 Khotenashvili Bela
92 Karavade Eesha
95 Shvayger Yuliya
96 Pustovoitova Daria
97 Vega Gutierrez Sabrina
100 Batchimeg Tuvshintugs
(more later for the range 101-255!!)
Last edited by outrun on February 5th, 2017, 12:00 pm, edited 2 times in total.
 
User avatar
outrun
Topic Author
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: Is Hou Yifan right??

February 5th, 2017, 11:57 am

Some things we can test:
* what is the probability that  Yifan is paired against 7 or more woman in this tournament, assuming random pairing? Assuming "strength biased" pairing?
* what is the probability that something improbably happens given the number of top tournaments (number of tournaments she played)?
* how were the other woman paired (note there is a bias in all woman we need to correct for if Yifan was biasedly paired with other woman)? Were woman in this tournament in general paired to more woman? Or is Yifan an exception?

I can imagine something is going on: 
* she's clearly upset -could be paranoid, or she could be running into things-, 
* she might have stepped on some toes: "In May 2016, Hou dropped out of the current Women's World Championship cycle, effectively relinquishing the crown.". * I don't believe they would do random pairings, I can imagine that they want to have good players meet later on, not in the early rounds..

She could be over-sensitive, or people could be trying to be in her way. Let's try to find out!
 
User avatar
outrun
Topic Author
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: Is Hou Yifan right??

February 5th, 2017, 11:34 pm

..some initial results, I collected the player list from the competition site, enriched it with gender data from the fide database, linked up the matched and extracted who-played-who, and this is what I got:

42 females, 213 males

Below you see the pairs of who-played-who, the x and y axis are the rank of the player in the competition, 1 is the best player (elo score), 2 the 2nd best. etc. It's clearly not random! What are those lines?? Maybe it's some standard algo? Maybe we should make a similar pairing plot based previous year's competition?
The red dots are Yifan's opponents. Her conditional distribution looks the same as players with similar rank. One thing to look for is to see if they swapped a selected male opponent with a female one? Perhaps a bit of nudging.

Image
And here are the p-values that test for the distribution of observed M/F ratios. The y-axis is log10, and we have limited ratios (only 10 games played) but in general p-values should be uniformly distributed (the red line, between 1/250 t the left to 1). Yifan's pvalue is the one at the bottom left, her p-value is approx 1/5.000. 
Image
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Re: Is Hou Yifan right??

February 6th, 2017, 2:27 am

Interesting!

It looks like the algorithm AVOIDs matching players of similar rank (otherwise the diagonal hole would be filled). The line seems to explicitly match players of rank x to those of x - N/2 (N = the population size) perhaps with some deviations in the pattern of prior pairings or scheduling conflicts.
 
User avatar
outrun
Topic Author
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: Is Hou Yifan right??

February 6th, 2017, 6:55 am

Yes! Tonight I'll split it by round. I've read somewhere that in the "Swiss system" you initiate pairing based on ranks (the off disgonal lines?) and in the next rounds you let winners of the previous round play losers of the previous round. If the observed pairing becomes explainable then we can look for manipulation (or reject it).
 
User avatar
MattF
Posts: 6
Joined: March 14th, 2003, 7:15 pm

Re: Is Hou Yifan right??

February 6th, 2017, 11:22 am

 Hou Yifan is wrong on numerous levels but first you need to know some background. The annual Gibraltar tournament is probably the most female-friendly chess tournament in the world. This year they had Womens Prizes of £15K, £10K, £5K all the way down to 12th £500. This is incredibly generous and not surprisingly attracts a huge number of strong female players. Hou is fully aware of this because in 2015 she had a breakout result of 3rd= picking up the women's prize too. She actually took home more money than the winner! So this is a classic case of "biting the hand that feeds you". I hope GM Stuart Conquest (the organizer and unquestionably one of the good guys) tells her to GTFO for future tournaments.

Cliffs:
  • The pairings were correct according to FIDE rules and generated by the standard software.
  • The high number of female opponents is due to the large number of strong female players at Gibraltar.
  • As a recent women's world champion (!) Hou Yifan should have been delighted to play so many women.
  • If you have an issue with the organisers you don't deliberately throw a game and question the integrity of the tournament.
Last edited by MattF on February 6th, 2017, 11:46 am, edited 1 time in total.
 
User avatar
MattF
Posts: 6
Joined: March 14th, 2003, 7:15 pm

Re: Is Hou Yifan right??

February 6th, 2017, 11:41 am

Yes! Tonight I'll split it by round. I've read somewhere that in the "Swiss system" you initiate pairing based on ranks (the off disgonal lines?) and in the next rounds you let winners of the previous round play losers of the previous round. If the observed pairing becomes explainable then we can look for manipulation (or reject it).
Don't bother. Do you really think chess geeks aren't all over this? The entire tournament pairings have already been duplicated independently using the same software. Hou Yifan is simply wrong.
 
User avatar
outrun
Topic Author
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: Is Hou Yifan right??

February 6th, 2017, 12:03 pm

Yes! Tonight I'll split it by round. I've read somewhere that in the "Swiss system" you initiate pairing based on ranks (the off disgonal lines?) and in the next rounds you let winners of the previous round play losers of the previous round. If the observed pairing becomes explainable then we can look for manipulation (or reject it).
Don't bother. Do you really think chess geeks aren't all over this? The entire tournament pairings have already been duplicated independently using the same software. Hou Yifan is simply wrong.
Thanks!
They did?? Has anyone written something about it (link?)?
 
User avatar
Paul
Posts: 7047
Joined: July 20th, 2001, 3:28 pm

Re: Is Hou Yifan right??

February 6th, 2017, 3:22 pm

Yes! Tonight I'll split it by round. I've read somewhere that in the "Swiss system" you initiate pairing based on ranks (the off disgonal lines?) and in the next rounds you let winners of the previous round play losers of the previous round. If the observed pairing becomes explainable then we can look for manipulation (or reject it).
Don't bother. Do you really think chess geeks aren't all over this? The entire tournament pairings have already been duplicated independently using the same software. Hou Yifan is simply wrong.
Thanks!
They did?? Has anyone written something about it (link?)?
Yes, me, in this thread "...crazy/paranoid..."