A Yale professor used real data to demonstrate a potentially serious flaw in the system of judging the figure skating competitions in this year's Winter Olympics -- one that he says can leave the determination of the medal winners to "dumb luck" in close contests.
John W. Emerson, assistant professor in the Department of Statistics, likes to show his students real-world applications of probability, statistics and number crunching. He frequently uses sports statistics in the classroom, and last year estimated the University of Connecticut's chances of repeating as the national champion in the NCAA basketball tournament.
While gathering data for his introductory statistics class, Emerson came across the score sheets for the 2006 European women's skating championship, in which only 84-hundredths of a point separated the second-place and fifth-place skaters in the Short Program, one of the competition's two parts.
That contest used a new judging system introduced by the International Skating Union (ISU) after a scandal during the 2002 Winter Olympics when it was discovered that one judge was deliberately skewing her scores to favor a pair of Russian skaters over the Canadian team. The scandal resulted in the first-ever award of duplicate gold medals.
The revised judging system of the ISU has two new components: rather than having points deducted for mistakes, skaters earn points for successfully completing elements of their routines; and instead of using the scores from all 12 judges, a computer randomly selects nine judges' scores to determine the final tally of each program segment.
It is this second component that Emerson says is problematic.
"Does the new scoring system increase fairness?" Emerson wrote in a paper titled "The Computer: A Phantom Figure Skating Judge?" (posted online at www.stat.yale.edu/~jay). "On some level it does, but the system has introduced the unsettling possibility of dumb luck influencing the medal standings. In a close competition with skaters separated by only a few points, the outcome will likely be determined by the random choices of panels of nine judges."
Emerson noted that there are 220 possible combinations of nine-judge panels for each of the two program segments, or 48,400 combinations of panels of judges in total; only one of these actually determines the final outcome. He looked first at the scores from the Short Program of the European competition, and found that less that one-quarter (50) of those possible panels (220) would have resulted in the same ranking of the skaters following the Short Program.
"Random elimination of a different set of judges could have radically changed the standings," Emerson wrote. In fact, in that particular competition, he noted, the only ranking that was unchanged in all 220 possible panels was that of the first-place skater.
Once the Olympics were underway, Emerson applied his methodology to the pairs skating competition. He posted his conclusions online at the above address in a paper titled "Olympic Figure Skating: Silver and Bronze, or Bronze and Silver?"
His calculations again showed that randomly choosing a different panel of judges could have resulted in a different outcome -- with 12% of the possible panels awarding a bronze to Pang Qing and Tong Jian of China instead of ranking them in fourth-place. Had the computer picked a different set of scores, bronze medalists Shen Xue and Zhao Hongbo, also of China, could have finished as high as silver (in 4% of the possible panels) or as low as fourth (12%).
"My recommendation: use the scores of all 12 judges," wrote Emerson, adding, "Unfortunately for Pang and Tong, this would have awarded the medals in the observed order -- they would still have finished fourth."
Emerson also added in a lighthearted note that his discovery "started out as a fun example for the students, and it turned into a monster that ate my weekend.
"I should point out, however," he continued, "that in no way are the results of past competitions, or the Olympics this week, illegitimate. Rules are rules, and they will be applied fairly to determine the medalists at the Olympics. There are concerns about nationalism and block voting influencing the results." Nevertheless, Emerson said, if the random elimination "addresses this concern (and I'm not sure it does), there is still a cost to be paid, by the skaters in close competitions."
T H I S
W E E K ' SS T O R I E S
University divests from its holdings in Sudan . . .
New degree program promotes 'green' architecture
Changing the world -- one spring break at a time
University Library launching 'Iraq ReCollection' project
Yale and the 2006 Winter GramesUnited Way honors Yale for goal-topping campaign
Elimelech elected to NAE in honor of work on water-quality control
Endowment will usher in 'new era' for Yale women's ice hockey
Partnership to increase India's environmental resources
Ozone causes premature death even at low levels, study finds
Columnist decries lack of response to genocide in Darfur
Donated books a 'reminder' of once-blacklisted Yale neuroscientist
Slifka Center hosting Jewish Alumni Weekend
More on the Grammys ...
Campus Notes
Bulletin Home|Visiting on Campus|Calendar of Events|In the News
Bulletin Board|Classified Ads|Search Archives|Deadlines
Bulletin Staff|Public Affairs|News Releases|
E-Mail Us|Yale Home