Judging wine judges

As reported in The Telegraph:

A few years ago, a frustrated vintner named Robert Hodgson, who had a background in statistics, thought of a way of testing the testers. He wondered what would happen if he supplied 100 wines for consideration but, without the judges knowing, slipped each wine to them three different times...

Having sought the agreement of the chief judge, G M “Pooch” Pucilowski, Hodgson ran his study for four consecutive years. When the results were calculated, they staggered him, disappointed Pooch and infuriated others...

“We gave each judge a flight of 30 wines,” he says. “They’d all have three samples of the same wine, but they didn’t know it. These samples were arranged randomly. When I first saw the results I could hardly believe them. They scored the identical wines like they were different. It was staggering.” Out of a 20-point scale (scored, for esoteric reasons, between 80-100), an identical drink would typically vary by four points from one tasting to the next.

More at the link, including a discovery that judges who were excellent one year might not be able to repeat their performance on subsequent years.