Wine tasting: junk science?

[AdamSmith]

Wine-tasting: it's junk science

Experiments have shown that people can't tell plonk from grand cru. Now one US winemaker claims that even experts can't judge wine accurately. What's the science behind the taste?

• David Derbyshire
• The Observer, Saturday 22 June 2013

Some academics have cast doubt on the ability of professional tasters to judge wines consistently. Photograph: A G Holesch/Getty Images/Imagebroker RF

Every year Robert Hodgson selects the finest wines from his small California winery and puts them into competitions around the state.

And in most years, the results are surprisingly inconsistent: some whites rated as gold medallists in one contest do badly in another. Reds adored by some panels are dismissed by others. Over the decades Hodgson, a softly spoken retired oceanographer, became curious. Judging wines is by its nature subjective, but the awards appeared to be handed out at random.

So drawing on his background in statistics, Hodgson approached the organisers of the California State Fair wine competition, the oldest contest of its kind in North America, and proposed an experiment for their annual June tasting sessions.

Each panel of four judges would be presented with their usual "flight" of samples to sniff, sip and slurp. But some wines would be presented to the panel three times, poured from the same bottle each time. The results would be compiled and analysed to see whether wine testing really is scientific.

The first experiment took place in 2005. The last was in Sacramento earlier this month. Hodgson's findings have stunned the wine industry. Over the years he has shown again and again that even trained, professional palates are terrible at judging wine.

"The results are disturbing," says Hodgson from the Fieldbrook Winery in Humboldt County, described by its owner as a rural paradise. "Only about 10% of judges are consistent and those judges who were consistent one year were ordinary the next year.

"Chance has a great deal to do with the awards that wines win."

These judges are not amateurs either. They read like a who's who of the American wine industry from winemakers, sommeliers, critics and buyers to wine consultants and academics. In Hodgson's tests, judges rated wines on a scale running from 50 to 100. In practice, most wines scored in the 70s, 80s and low 90s.

Results from the first four years of the experiment, published in the Journal of Wine Economics, showed a typical judge's scores varied by plus or minus four points over the three blind tastings. A wine deemed to be a good 90 would be rated as an acceptable 86 by the same judge minutes later and then an excellent 94.

Some of the judges were far worse, others better – with around one in 10 varying their scores by just plus or minus two. A few points may not sound much but it is enough to swing a contest – and gold medals are worth a significant amount in extra sales for wineries.

Hodgson went on to analyse the results of wine competitions across California, and found that their medals were distributed at random.

"I think there are individual expert tasters with exceptional abilities sitting alone who have a good sense, but when you sit 100 wines in front of them the task is beyond human ability," he says. "We have won our fair share of gold medals but now I have to say we were lucky."

Robert Parker, above is the world's leading wine critic and his score is key to determining the price of a new vintage. But Orley Ashenfelter, a Princeton economist, invented a simple mathematical formula based on weather data to predict the price of vintages, which mimicked the predictions of Parker’s system. Photograph: Shahar Azran/WireImage

His studies have irritated many figures in the industry. "They say I'm full of bullshit but that's OK. I'm proud of what I do. It's part of my academic background to find the truth.''

Hodgson isn't alone in questioning the science of wine-tasting. French academic Frédéric Brochet tested the effect of labels in 2001. He presented the same Bordeaux superior wine to 57 volunteers a week apart and in two different bottles – one for a table wine, the other for a grand cru.

The tasters were fooled.

When tasting a supposedly superior wine, their language was more positive – describing it as complex, balanced, long and woody. When the same wine was presented as plonk, the critics were more likely to use negatives such as weak, light and flat.

In 2008 a study of 6,000 blind tastings by Robin Goldstein in the Journal of Wine Economics found a positive link between the price of wine and the amount people enjoyed it. But the link only existed for people trained to detect the elements of wine that make them expensive.

In 2011 Professor Richard Wiseman, a psychologist (and former professional magician) at Hertfordshire University invited 578 people to comment on a range of red and white wines, varying from £3.49 for a claret to £30 for champagne, and tasted blind.

People could tell the difference between wines under £5 and those above £10 only 53% of the time for whites and only 47% of the time for reds. Overall they would have been just as a successful flipping a coin to guess.

So why are ordinary drinkers and the experts so poor at tasting blind? Part of the answer lies in the sheer complexity of wine.

For a drink made by fermenting fruit juice, wine is a remarkably sophisticated chemical cocktail. Dr Bryce Rankine, an Australian wine scientist, identified 27 distinct organic acids in wine, 23 varieties of alcohol in addition to the common ethanol, more than 80 esters and aldehydes, 16 sugars, plus a long list of assorted vitamins and minerals that wouldn't look out of place on the ingredients list of a cereal pack. There are even harmless traces of lead and arsenic that come from the soil.

Three of wine's most basic qualities – sweetness, sourness and bitterness – are picked up by the tongue's taste buds. A good wine has the perfect balance of sweet from the sugar in grapes, sourness from the acids, particularly tartaric and malic acid, and bitterness from alcohol and polyphenols, including tannins.

Many wines are more acidic than lemon juice and are only palatable because that acidity is balanced by sweetness and bitterness. "It's the holy trinity of the palate – sugar, acid and alcohol," says Dr James Hutchinson, a wine expert at the Royal Society of Chemistry.

Professionals distinguish between the balance of these three basic elements and a wine's flavour. And here the chemistry gets more complicated.

The flavour of wine – its aroma or bouquet – is detected not by the taste buds, but by millions of receptors in the olfactory bulb, a blob of nervous tissue where the brain meets the nasal passage.

Chemists have identified at least 400 aroma compounds that work on their own and with others to create complex flavours – some appearing immediately on first sniffing, others emerging only as an aftertaste. Most of these are volatiles – aromatic compounds that tend to have a low boiling point and waft away from glasses and tongues towards the olfactory bulb.

Some of these, the primary volatiles, are present in the grape. Others, the secondaries, are generated by yeast activity during fermentation. The rest, the tertiary volatiles, are formed as wine matures in barrels or bottles.

More evidence that wine-tasting is influenced by context was provided by a 2008 study from Heriot-Watt University in Edinburgh. The team found that different music could boost tasters’ wine scores by 60%. Researchers discovered that a blast of Jimi Hendrix enhanced cabernet sauvignon while Kylie Minogue went well with chardonnay.

Over the last few decades, wine scientists have begun to identify the compounds responsible for some of the distinctive aromas in wine.

The grassy, gooseberry quality of sauvignon blanc, for instance, comes from a class of chemicals called methoxypyrazines. These contain nitrogen and are byproducts of the metabolism of amino acids in the grape. Concentrations are higher in cooler climates, which is why New Zealand sauvignon blancs are often more herbaceous than Australian ones.

The flowery aroma of muscat and gewürztraminer comes from a class of alcohol compounds called monoterpenes. These include linalool – a substance also used in perfumes and insecticide – and geraniol, a pale yellow liquid that doubles up as an effective mosquito repellent and gives geranium its distinctive smell.

The spicy notes of chardonnay have been attributed to compounds called megastigmatrienones, also found in grapefruit juice.

"People underestimate how clever the olfactory system is at detecting aromas and our brain is at interpreting them," says Hutchinson.

"The olfactory system has the complexity in terms of its protein receptors to detect all the different aromas, but the brain response isn't always up to it. But I'm a believer that everyone has the same equipment and it comes down to learning how to interpret it." Within eight tastings, most people can learn to detect and name a reasonable range of aromas in wine, Hutchinson says.

Detecting and finding the right vocabulary may be within everyone's grasp. But when it comes to ranking wines, Hutchinson shares Robert Hodgson's concerns.

"There's a lot of nonsense and emperor's new clothes in the wine world," Hutchinson says. "I have had a number of wines costing hundreds of pounds that have disappointed me – and a number costing between £5 and £10 which have been absolutely surprising."

People struggle with assessing wine because the brain's interpretation of aroma and bouquet is based on far more than the chemicals found in the drink. Temperature plays a big part. Volatiles in wine are more active when wine is warmer. Serve a New World chardonnay too cold and you'll only taste the overpowering oak. Serve a red too warm and the heady boozy qualities will be overpowering.

Colour affects our perceptions too. In 2001 Frédérick Brochet of the University of Bordeaux asked 54 wine experts to test two glasses of wine – one red, one white. Using the typical language of tasters, the panel described the red as "jammy' and commented on its crushed red fruit.

The critics failed to spot that both wines were from the same bottle. The only difference was that one had been coloured red with a flavourless dye.

Other environmental factors play a role. A judge's palate is affected by what she or he had earlier, the time of day, their tiredness, their health – even the weather.

For Hutchinson and Hodgson the unpredictability means that human scores of wines are of limited value.

"It's very subjective and there's a lot of politics marring it," says Hutchinson. "People should use it as one indicator and not as an end-all. It would be a great sadness if people were only driven by what critics say."

Is there a scientific basis for the belief that red wine does not go with seafood? Researchers from Japanese drinks firm Mercian tested 64 varieties of wine with scallops, and concluded that the iron content of red wine speeded up the decay of fish, resulting in an overly ‘fishy’ taste. Photograph: Alamy

So if people cannot be relied on to judge wine, how about machines?

"In terms of replicating what a human can do we are a long way off," Hutchinson says. "The one thing we can do well, though, is a lot of amazing analytical chemistry that allows us to detect a huge range of different compounds in a glass of wine.

''We can start to have an indication of how the acidity balances with the sweetness and different levels of flavour compounds.

"But the step we haven't got to is how that raw chemical information can be crunched together and converted into something that reflects someone's emotional response. That might be something we can never achieve."

Meanwhile the blind tasting contests go on. Robert Hodgson is determined to improve the quality of judging. He has developed a test that will determine whether a judge's assessment of a blind-tasted glass in a medal competition is better than chance. The research will be presented at a conference in Cape Town this year. But the early findings are not promising.

"So far I've yet to find someone who passes," he says.

PUNGENT OVERTONES

In 2007, Richard E Quandt, a Princeton economics professor, published a paper entitled "On Wine Bullshit: Some New Software?" The study sought to describe the "unholy union" of "bullshit and bullshit artists who are impelled to comment on it", in this case wine and wine critics. Quandt compiled a "vocabulary of wine descriptors" containing 123 terms from "angular" to "violets" via other nonsense descriptions such as "fireplace" and "tannins, fine-grained".

Then, with the help of colleagues, he built an algorithm that generated wine reviews of hypothetical wines using his "vocabulary of bullshit". For instance: "Château L'Ordure Pomerol, 2004. Fine minerality, dried apricots and cedar characterise this sage-laden wine bursting with black fruit and toasty oak." He concluded that whether his reviews were "any more bullshit" than real ones was a "judgment call". Sadly, he didn't explore how long it would take a monkey to type a wine review.

http://www.guardian.co.uk/lifeandstyle/2013/jun/23/wine-tasting-junk-science-analysis

[ihpguy]

It's all Greek to me. Well for this site, not that type of Greek. Impossible to decipher. When back in the states and I go to my favorite wineshop. tell the manager, resident expert my favorite wine, how it is produced, etc. they are usually at a loss to give me a comparable bottle. In the past, whatever the recommended was never as good. They are lost normally or wrong. And ditto for the Robert Parker scores and desriptions. An okay generalization, but never spot on. So....

Last time back in the States, I just spent the funds and got me a few bottles of Chateau St. Jean Cinq Cepages, produced from the grapes from 5 different varieties of old vines at $60/bottle. My two friends with whom I stayed became addicted to the stuff. A wonderful way to get blasted chatting with old friends over great wine and some artesanal cheese. A bit different from the Spanish Rioja or 2-buck Chuck at TJ's

[RA1]

_{I completely agree with the thrust of this article. Expectation can supersede virtually all else. I have played this little game several times with folks who can afford expensive wines and others who just like to drink wine. A well chosen $12 bottle of wine was enjoyed just as much as a $100 bottle of wine when no mention of price was made.}

_{We may as well revert to something similar to Robbie the robot sampling the "rotgut" provided by the cook (Forbidden Planet) and then produced gallons and gallons of the stuff.}

_{Best regards,}

_RA1

[AdamSmith]

I had always suspected as much, but this guy's research and analysis is pretty impressive.

[TampaYankee]

I haven't read the article and I have my own experience to go on. I suspect that many people may not be able to discern the flavors and aromas in wines. Many people do not even like dry wines and can't get past that. What I do know from experience is that I have tasted the following notes in many many (grape) dry wines that I have consumed over 40+ years -- not all in the same wine by any means and not every wine had least one. Some wines can portray little more than wetness and a mysterious taste if any, but those are mostly restricted the very low priced bins usually -- not to damn all low priced wines.

choclolate

cherry

blackberry

leather

tobacco

tar

oak

vanilla

butter

pineapple

grape juice (rarely)

pear

gravel

grass

vinegar

I have had wines with various floral and fruity bouquets, as well as oak, leather, tobacco and even barnyard bouquets. I demurred on the barnyard offering. Probably just a very bad cork and mold rather than a non-fruit fortification. (yikes)

I have had others share these experiences as I often do not drink these types of wine alone but with a meal and family or friends. I'm not a wine snob as my usual fare is Paisano, a cheap delicious California jug wine that serves my everyday purposes. But in the past I have cellared as many as a dozen cases of wines at a time from around the world, more than a few rated 90+ (too often disappointing on maturity for such ratings).

I have no doubt that some cannot discern these wine qualities and that they convince themselves they do. I also expect that even so-called experts may disagree on some wines. As for me, I've been drinking wine so long and from so many sources with so many disappointments that I have few expectations upon opening a wine, especially in the last ten years, especially based on price. One wine that surprised me just recently, enough to make me shell out for half a case, which I rarely do in retirement, is:

Four Vines, Old Vine Cuvee, 2009, Zinfandel, California $12 - $14 dollars most places.

If I were still drawing a regular paycheck I would have purchased at least two cases for the cellar. I'll let those who have an opportunity to try it determine for themselves the tastes and aromas this beauty offers.

One more point, as far as telling plonk from grand cru, I do not disagree with that in general, but from the point of view that in many cases little separates the quality of wines. There are many formerly great vineyards trafficking on their reputation that just no longer deserve it. Definitely true in France and also true in California, Washington State and Australia, as some former greats have become so-so, but try to demand the money they got in the past. That is the nature of wine as a business, and distributors and many retailers aide and abet it because money is money.

Finally, I expect the same type of article could be written for coffee or vodka or myriad of other products that rely on the senses to evaluate. Those article would be just as true and just as false as I suspect this one is.

I know I said finally above but...

I don't knock studies in general and I don't know anything about this study to knock it other than I don't share it's sweeping conclusion on the face of it. However, all studies are subject to some level of corruption no matter how much care is take to not corrupt the study. It all comes down to the Heisenberg Uncertainty Principle, the very act of trying to measure something or observe a process effects the outcome producing some difference from that which takes place without measurement or observation.

Take for example these experts who differ on their analysis. Putting them under a microscope induces a psychological stress which may even produce physical stress that might alter the sensitivity of their senses. They are putting their 'rep', their knowledge and expertise, on the line publicly for people to scrutinize. Most sports and entertainment stars, even business stars usually have to have their 'head in the zone' to succeed at the levels in which they compete. If they don't have their head in it, well... Just look at women's gymnastics as an example.

As for me, I'm under no pressure when I drink wine. I have nothing on the line, no stress to send cortisol flowing through my system. I sit and drink and notice whatever my tongue and nose detect. I'm in a relaxed state. My results are unbiased by outside forces or measures. No one else told me what to expect or quizes me as to whether I pass or fail. I just taste what I taste, real or not. That leads to the question of what is reality where I will not go.

The Heisenberg Uncertainty Principle is very real. We have to live with that and do the best we can because observations and measurements are very necessary. Just be aware that interpreting results may not always be as straightforward as it seems or should be.

[RA1]

_{Pity you are not under stress when you drink wine. It can be such a good stress reliever.}

_{Best regards,}

_RA1

[TampaYankee]

_{Pity you are not under stress when you drink wine. It can be such a good stress reliever.}

_{Best regards,}

_RA1

Retirement cured that.

[RA1]

They say retirement is something you don't do until you die. However, you are obviously engaged with this site as well as other things so good for you. Personally, I fear retirement. The only way I would retire is if I can no longer pass the FAA medical requirements or have some other impairment. Not flying would be a huge negative in my life. However, there are ways around even not passing the FAA medical requirements and that mainly is to take a qualified pilot with you. He or she would be pilot in command but you can still fly the airplane. However, that takes money and not all physical or mental impairments can be over come safely even under the described circumstance. Being already crazy takes the pressure off.

From what you write I assume you prefer reds which I generally agree but a nice dry white wine can be a nice change of pace, especially during hot weather which presumably you have in adequate supply in the TPA area.

Best regards,

RA1

[wayout]

I will use a wine's point rating as a guide to try new wines. One needs something to help make a decision when faced with the myriad of choices at the wine shop. Do I always agree with the ratings, probably not but overall I think it does help me. I have considered however that perhaps a higher rating does influence my final opinion subconsciously but I really have no idea how much.

Do you suppose we should conduct a similar study for escort tastings?

[TampaYankee]

They say retirement is something you don't do until you die. However, you are obviously engaged with this site as well as other things so good for you. Personally, I fear retirement. The only way I would retire is if I can no longer pass the FAA medical requirements or have some other impairment. Not flying would be a huge negative in my life. However, there are ways around even not passing the FAA medical requirements and that mainly is to take a qualified pilot with you. He or she would be pilot in command but you can still fly the airplane. However, that takes money and not all physical or mental impairments can be over come safely even under the described circumstance. Being already crazy takes the pressure off.

From what you write I assume you prefer reds which I generally agree but a nice dry white wine can be a nice change of pace, especially during hot weather which presumably you have in adequate supply in the TPA area.

Best regards,

RA1

I drink reds almost exclusively anymore.

I always favored them by a large margin but I did drink whites much more twenty years ago. Today one is much more likely to stumble across a modestly priced decent red than white IMO, even though there are a few exceptions, almost by accident it seems. Most modest-priced whites are fit only for marinating goat and treating sink clogs, again IMO. The taste on the tongue is likened to fingernails on a chalk board -- good for waking one from a semicomatose slumber like a smack upon the face, or... for clearing the sink drain.

'My white' is aged in oak to impart hints of vanilla and a buttery, chewy style with a strong backbone and lower notes with lingering after-notes that continue to charm the tongue. Most of what is produced today is thin textured pineappley higher notes. One of the differences is the amount of time spent in oak barrels. That costs money and takes more time which also costs money. That is not to say that there are not good examples out there of the new style but be prepared to leave a limb behind at your wine store because most vintners price the better whites as if they are extremely proud of them, and frankly, they are a big gamble to the buyer. I receive no satisfaction from paying big for a bad wine, whatever the label.

[TampaYankee]

I will use a wine's point rating as a guide to try new wines. One needs something to help make a decision when faced with the myriad of choices at the wine shop. Do I always agree with the ratings, probably not but overall I think it does help me. I have considered however that perhaps a higher rating does influence my final opinion subconsciously but I really have no idea how much.

I subscribed to Wine Spectator for more than a decade back in the late 80's early 90's and found them an invaluable source of information. Not only did it inform me of yearly releases but gave me background on different countries and regions within countries a well as varietals -- taste, growing conditions and regions of best production. I scored many great values based on their recommendations.

Eventually, I got buried under all of the back editions. It also accepted more advertising so that I felt it had been discovered by 'too many' giving me the sense that its flavor had moved somewhat from judge to 'advocate' in some of their recommendations. That and the fact that their best benefit to me of revealing unknown modest-priced gems waned as prices increased in general and other information outlets also were spilling the beans on what few gems were to be found in the current market.

I also felt pretty confident in my own knowledge of the subject and my ability to seek out information on the internet as well as networking with a few friends and family to keep abreast of the better local offerings.

If I were a novice I would consult WS and other wine info outlets even with the changes I think WS has undergone over the decades These outlets are still a great source of knowledge about wine in general and new offerings presented every year. In the land of the blind, the one eyed man is king.

[AdamSmith]

With TY all the way on whites today. The only exception I would make is well rated first-growth chablis. The one white varietal in which when lucky one can find the same depth & interest as in many reds.

Alas not, however, without considerable damage to the wallet.

[lookin]

One of the differences is the amount of time spent in oak barrels. That costs money and takes more time which also costs money.

They can save their money as far as I'm concerned.

I was at a small winery last year and asked the winemaster - mistress, actually - why there were two different-colored oak barrels. She told me the darker ones were French toasted oak barrels that weren't as harsh as the lighter local oak barrels. It finally dawned on me that the harshness I picked up in some white wines was from oak. She went on to tell me that oak had become a fad in recent years and, to reduce the time it took to impart oak flavor, the Australians had gone so far as to flow the wine over a bed of oak chips. Yech! She said that, thankfully, the fad was starting to die out and that she was glad the emphasis was swinging back to the grapes.

I've found that most places will be able to make a recommendation for low-oak whites, and I always ask.

[TampaYankee]

I can't speak to this particular lady, only my experience including what I see on the shelves today. However, I do take exception with her remark that oak had become a fad in recent years. That is untrue. Oak has been used for centuries in France and was adopted in the California and Washington State wineries on start-up. However, oak in white wines is used predominantly in Chardonnay. So it was in France, so it was everywhere else. Most other whites see only light oak or none, eg. Chablis & Sauvignon Blanc. Maybe some newer wineries are trying to oak-up other white varietals recently, I do not know as I do not buy whites except in a blue moon on a good recommendation. Oak is not a recent fad even if oak chips are.

In my experience oak seems to mellow and round the high notes harshness and give it a vanilla note and a lower note tannin backbone which is what you probably do not like.

As for wine over a bed of oak chips, I first encountered this maybe 20 years ago or so. I was not a fan at all. Very strange taste and bouquet compared to what I was used to. I do not believe that wine in stainless steel over oak chips rushed to fruition in three months can compare favorably with wine spending one or two years in a breathing oak barrel left on its own in a cool dry cave or the equivalent.

Many people do like the less oaky whites. Those I knew were not fans of most reds, generally. Whites probably outsell reds today as many females are drinking whites casually at home or when out, while their male counterparts probably imbibe beer.

[RA1]

Some of this conversation reminds me of the Jack Daniel's ads which (paraphrased) said, charcoal mellowed, drop by drop. Sure, many gallons per minute are made up of drops.

There are at least 3 very large liquor stores in MEM that specialize in wine. I have had very good luck just asking in one of them for advice and then telling them how much I wish to spend.

I have had part of a bottle of $300 wine (which I did not pay for) and thought it good but not $280 better than some $20 bottles enjoyed at other times.

You might be interested to know that ( a few years ago) a, since departed, local restaurateur and bon vivant named John Grisanti paid the most for a single bottle of wine and then followed it up the next year by exceeding the first price for a Jeroboam. I do not now recall the price but one was at least 20 something thousand and the next 30 something thousand. He sold the contents by the small glass and donated all proceeds to charity. Both of these bottles were pre blight. I am sure the story can be researched using Google.

Best regards,

RA1

[AdamSmith]

There are at least 3 very large liquor stores in MEM that specialize in wine. I have had very good luck just asking in one of them for advice and then telling them how much I wish to spend.

That is how I have usually bought wine, if not ordering/pre-ordering a case. It has the benefit of leading you to something you can purchase right then and there.

I got to know several good wine shops in & around Boston that were owned & run by a knowledgeable buyer, had helpful (and honest!) staff, and to boot ran 'cast-off' bins where they would put out, on mark-down, pretty good (sometimes quite good) bottles that were the last of a lot that they wanted to be rid of. (One of the best: http://www.thewineandcheesecask.com/)