Applying financial analyst John Moody's rating system to wine show judging

Reprinted from Glug website - the recommended place to buy wine on the web
Thursday, 1st May, 2014  - David Farmer 

John Moody age 88 in 1956

Can you make a living out of the salmon that John West rejects? I think so as while I know nothing about grading salmon I find I can discover and buy plenty of interesting wines that never make the grade in the show judging.

I have a high opinion of scientists as they place numbers on things which can be duplicated and because of that their theories and opinions make sense. As you move away from measurement by instruments and enter fields where experience and 'feeling' take over from hard numbers I get increasingly nervous.

Tasting and grading wines seems pretty simple. Presented with a long line of wines the idea is you pull forward the good wines and push back the poor ones. Then you further sub-divide the good ones. If required you then give a number to the good wines based on the system being used at that tasting. In essence this is how grading wines has developed over the last 100 years.

I developed a cautionary approach very early in my career. The light went on a great number of times and I vividly remember one moment at the National Wine Show in Canberra in the early 1980s. I was closely comparing wines in a Chardonnay class with the judging results and kept coming back to the number 3 wine which did not even rate a bronze but was my best. The wine was from the Neudorf winery, Nelson, New Zealand, a new region making wines with unfamiliar tastes.

As any winemaker knows entering shows is a lottery and being awarded no medals in Brisbane does not mean you will not do well in Adelaide. I was having a coffee with a local winemaker in Tanunda the other day and he related that at the local Barossa Show he was awarded zilch but a few weeks later and 100 kilometres away at the Adelaide Wine Show the same wines collected three gold medals.

Even so while we accept that well trained professional will make mistakes and lots of them, on the whole a group of judges should be able to indicate the better wines. It helps if the judges are given some clues such as the vintage or the variety and wines are generally grouped such that like goes with like. Without these hints the difficulty barrier is raised to high.

So a judging is seen as the best result on the day, by dedicated experienced professionals, and you must expect the results will vary between tastings.

Tastings of this nature are what I call flat tastings as they are one dimensional. Like many others I have wondered if a better method of grading is possible, one perhaps bringing in other dimensions about a wine.

This issue needs to be debated as no matter how you view judging, the results of numerous statistical studies now question whether results can be duplicated and indeed how expert are the experts.

Searching for a Three Dimensional Judging System

Here are some factors, all well known, which cause concern and might be considered in a more complex judging system which expands the ratings of a few judges. I have added a short note to clarify what is suggested though many of these points could be expanded into an essay.

1. Judging the judges. I came across this quote the other day; "A panel of 13 expert wine judges started to swirl, sniff, sip..", which tells us it is considered some tasters are rated as experts judges while others presumably have the lower rating of judges while the rest of us are amateurs or worse.

2. Comparing like with unlike wines. When appraising different styles together it is incredibly hard to have a just result. Who can say the Champion wine of a show is an old port or a young chardonnay.

3. Comparing wines from different regions or countries. Should wine from Lebanon be judged against wine from France?

4. Comparing different varieties. Can you compare a tarrango with a cabernet?

5. What is the tasting trying to achieve. Is the tasting narrowly defined or does it judge the wines of the world.

6. Comparing the ages of the wines.

7. Comparing the vintages of wines. Some vintages are better than others and if a wine from a poor vintage is judged as better than that of a greater vintage should we be concerned.

8. Should you compare wines from regions with little history with those that have a long history? In my experience it takes a long time for wines from new regions to reach a satisfactory level.

9. Does the wine already have a medal history?

10. Is the wine from a family or lineage with a long history of making great wines?

11. Does the wine have ability to age?

12. The price of the wine. This is a very contentious issue with no easy answer. Alas it must be considered as if the public are constantly told that experts cannot tell the difference between cheap and expensive wines, which is what they are now regularly told, there will be consequences.

The price the wine is sold for must be considered an important clue to wine quality though this is such an important topic it is best covered in separate articles.

13. The wine price also intermingles with the investment grade of the wine and can it be readily turned into cash.This relates back to the provenance of the wine

14. The ability of the winemaker or respective entity of the maker. Here I ponder whether a Penfolds or Guigal wine should have a rating added to the final score i.e. the issue of provenance.

15. The fashions of judging. Certain styles may be favoured which down-grades others. To think judges are unaffected by swings of fashion and practice their craft at a level floating above market swings is to not consider the problems we have in making decisions.

16. Should judges only judge what they know? I believe we all develop easily a cellar palate no matter how hard we try not to. Thus I am not a believer that all judges are equal and can be called up to tackle any wine style without fear or favour. To ask judges from Central Otago to judge Barossa wines asks for trouble.

Examples of Three Dimensional Judging

The Royal Adelaide Show has a Trophy called; the Gramp, Hardy, Hill-Smith Prize for Outstanding Wine of Provenance which was created in 2009, to recognise wines that reflect their region, lineage and longevity.

Another judging system is the Langton's Classification of Australian wine which links a lengthy history of excellence with the interests of consumers in buying the wine.

A further example is the interesting innovation of an eastern states show, from I recall the Southern Highlands, where wines in one class were from vineyards above a certain height, I think above 500 metres.

It is true that a competent judge is meant to take all of these factors into account and no doubt a good judge after a long apprenticeship can and does consider many complex factors.

Looking for a Better Way

Questions such as those listed from 1-16 illustrate some of the problems while the examples above offer imaginative ways forward. There may be a better way to explain to customers the grading of wines by moving beyond the current flat one dimensional way. This in turn made me ask did a better system of grading already exist, perhaps in education, business or science, which could be applied to wine.

After a bit of searching I stumbled across the inventiveness of John Moody who in 1900 developed a rating system for financial paper. I found his ideas had a three dimensional character to grading which offered, if not solutions for wine judging, a display of technique which could perhaps be adapted.

The Thinking of John Moody and Judging

Moody tackled the task of rating the likelihood of a borrower paying the agreed interest and the principle back on a loan. This is a grading system with money on the table and the judgements lead to real gains or losses.

Moody divides the ability of an entity to repay debt with a simple ABC rating but added lower case letters and numbers for the many sub-divisions needed to highlight variants to the basic risk.

A good example is classifying the difference in the likelihood of repayment of short term versus long term obligations.

Moody's system evolved into this grading: 

Aaa Aa1 Aa2 Aa3 A1 A2 A3

Baa1 Baa2 Baa3 Ba1 Ba2 Ba3 B1 B2 B3

Caa1 Caa2 Caa3 Ca C

As I studied the Moody system I found it offered many parallels to wine judging and I list some of the striking features. Moody offers hundreds of indicators which he used in his rating system and below I offer seven to illustrate the complexity of his rating system.

1. Short term versus long term debt. This reminds me of the ability of a wine to age.

2. The rating of the issuer of the debt. Some issuers of debt have a poor record and I see parallels in the divergence of winemaker's abilities.

3. A sovereign state has a different ability to pay to municipals. As the classified growths and other Bordeaux's differ should we have a base rating of some regions over others?

4. The investment manager rating of the advisor to the offer. This of course points at the quality of judges and asks if they should be graded.

5. The rating of the country which considers its past history of servicing debt and outstanding debt levels. This is harder but all things being equal I would sooner cellar Australian wine than Argentinean, at this moment, on a risk reward assessment.

6. The corporate rating. This goes right to the heart of grading as a Penfolds wine is safer to buy and cellar than other companies no matter how they have been judged.

7. A speculation grade liquidity rating which is a punt on a new entity with no history. I particularly like this idea as it asks should we invest in the new region or the new wine maker.

I had these thoughts in my mind when I opened the Winestate September/October, 2013 edition and turned to 'The World's Greatest Syrah and Shiraz Challenge' with 582 wines tasted.

To say the result puzzled and disturbed me is an understatement and I will explain why in another posting - Judging the methods of a wine magazine's wine judging.