Lies, damned lies and statistics. There could hardly be a better – or rather worse – case than the survey results put out today by Tripadvisor.com. These results completely fail to substantiate the widely reported headline claim that “London' s Cuisine [is] Worst in Europe”.
The basis for the headline is the following worst-food ‘results’:
1. London (10%) 2. Moscow (7%) 3. Warsaw (6%)
However, equally misleadingly, the headline could have been “London’s cuisine eclipsed only by Paris”. The best-food results are:
1. Paris (18%) 2. London (17%) 3. Rome (13%)
Thus, it would seem that more people surveyed have a positive view of London cuisine than a negative one, which is completely at odds with the screaming headline.
Tripadvisor has created what is probably the world’s leading online travel reference site on the model of balancing reports – positive and negative – on particular establishments.
As an organisation not so different from Tripadvisor in aims (if rather smaller in scale), it seems to us that seeking cheap headlines on the basis of a self-evidently unbalanced and incomplete assessment of survey results is a very poor advertisement indeed for the consumer-research-driven business model.
PS (26 June) How funny, in the light of the last comment, to read the following commentary on the statistical methodology of our old friends, Zagat Survey, in a Wall Street Journal blog.
As the blog so rightly points out, sheer ‘head count’ polls – such as the coffee house results they’re currently making a big fuss about – tell you nothing more than that big companies (such as Starbucks) have more fans than smaller companies. Of course they do. They have more foes too, but if you only look at the plus side of the equation you’ll invariably find that the big guys are ‘better’.
Zagat’s guidebooks, we understand, are written on the averaging basis – looking at the views of fans and foes – so one can only guess as to why they’ve suddenly decided to base so much on the indefensible methodology of just totting up the fans. Surely couldn’t have something to with the size of Starbucks’ advertising budget, and all those spin-off Zagat name checks? Perish the thought.
For completeness, we should note that Harden’s does publish the results of various crude nominations polls every year – they are set out on pages 11 and 12 of the London guide. There is a certain interest in knowing what places people are talking about. But we’ve never based our restaurant appraisals on them, and never based PR on the results unless the ‘simple’ analysis is just a shorthand way of expressing a deeper truth. For example, we’ve often said that Gordon Ramsay topped a particular year’s best chef poll… but we only did this in the context that the more subtle and detailed analysis of the survey also suggested that he really was best chef too.
The whole point of being an ‘expert’ statistically-based publisher is that the public should be able to trust you only to make (and publicise) survey interpretations that can reasonably be sustained from a full and careful analysis of the data.
Follow hardensbites Follow hardensparty
Become a Harden's fan
