Act 1 – An Inciting Incident
A lucky fly on the wall of a Mini Cooper recently traveled back with 4 passengers from State Bird Provisions, a Michelin starred restaurant in San Francisco. The foursome had spent the evening there celebrating Kissy Suzuki’s 56th birthday. Flymaster Fly™ reported back on the conversation –
Kissy – The dim sum there was great – I loved it! The crowd was fun too!
Dr. No – What a hellhole. I’d sooner call it ‘Bird Flu’. Thank goodness all of you ate my dishes. I’m glad you all enjoyed it, but I’m having some kabab as soon as I get home.
Cash-Strapped Khashoggi (CSK) – It was alright. It doesn’t stand up to the places I used to frequent with my crew in Hong Kong, but I suppose it was fine. Nothing amazing, though.
Sonja Petrovskaya – Ambiance was fun, but the servers were too familiar. I’d have to agree with CSK – food was just alright. Dim sum dumbed down. Fun place overall though.
What’s wrong with this picture? This crowd of four comes from the same socio-economic background (yes, even CSK at this point), typically shares similar tastes, and this place has the Michelin stamp of approval, after all. If they all posted reviews to Yelp, who should I believe? Worse yet, who do I trust when it’s not just a crowd of four, but a crowd of 1,490 and counting, and I don’t even know anything about those people?
Act II – The Conflict
This brings us to the problem of relevance on Yelp, or the lack thereof, when it comes to reviews of experiences that are highly subjective, such as restaurants.
The fact of the matter is, Yelp provides no dimension to capture this when reviews are submitted. As a consumer of reviews, I’m therefore left to blindly guess the relevance of a given review, or even the relevance of the average star rating from1,500 people vis-à-vis my own taste. The point people make that about the “law of large numbers” – that an average over so many people should be indicative – doesn’t hold, because I don’t know who that group is, or what standard they represent. “What is their average an average of?”
To summarize, these are the reasons why a Yelp restaurant review fails to capture and communicate relevance, and also why their level of accuracy can’t be trusted:
- (1) No measure of the reviewer’s taste(s) relative to my own (ie I don’t know who these people are + only 1% of Yelpers post reviews – there is already some sort of inherent bias in who falls under the character of “the Yelp reviewer”)
- (2) No measure of context of the reviewer’s visit to the restaurant
- (3) No segmentation of review criteria (ie. Cuisine, setting, service, etc are not broken out. A lump sum star score average doesn’t point to which of these factors may have influenced the rating more or less)
- (4) Social influence bias (“Research at MIT suggests [online reviews] are systematically biased. When it comes to online ratings, our herd instincts combine with our susceptibility to positive “social influence.” When we see other people have enjoyed a hotel or restaurant — and rewarded them with a high online rating — this can cause us to feel the same positive feelings and to likewise provide a similarly high rating.” On Yelp: 21.9% are 3.5 stars, 61.9% are 4 stars, and 16.2% are 4.5 stars – also demonstrating a herding effect to “middle” ratings)
- (5) Fraudulent reviews
- (6) Yelp filters reviews, but doesn’t do the best job filtering
In Act III, we focus on potential solutions for those 3 first points impacting “relevance” of reviews.
Act III – Dénouement
For possible solutions to the problem of relevance, we can tear a page out of “Urban Daddy’s” playbook – yes, you read correctly, “Urban Daddy”.
If Yelp were to incorporate some simple categories of criteria such as these when reviews are posted, this would allow users to then look up a type of review with a higher level of granularity. I, as a consumer, could then have these categories to filter down to reviews more relevant to my particular context and taste(s). Additionally, having a given review broken out into simple categories of “Cuisine”, “Ambiance”, “Noise level”, etc, “Overall” would further enhance relevance for readers.
This type of solution could also help increase relevance of other types of more subjective experiences/products that are reviewed – hotels, tours, etc…
WORD COUNT: 742