After Yelp referral leads to $2K moving company ripoff, thoughts on "genuine reviews"

http://www.matthewpetty.com/ Matthew Petty

It’s good to have many people reviewing places, but it’s not scaleable to far. I’ve seen restaurants with 1000 reviews, and an average score of 3/5. What does that tell you?

ChicagoD

That the restaurant is mediocre. Which tells me I might or might not like it. Which I knew before. Dammit.
- Antinous / Moderator
  
  It took me a while to figure out ratings at Amazon Marketplace. At first glance, 94% positive seems like a good rating, but after a few transactions, I realized that six out of every hundred complaints is a pretty terrible record for a service that consists of mailing the correct product in a timely fashion. You really have to consider the nature of the business when you judge the rating.
  - http://www.kmoser.com kmoser
    
    You forgot to take into account the biggest variable: the USPS.
    - Antinous / Moderator
      
      That’s easy to factor into my rating. I just look at the date stamp on the package. I expect an order to be mailed within two business days of being placed. If the post office takes three weeks to deliver it, I don’t blame the seller. But I have gotten packages that were mailed ten days after purchase, and they got bottom reviews.
      
      You have to look at the review climate; if there are very many seller’s with 98% and 99% ratings, a 94% is a terrible rating.
  - That_Anonymous_Coward
    
    Obligatory… and you really do need to hover over to get the sad part of the joke…
    https://xkcd.com/325/
    You can do this one in every 30 times and still have 97% positive feedback.
  - LogrusZed
    
    Another issue is the absolute inability to offer a “zero star” rating in cases where it is warranted such as: counterfeit merchandise (very common with consumer electronics), damaged/unusable goods in which the seller ceases contact with the consumer thus avoiding replacing, etc.
    - Antinous / Moderator
      
      Yeah, that’s kind of messed up. On the other hand, the system won’t accept a star rating unless you complete the essay question, too. I usually end up putting a period in that field since I don’t really have much to say, having rated the timeliness and appropriateness of the product via the star system.
  - LikesTurtles
    
    Unless I’m missing something a 94% rating doesn’t mean that 6% of the customers were unhappy, it’s 94% of the best possible rating. Marketplace transactions are rated on a five star system so a bunch of four star ratings would create a less than 100% rating even though four star ratings aren’t something I would worry about. Is this not how the system works?
    - Antinous / Moderator
      
      Yes. But where the seller’s sole obligation is to mail the correct product in a timely fashion, anything less than a five-star rating is a red flag. It’s not like a restaurant where you could get a great appetizer and a crappy dessert, be seated quickly but have an inattentive waiter. That’s why you can’t translate ratings between different services.
novium

That it’s probably a mediocre restaurant? ;) Actually, that’s just my experience in the city I live in. Perhaps people living/visiting here just have bad taste- I’ve noticed that great restaurants here tend to have <20 reviews total and the least-impressive-while-not-necessarily-bad restaurants have reviews in the hundreds. (And many of the reviews will be ecstatic.)

It's become my own mental sorting system: I look for restaurants with 15 or so reviews that are fairly good but not stellar, especially those that praise the food in general, but complain about minor, subjective things like: "Fruit plate had too many strawberries." The people that write those kind of reviews always seem to mark them down a star or two for that sort of thing, but it's obvious the food is good….where as the hordes tend to be all, "BEST BREAKFAST EVER! THIS RESTAURANT MAKES VACATIONING HERE WORTH IT!!!111"
- http://www.matthewpetty.com/ Matthew Petty
  
  You’re right. But often I’ve seen a score dragged down by multiple super-negative reviews, written by angry people because one little thing went wrong. For example, someone was left standing for too long before being seated, and suddenly the food is “disgusting”.
  
  xkcd on online app reviews: http://xkcd.com/937/
- LikesTurtles
  
  A few years ago I was really into Yelp. Had Elite status and was one of the top reviewers in my city. But I stopped when I noticed that I was writing stuff like “too many strawberries”. My desire to write what I felt was a complete and balanced review had led me to take a critical eye to everything. I wasn’t enjoying my meals like I use to because instead of just brushing off minor things, I made a mental note of them to use later in my review. Even worse was that I noticed that many of my friends on Yelp were doing the same thing but extended this behavior into their personal life. No wonder the discussion boards can get so nasty. The Elite members tended to break up into constantly shifting cliques depending on who had last stabbed who in the back. It was like high school all over again. This was particularly disturbing to see in people who I knew before Yelp because they had never acted that way before.
  
  While it was fun at first, ultimately Yelp was poisoning my soul (or maybe it would be better to say Yelp was helping me to poison my own soul). Once I stopped writing reviews, just about everything in life seemed to get better. Over time most of my friends who I introduced to Yelp stopped writing reviews too. But I still hear from time to time about the non-stop social chaos that is the clique of the frequent reviewers. Glad to no longer be part of it.
  - Martijn Vos
    
    It seems that critics in general are often not happy people. So what happens when sites like yelp turn everybody into critic? Suddenly everybody is unhappy.
    - novium
      
      I don’t know. I think things can be taken too far, but I also think that there’s room for people who say, “well, I guess there were too many strawberries,” because they’re trying to say something about what the experience was actually like and not read like advertising copy. Plus, to a certain extent, I think a little bit of amateur criticism can increase enjoyment, because then you can really appreciate skill and difference and novelty. And I think this applies to many things- food, wine, books, music, television, whatever. I once met someone for whom hershey’s chocolate and delicious high-quality dark chocolate were nearly indistinguishable, except for the sweetness…and then benefit of that, I guess, is that they’d enjoy it all equally, but the downside was they missed the sublime pleasure of the stunningly good chocolate.
  - bkad
    
    Possibly, if you care about reviews and dining, you were also selecting restaurants that were better than average to begin with. You never see 1s and 2s, so one’s sense of ‘average’ gets out of wack.
- bkad
  
  I do the same thing, I read the negative reviews and try to determine whether the reviewer was being unfair (complaining about speed of service in tiny places the busiest night of the weak, complaining about failure of servers to act as the diner’s personal slave) or subjective (too many strawberries). Similarly on Amazon, you will sometimes see complaints (paraphrased) which say, “You sent me exactly what I ordered but it wasn’t what I wanted!”). If that’s the worst anyone can say about you, you’re an awesome business.

http://profile.yahoo.com/VPMCBLG2HDEONJA7LLABL4EFSI AshleyC

Companies can also pay yelp extra to push negative reviews to the back. :/

http://www.hisnameistimmy.com Tim in SF

This is a persistent rumor for which I’ve yet to see any evidence.
- That_Anonymous_Coward
  
  There are several lawsuits and news stories about it…
  http://www.eastbayexpress.com/ebx/yelp-and-the-business-of-extortion-20/Content?oid=1176635
  
  I bet you think the BBB rating is sacred too… they got caught selling good ratings.
- http://jimbeach.net mindfu
  
  I have seen this exact thing happen to a friend of mine who owns a restaurant in Brooklyn. She has almost nothing but 4 and 5 stars reviews, but most of the negative ones are above the fold. She called Yelp and asked to have them at least filtered or shown chronologically, and was told point blank that the only way that was possible was if she purchased advertising.
  
  When instead she demanded not to be listed at all in Yelp, she was told that couldn’t happen either.
  
  Yelp is a useful service, but is twisted more than a little towards the shitty by the capitalist model they’re pursuing.

NoOneSpecific

And what is YELP’s liability as the company who’s business model is basically recommending or vouching for said moving company? Are they liable for what is written on their site? Should they be held liable for not properly vetting these companies?

If you say ‘No” now, realize that it is only a matter of time until someone launches a Class Action against a ‘Yelp’ or an ‘Air BnB’ and the whole model changes.

EDIT:
YELP is basically positioning themselves as an introduction service. More than just advertising media like the Yellow Pages as they take money not only for the ad but also to down rank negative reviews to the bottom of the list. With that one component they are now in a position to be liable should the company defraud you as they are in essence vouching for said company.

http://www.nathanhornby.com/ Nathan Hornby

They’re not reviews provided by Yelp, they just house them.

Suing Yelp for a bad experience would be like suing the yellow pages because a plumber ripped you off.

Plus there’ll be disclaimers. There are always disclaimers.
- Martijn Vos
  
  But what if Yelp hides negative reviews? Then they’re suddenly intentionally misleading. Then it’s more than simply housing other people’s reviews.
- Marc45
  
  Tell that to MegaUpload…
  
  But according to the original complaint, Yelp actually engages in moderating reviews. IANAL, but this would seem they have some liability there.
  
  Let’s face it, companies like Yelp aren’t here to assist the consumer. They’re here to assist the businesses that are reviewed. If they filter out “bad” reviews, then businesses are happy and a disgruntled consumer is more or less irrelevant.
  
  Personally I never look at Yelp as truly useful since the reviews are suspect.
  - Bad Juju
    
    Indeed, no trust for Yelp here.
    Speaking of plumbers, all but 3 of the reviews for the closest one were all entered on 9/11/2011. Obvious shill is obvious?
    Oh, two of the others were on 4/20. Happy Birthday!
    
    If these reviews were Craigslist ads, they’d have been flagged into oblivion, but that’s just not good for business!
- LogrusZed
  
  That isn’t an accurate comparison.
  
  There is no pretense that the Yellow Pages large ads are a recommendation; they are absolutely transparent in the fact that ad placement and size is specifically related to the alphabet and the willingness of the merchant to pay for the size of the advertising.
  
  Yelp! is in the position of advertising themselves specifically in a capacity as a democratic ratings system where the user of a service provides the basis for a score that potential users can do their own filtering; but in practice they are doing their own filtering and de-facto colluding with the merchants in a manner which has the effect of gaming the system to the disadvantage of the consumer.
  
  I don’t know enough about the Yelp! business model to know if there is motive for intentional collusion or if it it merely a result of laziness and/or incompetence. The result, however, is the same for the potential consumer: They are fed inaccurate information.

grimc

I feel for the guy, but the solution isn’t an algorithm–it’s digging a little deeper than a couple of Yelp reviews. You’ve got to have better judgment than to trust a company that has 2 5-star reviews with all of your belongings.

novium

His solution wasn’t an algorithm. It was identity verification. But he has a good point that their current methodology is a bad one:, and I have to agree. I’ve looked at plenty of listings there where all of the bad (and probably more honest) reviews were hidden, but all of the good (and suspicious) reviews were visible. Even recently, with that terrible restaurant in New York that got mentioned here.
- chf64
  
  The solution is to not use a sole source when making a decision about a product or service. Always use multiple sources.
  
  I apply this principle to everything I reasonably can in life. Gather information from multiple sources and put together your own opinion.

MaximumOvertroll

I’ve got a brilliant “smarter algorithm” for yelp, it’s this brand new thing called “captcha”.
This is why I dont use services that let you pay for good reviews. Hey, I’ve got an even beter idea! DON’T LET PEOPLE PAY FOR REVIEWS!

http://bhtooefr.org/ Eric Rucker

Captcha won’t help when you have “reputation managers” gaming the system – there’s enough money to pay for a real person to decode the captcha.

novium

I really hate the proposed solution- using identify verification like the credit companies do, e.g. “Which of these four addresses have you lived at”. Because yeah, that’s exactly what I want to do when creating an account on every random website out there: not only tie it to my real identity, but have it pull sensitive information on me. It’s bad enough with the credit companies, but there I’ve got no real choice and it’s slightly better than the alternative of even easier identity theft. But for reviews of businesses? Really? A better solution isn’t just I don’t know, using the algorithm he originally proposed, or even just crowd sourcing it? Haven’t studies shown that people are pretty good at recognizing fake reviews? Why not just have it so that you’ve got to validate some random review before you can post one? (Or get points for doing so or something).

ChicagoD

Well, honestly, if you don’t have any friends or acquaintances who have moved with a moving company and can’t afford a national moving company, looking up reviews seems to me like a reasonable step. I can’t really fault the guy for that.

http://www.hisnameistimmy.com Tim in SF

I agree that making the Filtered Comments link gray, in parenthesis, and at the bottom of the page are all BAD UI decisions.

Still, I generally do not want to read Yelp reviews from people who are not regular Yelp reviewers.

http://boingboing.net/ Rob Beschizza

1. Don’t use Yelp for anything more expensive than a meal.

2. Don’t use professional movers at all. Most of them are crooks. Instead, use commercial freight services that are willing to deal with individuals, such as ABF.

https://me.yahoo.com/ramses0#576cd Robert

ABF U-Pack was great for me compared to U-Haul’s U-Box. They already have a network of trucks, so their cost-per-mile is low, and then they basically just tack on +$50-100 on both ends to pay for somebody to drive the last mile(s) to your place.

I had a great, personable experience with them and a fairly terrible experience with U-Haul (which doesn’t have that driving network or a strong culture of customer-interaction).

Sorry U-Haul! Maybe next time you’ll call to let me know you can’t keep your promised / requested “sometime before noon drop-off-the-box” in advance of me driving from Dallas to Houston in order to receive it from you.
http://jimbeach.net mindfu

I personally had a great experience with a website called Uship. It’s like ebay for shipping – you put in where you’re moving to and how much stuff, and them moving companies bid for the right to move your stuff. You can see the company’s reviews and their percentage positive rating. No company knows who you actually are until you accept their bid.

Yelp is not designed to offer the kind of specialized service for movers, IMHO – as well as many other services, no doubt.

http://profile.yahoo.com/YQVGZSB6QUK5HKA6IUP6VKAU34 MichaelB

I had an identical experience with looking up a company when doing a home remodel — Yelp hid the negative review that exactly matched my horrifying experience, and I didn’t find it until afterwards.

tmdpny

Honestly, for such a big decision, you should always always check the BBB before you put money down. They check and verify and find resolutions for issues, and objective and can always be trusted. It’s one thing to wonder where you are eating tonight, it’s quite another to hand over ALL of your material possessions to.

johnjupiter

uhm. read the filtered reviews. THERE’S A SOLUTION!

LogrusZed

Consumers, in any medium, are inundated with fine-print or murky TOS agreements to the point where nobody reads them all, this goes for the minimum of warnings businesses provide to indemnify themselves from actions. If you claim to be as cautions as you’re admonishing others for failing to be you’re a damn liar or the kind of psychotic that has notebooks filled in the manner of Kevin Spacey’s character in Seven and possibly thousands of labelled jars of your own excrement in your home.

Even the link for filtered reviews is shaded in low-contrast to the background, unless you use a text modifying plugin for your browser then you’re unlikely to even notice that they filter reviews.

fuzzyfuzzyfungus

Honestly, in the case of Yelp and their history of helpfully modifying the reviews of those businesses willing to pay up, I’m not sure that this is a design problem. It’s a perverse incentives problem that happens to manifest as a set of design decisions.

http://jimbeach.net mindfu

Well put.

http://www.facebook.com/profile.php?id=8834925 Chauncey Scott

Whenever the ability is there. I sort reviews, to read the negative ones first. Then I determine, if I were to be in a situation, would I too, see it as a negative.

techdeviant

I’ve successfully used angieslist to find good companies, including a moving company that worked just fine for me. I don’t know if the right solution is to charge users for good information though.

foobar

Giving an entity like Yelp access to financial/credit information is just begging for fraud.

Dawn Grobe

I have the same kind of complaint, albeit on a smaller financial scale, with sites like Amazon and iTunes. I tend to read the most critical reviews first to see if they are complaints against the third-party retailer, user error (e.g. didn’t read the description), or an actual problem with the product. The other thing I look for is the 5-star “I haven’t used it yet, but I am so excited to have ordered it!!!!1!” reviews. All of these factor into the overall score of the product regardless of the users actual interaction with the product because they are only allowed to rate the item on one scale.

Of course, I also hate audiobook reviews that say “This book was really great.” I can read an Amazon review to see if people liked the story, I want to know if I actually want to listen to the reader or not.

http://burntheflag.ca Jardine

Woah, woah, woah. There are people you can pay to move your shit for you? Then why do I keep having to carry my relatives’ shit around?

DisqThis

Maybe your relatives will change their mind when you get more negative reviews.

technosean

Yelp is completely useless. Shady practices. Paid-for suppression of negative reviews and escalation of positive ones. If you read Yelp you’d think every place was a 4-star restaurant! I can’t possibly take Yelp seriously.

Sigh. I do wish there was something like Yelp but not bought by the places reviewed.

penguinchris

I find Yelp to be very useful – but you have to use it carefully, just like any online review system. When I lived in SoCal I used it to find someplace new to eat at least once a week. All my favorite regular places there are places I found via Yelp, too, but I tried new things as often as possible and Yelp is currently the best resource to help with that.

When I visit SF, NYC, or anyplace like that I always check Yelp for good places to eat – it’s especially important to do a bit of research if you’re traveling. I’ve never been outright burned by someplace I found via Yelp, though of course I’ve been mildly disappointed (usually just because of personal tastes).

I know full well all the shenanigans that go on behind the scenes, but as a regular user I also know that it’s easy to figure out what’s going on – unfortunately you can’t always take everything at face value.

I stopped providing my own reviews for a variety of reasons, such as not wanting to provide value (my reviews are good quality) to a company that does shady, nigh-unethical stuff. But as a resource in its current state it’s quite valuable.

I did find an excellent place to get a hair cut via Yelp, but I probably wouldn’t trust their reviews to help me find anything else besides food places.

Andy

Yelp is not a neutral party, so I would never trust reviews from them. They take money from places that are reviewed on their site. Or at least they try. Look at their default sort, “Yelp Sort”. WTF is that?? It’s where they decide who’s reviews show up first on the page.

LikesTurtles

I don’t know what Yelp does internally but business owners certainly can pay reviewers for positive ratings. Back when I was Elite restaurants would give us freebies which no doubt influenced our ratings. This was especially true when we’d publicly discuss meeting up to eat and drink at a particular bar or restaurant. Sometimes they’d even have a table ready for us without any of us even calling ahead. For the business it paid to monitor the talk threads and events pages to know when we were coming. No doubt we also received extra attentive service because they knew we would later write reviews. A couple of plates of free appetizers and a drink special that mysteriously was only offered to us would equal a dozen or so positive reviews. Not much Yelp can do to guard against that. They’re certainly not going to punish Elite members who accept freebies since they’re the ones providing the majority of the site’s content.

BarBarSeven

I used to post on Yelp fairly regularly. Focusing mainly on local shops and such. It was a useful resource at one point. But then I started to notice some very obviously “shilly” posts in reviews of local restaurants

It was quite bad and quite obvious since many of the places had barely opened for a day and suddenly *BOOM* 1/2 a dozen glowing reviews. And they were clearly too specific and perfect to be just a random first day customer.

Deleted my account on Yelp and basically only use it–if I ever do–like the Yellow Pages: just for the bare bone basics. The heuristics needed to create a clean and trustworthy open review system doesn’t exist yet.