Justin Vincent chose a moving company based on positive Yelp reviews. Things did not work out well. "I spent 40 days without any furniture and quite a few of my belongings have been misplaced – forever." Turns out negative reviews were there, but suppressed. Could smarter design have prevented this? (via @blam)

55 Responses to “After Yelp referral leads to $2K moving company ripoff, thoughts on "genuine reviews"”

  1. It’s good to have many people reviewing places, but it’s not scaleable to far. I’ve seen restaurants with 1000 reviews, and an average score of 3/5. What does that tell you?

    • ChicagoD says:

      That the restaurant is mediocre. Which tells me I might or might not like it. Which I knew before. Dammit.

      • Antinous / Moderator says:

        It took me a while to figure out ratings at Amazon Marketplace.  At first glance, 94% positive seems like a good rating, but after a few transactions, I realized that six out of every hundred complaints is a pretty terrible record for a service that consists of mailing the correct product in a timely fashion.  You really have to consider the nature of the business when you judge the rating.

        • kmoser says:

          You forgot to take into account the biggest variable: the USPS.

          • Antinous / Moderator says:

            That’s easy to factor into my rating.  I just look at the date stamp on the package.  I expect an order to be mailed within two business days of being placed.  If the post office takes three weeks to deliver it, I don’t blame the seller.  But I have gotten packages that were mailed ten days after purchase, and they got bottom reviews.

            You have to look at the review climate; if there are very many seller’s with 98% and 99% ratings, a 94% is a terrible rating.

        • That_Anonymous_Coward says:

           Obligatory… and you really do need to hover over to get the sad part of the joke…
          https://xkcd.com/325/
          You can do this one in every 30 times and still have 97% positive feedback.

        • LogrusZed says:

           Another issue is the absolute inability to offer a “zero star” rating in cases where it is warranted such as: counterfeit merchandise (very common with consumer electronics), damaged/unusable goods in which the seller ceases contact with the consumer thus avoiding replacing, etc.

          • Antinous / Moderator says:

            Yeah, that’s kind of messed up. On the other hand, the system won’t accept a star rating unless you complete the essay question, too. I usually end up putting a period in that field since I don’t really have much to say, having rated the timeliness and appropriateness of the product via the star system.

        • LikesTurtles says:

          Unless I’m missing something a 94% rating doesn’t mean that 6% of the customers were unhappy, it’s 94% of the best possible rating. Marketplace transactions are rated on a five star system so a bunch of four star ratings would create a less than 100% rating even though four star ratings aren’t something I would worry about. Is this not how the system works?

          • Antinous / Moderator says:

            Yes. But where the seller’s sole obligation is to mail the correct product in a timely fashion, anything less than a five-star rating is a red flag. It’s not like a restaurant where you could get a great appetizer and a crappy dessert, be seated quickly but have an inattentive waiter. That’s why you can’t translate ratings between different services.

    • novium says:

      That it’s probably a mediocre restaurant? ;) Actually, that’s just my experience in the city I live in. Perhaps people living/visiting here just have bad taste- I’ve noticed that great restaurants here tend to have <20 reviews total and the least-impressive-while-not-necessarily-bad restaurants have reviews in the hundreds. (And many of the reviews will be ecstatic.) 

      It's become my own mental sorting system: I look for restaurants with 15 or so reviews that are fairly good but not stellar, especially those that praise the food in general, but complain about minor, subjective things like: "Fruit plate had too many strawberries." The people that write those kind of reviews always seem to mark them down a star or two for that sort of thing, but it's obvious the food is good….where as the hordes tend to be all, "BEST BREAKFAST EVER! THIS RESTAURANT MAKES VACATIONING HERE WORTH IT!!!111" 

      • You’re right. But often I’ve seen a score dragged down by multiple super-negative reviews, written by angry people because one little thing went wrong. For example, someone was left standing for too long before being seated, and suddenly the food is “disgusting”.

        xkcd on online app reviews: http://xkcd.com/937/

      • LikesTurtles says:

        A few years ago I was really into Yelp. Had Elite status and was one of the top reviewers in my city. But I stopped when I noticed that I was writing stuff like “too many strawberries”. My desire to write what I felt was a complete and balanced review had led me to take a critical eye to everything. I wasn’t enjoying my meals like I use to because instead of just brushing off minor things, I made a mental note of them to use later in my review. Even worse was that I noticed that many of my friends on Yelp were doing the same thing but extended this behavior into their personal life. No wonder the discussion boards can get so nasty. The Elite members tended to break up into constantly shifting cliques depending on who had last stabbed who in the back. It was like high school all over again. This was particularly disturbing to see in people who I knew before Yelp because they had never acted that way before.

        While it was fun at first, ultimately Yelp was poisoning my soul (or maybe it would be better to say Yelp was helping me to poison my own soul). Once I stopped writing reviews, just about everything in life seemed to get better. Over time most of my friends who I introduced to Yelp stopped writing reviews too. But I still hear from time to time about the non-stop social chaos that is the clique of the frequent reviewers. Glad to no longer be part of it.

        • Martijn Vos says:

          It seems that critics in general are often not happy people. So what happens when sites like yelp turn everybody into  critic? Suddenly everybody is unhappy.

          • novium says:

            I don’t know. I think things can be taken too far, but I also think that there’s room for people who say, “well, I guess there were too many strawberries,” because they’re trying to say something about what the experience was actually like and not read like advertising copy. Plus, to a certain extent, I think a little bit of amateur criticism can increase enjoyment, because then you can really appreciate skill and difference and novelty. And I think this applies to many things- food, wine, books, music, television, whatever. I once met someone for whom hershey’s chocolate and  delicious high-quality dark chocolate were nearly indistinguishable, except for the sweetness…and then benefit of that, I guess, is that they’d enjoy it all equally, but the downside was they missed the sublime pleasure of the stunningly good chocolate.

        • bkad says:

           Possibly, if you care about reviews and dining, you were also selecting restaurants that were better than average to begin with. You never see 1s and 2s, so one’s sense of ‘average’ gets out of wack.

      • bkad says:

         I do the same thing, I read the negative reviews and try to determine whether the reviewer was being unfair (complaining about speed of service in tiny places the busiest night of the weak, complaining about failure of servers to act as the diner’s personal slave) or subjective (too many strawberries). Similarly on Amazon, you will sometimes see complaints (paraphrased) which say, “You sent me exactly what I ordered but it wasn’t what I wanted!”). If that’s the worst anyone can say about you, you’re an awesome business.

  2. AshleyC says:

    Companies can also pay yelp extra to push negative reviews to the back. :/

    • Tim in SF says:

      This is a persistent rumor for which I’ve yet to see any evidence. 

      • That_Anonymous_Coward says:

         There are several lawsuits and news stories about it…
        http://www.eastbayexpress.com/ebx/yelp-and-the-business-of-extortion-20/Content?oid=1176635

        I bet you think the BBB rating is sacred too…  they got caught selling good ratings.

      • mindfu says:

        I have seen this exact thing happen to a friend of mine who owns a restaurant in Brooklyn. She has almost nothing but 4 and 5 stars reviews, but most of the negative ones are above the fold. She called Yelp and asked to have them at least filtered or shown chronologically, and was told point blank that the only way that was possible was if she purchased advertising.

        When instead she demanded not to be listed at all in Yelp, she was told that couldn’t happen either.

        Yelp is a useful service, but is twisted more than a little towards the shitty by the capitalist model they’re pursuing.

  3. NoOneSpecific says:

    And what is YELP’s liability as the company who’s business model is basically recommending or vouching for said moving company? Are they liable for what is written on their site? Should they be held liable for not properly vetting these companies?

    If you say ‘No” now, realize that it is only a matter of time until someone launches a Class Action against a ‘Yelp’ or an ‘Air BnB’ and the whole model changes.

    EDIT:
    YELP is basically positioning themselves as an introduction service. More than just advertising media like the Yellow Pages as they take money not only for the ad but also to down rank negative reviews to the bottom of the list. With that one component they are now in a position to be liable should the company defraud you as they are in essence vouching for said company.

    • They’re not reviews provided by Yelp, they just house them.

      Suing Yelp for a bad experience would be like suing the yellow pages because a plumber ripped you off.

      Plus there’ll be disclaimers.  There are always disclaimers.

      • Martijn Vos says:

        But what if Yelp hides negative reviews? Then they’re suddenly intentionally misleading. Then it’s more than simply housing other people’s reviews.

      • Marc45 says:

        Tell that to MegaUpload…

        But according to the original complaint, Yelp actually engages in moderating reviews. IANAL, but this would seem they have some liability there.

        Let’s face it, companies like Yelp aren’t here to assist the consumer. They’re here to assist the businesses that are reviewed. If they filter out “bad” reviews, then businesses are happy and a disgruntled consumer is more or less irrelevant.

        Personally I never look at Yelp as truly useful since the reviews are suspect.

        • Bad Juju says:

          Indeed, no trust for Yelp here.
          Speaking of plumbers, all but 3 of the reviews for the closest one were all entered on 9/11/2011.  Obvious shill is obvious?
          Oh, two of the others were on 4/20. Happy Birthday!

          If these reviews were Craigslist ads, they’d have been flagged into oblivion, but that’s just not good for business!

      • LogrusZed says:

         That isn’t an accurate comparison.

        There is no pretense that the Yellow Pages large ads are a recommendation; they are absolutely transparent in the fact that ad placement and size is specifically related to the alphabet and the willingness of the merchant to pay for the size of the advertising.

        Yelp! is in the position of advertising themselves specifically in a capacity as a democratic ratings system where the user of a service provides the basis for a score that potential users can do their own filtering; but in practice they are doing their own filtering and de-facto colluding with the merchants in a manner which has the effect of gaming the system to the disadvantage of the consumer.

        I don’t know enough about the Yelp! business model to know if there is motive for intentional collusion or if it it merely a result of laziness and/or incompetence. The result, however, is the same for the potential consumer: They are fed inaccurate information.

  4. grimc says:

    I feel for the guy, but the solution isn’t an algorithm–it’s digging a little deeper than a couple of Yelp reviews. You’ve got to have better judgment than to trust a company that has 2 5-star reviews with all of your belongings.

    • novium says:

      His solution wasn’t an algorithm. It was identity verification. But he has a good point that their current methodology is a bad one:, and I have to agree. I’ve looked at plenty of listings there where all of the bad (and probably more honest) reviews were hidden, but all of the good (and suspicious) reviews were visible.  Even recently, with that terrible restaurant in New York that got mentioned here. 

      • chf64 says:

        The solution is to not use a sole source when making a decision about a product or service.  Always use multiple sources.

        I apply this principle to everything I reasonably can in life. Gather information from multiple sources and put together your own opinion.

  5. MaximumOvertroll says:

    I’ve got a brilliant “smarter algorithm” for yelp, it’s this brand new thing called “captcha”.
    This is why I dont use services that let you pay for good reviews. Hey, I’ve got an even beter idea! DON’T LET PEOPLE PAY FOR REVIEWS!

  6. novium says:

    I really hate the proposed solution- using identify verification like the credit companies do, e.g. “Which of these four addresses have you lived at”.  Because yeah, that’s exactly what I want to do when creating an account on every random website out there: not only tie it to my real identity, but have it pull sensitive information on me. It’s bad enough with the credit companies, but there I’ve got no real choice and it’s slightly better than the alternative of even easier identity theft.  But for reviews of businesses? Really? A better solution isn’t just I don’t know, using the algorithm he originally proposed, or even just crowd sourcing it? Haven’t studies shown that people are pretty good at recognizing fake reviews? Why not just have it so that you’ve got to validate some random review before you can post one? (Or get points for doing so or something). 

  7. ChicagoD says:

    Well, honestly, if you don’t have any friends or acquaintances who have moved with a moving company and can’t afford a national moving company, looking up reviews seems to me like a reasonable step. I can’t really fault the guy for that.

  8. Tim in SF says:

    I agree that making the Filtered Comments link gray, in parenthesis, and at the bottom of the page are all BAD UI decisions. 

    Still, I generally do not want to read Yelp reviews from people who are not regular Yelp reviewers. 

  9. 1. Don’t use Yelp for anything more expensive than a meal.

    2. Don’t use professional movers at all. Most of them are crooks. Instead, use commercial freight services that are willing to deal with individuals, such as ABF.

    • Robert says:

      ABF U-Pack was great for me compared to U-Haul’s U-Box.  They already have a network of trucks, so their cost-per-mile is low, and then they basically just tack on +$50-100 on both ends to pay for somebody to drive the last mile(s) to your place.

      I had a great, personable experience with them and a fairly terrible experience with U-Haul (which doesn’t have that driving network or a strong culture of customer-interaction).

      Sorry U-Haul!  Maybe next time you’ll call to let me know you can’t keep your promised / requested “sometime before noon drop-off-the-box” in advance of me driving from Dallas to Houston in order to receive it from you.

    • mindfu says:

      I personally had a great experience with a website called Uship. It’s like ebay for shipping  – you put in where you’re moving to and how much stuff, and them moving companies bid for the right to move your stuff. You can see the company’s reviews and their percentage positive rating. No company knows who you actually are until you accept their bid.

      Yelp is not designed to offer the kind of specialized service for movers, IMHO – as well as many other services, no doubt.

  10. MichaelB says:

    I had an identical experience with looking up a company when doing a home remodel — Yelp hid the negative review that exactly matched my horrifying experience, and I didn’t find it until afterwards.

  11. tmdpny says:

    Honestly, for such a big decision, you should always always check the BBB before you put money down.  They check and verify and find resolutions for issues, and objective and can always be trusted.  It’s one thing to wonder where you are eating tonight, it’s quite another to hand over ALL of your material possessions to.

  12. johnjupiter says:

    uhm. read the filtered reviews. THERE’S A SOLUTION!

    • LogrusZed says:

       Consumers, in any medium, are inundated with fine-print or murky TOS agreements to the point where nobody reads them all, this goes for the minimum of warnings businesses provide to indemnify themselves from actions. If you claim to be as cautions as you’re admonishing others for failing to be you’re a damn liar or the kind of psychotic that has notebooks filled in the manner of Kevin Spacey’s character in Seven and possibly thousands of labelled jars of your own excrement in your home.

      Even the link for filtered reviews is shaded in low-contrast to the background, unless you use a text modifying plugin for your browser then you’re unlikely to even notice that they filter reviews.

  13. fuzzyfuzzyfungus says:

    Honestly, in the case of Yelp and their history of helpfully modifying the reviews of those businesses willing to pay up, I’m not sure that this is a design problem. It’s a perverse incentives problem that happens to manifest as a set of design decisions.

  14. Whenever the ability is there. I sort reviews, to read the negative ones first. Then I determine, if I were to be in a situation, would I too, see it as a negative. 

  15. techdeviant says:

    I’ve successfully used angieslist to find good companies, including a moving company that worked just fine for me. I don’t know if the right solution is to charge users for good information though.

  16. foobar says:

    Giving an entity like Yelp access to financial/credit information is just begging for fraud.

  17. Dawn Grobe says:

    I have the same kind of complaint, albeit on a smaller financial scale, with sites like Amazon and iTunes.   I tend to read the most critical reviews first to see if they are complaints against the third-party retailer, user error (e.g. didn’t read the description), or an actual problem with the product.  The other thing I look for is the 5-star “I haven’t used it yet, but I am so excited to have ordered it!!!!1!” reviews.  All of these factor into the overall score of the product regardless of the users actual interaction with the product because they are only allowed to rate the item on one scale.

    Of course, I also hate audiobook reviews that say “This book was really great.”  I can read an Amazon review to see if people liked the story, I want to know if I actually want to listen to the reader or not. 

  18. Jardine says:

    Woah, woah, woah. There are people you can pay to move your shit for you? Then why do I keep having to carry my relatives’ shit around?

  19. technosean says:

    Yelp is completely useless. Shady practices. Paid-for suppression of negative reviews and escalation of positive ones. If you read Yelp you’d think every place was a 4-star restaurant! I can’t possibly take Yelp seriously.

    Sigh. I do wish there was something like Yelp but not bought by the places reviewed.

    • penguinchris says:

      I find Yelp to be very useful – but you have to use it carefully, just like any online review system. When I lived in SoCal I used it to find someplace new to eat at least once a week. All my favorite regular places there are places I found via Yelp, too, but I tried new things as often as possible and Yelp is currently the best resource to help with that.

      When I visit SF, NYC, or anyplace like that I always check Yelp for good places to eat – it’s especially important to do a bit of research if you’re traveling. I’ve never been outright burned by someplace I found via Yelp, though of course I’ve been mildly disappointed (usually just because of personal tastes).

      I know full well all the shenanigans that go on behind the scenes, but as a regular user I also know that it’s easy to figure out what’s going on – unfortunately you can’t always take everything at face value.

      I stopped providing my own reviews for a variety of reasons, such as not wanting to provide value (my reviews are good quality) to a company that does shady, nigh-unethical stuff. But as a resource in its current state it’s quite valuable.

      I did find an excellent place to get a hair cut via Yelp, but I probably wouldn’t trust their reviews to help me find anything else besides food places.

  20. Andy says:

    Yelp is not a neutral party, so I would never trust reviews from them. They take money from places that are reviewed on their site. Or at least they try. Look at their default sort, “Yelp Sort”. WTF is that?? It’s where they decide who’s reviews show up first on the page. 

    • LikesTurtles says:

      I don’t know what Yelp does internally but business owners certainly can pay reviewers for positive ratings. Back when I was Elite restaurants would give us freebies which no doubt influenced our ratings. This was especially true when we’d publicly discuss meeting up to eat and drink at a particular bar or restaurant. Sometimes they’d even have a table ready for us without any of us even calling ahead. For the business it paid to monitor the talk threads and events pages to know when we were coming. No doubt we also received extra attentive service because they knew we would later write reviews. A couple of plates of free appetizers and a drink special that mysteriously was only offered to us would equal a dozen or so positive reviews. Not much Yelp can do to guard against that. They’re certainly not going to punish Elite members who accept freebies since they’re the ones providing the majority of the site’s content.

  21. BarBarSeven says:

    I used to post on Yelp fairly regularly. Focusing mainly on local shops and such. It was a useful resource at one point. But then I started to notice some very obviously “shilly” posts in reviews of local restaurants

    It was quite bad and quite obvious since many of the places had barely opened for a day and suddenly *BOOM* 1/2 a dozen glowing reviews. And they were clearly too specific and perfect to be just a random first day customer.

    Deleted my account on Yelp and basically only use it–if I ever do–like the Yellow Pages: just for the bare bone basics. The heuristics needed to create a clean and trustworthy open review system doesn’t exist yet.

Leave a Reply