How badly designed reputation systems create in-game mafias

Randy Farmer's short essay "The Dollhouse Mafia, or 'Don't Display Negative Karma'" explores the well-known problem in reputation systems in which users abandon accounts that get negative feedback, and shows just how bad the consequences of this design can turn out to be.

That feature was fine as far as it went, but unlike other social networks, The Sims Online allowed users to declare other users untrustworthy too. The face of an untrustworthy user appeared circled in bright red among all the trustworthy faces in a user's hub.

It didn't take long for a group calling itself the Sims Mafia to figure out how to use this mechanic to shake down new users when they arrived in the game. The dialog would go something like this:

"Hi! I see from your hub that you're new to the area. Give me all your Simoleans or my friends and I will make it impossible to rent a house."

"What are you talking about?"

"I'm a member of the Sims Mafia, and we will all mark you as untrustworthy, turning your hub solid red (with no more room for green), and no one will play with you. You have five minutes to comply. If you think I'm kidding, look at your hub-three of us have already marked you red. Don't worry, we'll turn it green when you pay..."

If you think this is a fun game, think again-a typical response to this shakedown was for the user to decide that the game wasn't worth $10 a month. Playing dollhouse doesn't usually involve gangsters.

The Dollhouse Mafia, or "Don't Display Negative Karma" (via Raph)

• [coaxial]

  While it is true that negative karma does encourage karma bankruptcy, and displaying raw scores does encourage gaming the system with group think, but mafias can be discouraged by active moderators. How hard can it be for a mod to simply create a new account and wait for the shakedown, then ban all the IPs of the mobsters?

• [AirPillo]

  Speaking from personal experience, The Sims Online had a host of other problems with it that I’d honestly have considered to be more off-putting than this scenario.

  In the end though the lesson is merely for players to ignore populist reputation systems as they are by nature so very fallible and misleading.

  The only ones worth paying attention to are the ones like in EVE online, where someone’s “Security Rating” is decreased automatically as a result of crimes committed against other players. If someone never steals from or attacks other players or commits crimes against peaceful NPC’s, they will never get a negative “security rating”.

  Of course, it’s increased by killing criminal NPC’s, so even that has its loopholes. Players can commit crimes and then spend some time killing NPC pirates to raise their rating back into a positive value.

• [Cicada]

  I suppose one could say this is an added bit of realism to the game– people do tend to encourage the ostracism those who don’t benefit them or their preferences. This is just rather direct about it.

• [phillamb168]

  This sounds very similar to what I imagine goes on in the suburbs. Zoning regulations = negative karma?

• [Felix Mitchell]

  This doesn’t seem so different from a positive-only reputation system that would still allow groups to manipulate it.

  If the mafia is downvoting all other people or upvoting themselves what’s the difference?

  I think I’d prefer a system similar to Civilization IV where your relationship with someone is based on past interactions. At war = -3, good trade = +2, gave technology = +1.

• [F. Randall Farmer]

  Two responses that address a few of the comment threads here -

  1) To the idea that community moderation should have simply dealt with this problem I say – why so? I was in charge of community moderation at The Sims Online at the time, and it was very clear to users and the support team that this feature was a design mistake. It did not work as intended. The red-circles were only ever used by abusers. This should not have been the community team’s problem to deal with.

  But, during my brief time there, this misfeature didn’t change, so I had to deal with it, and that included suspending the accounts of very active and engaged Mafia members. That really sucked and earned me more than a few haters. A real waste of paying customers when a simple patch would fix things.

  2) A few comments (here and elsewhere) propose alternate implementations. I find that fascinating given there was no statement of what the feature was trying to accomplish. Before understanding that, no alternate implementation should be considered.

  Lastly, for the authors of the few dismissive comments, I’d like to reiterate the invitation to consider reading the book-in-progress and provide substantive feedback – we really want the book to be the best it can be. We can take it! :-) http://buildingreputation.com/doku.php

  Randy Farmer (Co-author)

• [sidereal]

  The problem may be well-known, but I’d hope the trivial solution is just as well-known.

  The common exploit both in group up-reputation voting and group down-reputation voting is the group voting. What you really want to see as a moderator is abusive accounts being flagged by the disparate set of players that they abuse. It is straightforward to weight an up- or down-vote based on how often the voter votes alongside other voters, to the point that a consistent voting faction’s ability to vote would degrade to 0 rapidly. The countermeasure is to generate large numbers of unique accounts for rigged moderation. In systems where accounts cost money (e.g. The Sims), this is a non-problem. In other environments, weighting moderation by account activity or straightforwardly preventing dupe accounts (which has other benefits) works fine.

  Enjoy.

• [acmeaviator]

  “How hard can it be for a mod to simply create a new account and wait for the shakedown, then ban all the IPs of the mobsters?”

  This assumes that game producers are interested in fairness over profit – which they most certainly are not. I love my various lives in the metaverse, but a hard lesson I learned early on is that game devs and producers are creating an income source, not a utopia. Sure there is the occasional crackdown on farmers, RMT, or other “subversive” elements but generally these are limited actions to scare a few noobs or taken on a massive scale only if they will break the game.

  For example it made sense for Blizz to file (and win) a lawsuit against Glider because it allowed anyone to buy a cheap bot that would autoplay the game. Let that go on for long and soon many people are leaving. But it generally does not make sense for Blizz to ban a farmer who may be working with many accounts and paying blizz hundreds of $$$ a month in subscription fees.

  Ultimately game management decisions are based like all business decisions – what makes the most profit for the company.

• [Anonymous]

  Compare to the clueless aggressive academic who decided to use the Champions locked tier for fighting — the community used it almost exclusively for chat, yet after making sure that the publisher endorsed fighting there, he insisted on beating up every coffee-shop patron he saw. The argument being that the publisher, not the community, was the arbiter of such a largely populated environment.

  What’s the difference? Simply, Monarchy vs. Democracy.

• [Anonymous]

  Have you changed your anonymous comment system? I don’t remember having to leave my email address before. hmmm Methinks we’re not living up to our professed values.

  • [Antinous / Moderator]

    You don’t need to enter your e-mail. It’s just a glitch that the field is there.

• [Salamalecs]

  I’m gonna make an offer you can’t ref… Oh, nevermind.

• [Patrick Arcee]

  Simple tweak:

  It takes one “ring credit” to give a negative ring
  You start out with zero ring credits.
  You gain one ring-credit per 100 hours of play.

  Result: It’s a precious ability that you will reserve only for those who really, really deserve it. Even if you are determined to be jerkish, using bots on a game with free accounts would be slow, so it’s more worth your time to go steal coins from a public fountain.

  ——–
  ALSO…

  A few people mentioned games that rate based on actions (war, etc) but it’s worth adding that Sims already tracks relationship wins and losses. It might not totally distinguish a fist-fighting menace from someone who doesn’t know when it’s a bad idea to tickle, but at least it’s reasonable.

• [Ernunnos]

  That really sucked and earned me more than a few haters. A real waste of paying customers when a simple patch would fix things.

  Except people who engage in that kind of griefing will just move on to some other exploit mechanism. And griefing loses you customers too. Although those losses are harder to identify than the ones you suspend yourselves. If you ever could find a technical solution to all griefing, those griefers leave of their own accord, as you’re not giving them what they want. But every course of action will result in someone leaving. No, you can’t have both populations, as the thrill of griefing comes from compelling an unwilling victim. Nobody’s going to volunteer to be the victim, and if they did, it wouldn’t satisfy the griefers. In the end, the designers and maintainers can only choose who leaves and who stays. Who the market for the game really is.

  The Sims Online was a perfect example of this, and it was fascinating to watch that process play out.