How badly designed reputation systems create in-game mafias

Randy Farmer's short essay "The Dollhouse Mafia, or 'Don't Display Negative Karma'" explores the well-known problem in reputation systems in which users abandon accounts that get negative feedback, and shows just how bad the consequences of this design can turn out to be.
That feature was fine as far as it went, but unlike other social networks, The Sims Online allowed users to declare other users untrustworthy too. The face of an untrustworthy user appeared circled in bright red among all the trustworthy faces in a user's hub.

It didn't take long for a group calling itself the Sims Mafia to figure out how to use this mechanic to shake down new users when they arrived in the game. The dialog would go something like this:

"Hi! I see from your hub that you're new to the area. Give me all your Simoleans or my friends and I will make it impossible to rent a house."

"What are you talking about?"

"I'm a member of the Sims Mafia, and we will all mark you as untrustworthy, turning your hub solid red (with no more room for green), and no one will play with you. You have five minutes to comply. If you think I'm kidding, look at your hub-three of us have already marked you red. Don't worry, we'll turn it green when you pay..."

If you think this is a fun game, think again-a typical response to this shakedown was for the user to decide that the game wasn't worth $10 a month. Playing dollhouse doesn't usually involve gangsters.

The Dollhouse Mafia, or "Don't Display Negative Karma" (via Raph)