Proposed solution for in-game harassment

Penny Arcade TV's "Harassment" episode looks at the phenomenon of in-game trolling, with its disproportionate emphasis on racism, homophobia and sexism, and suggests a solution: identify players who are muted more often than the norm, and set them to "auto-muted" when they join games, and have guild efficacy decline based on the number of automuted players in them. The idea is to create social pressure that make bullying into something that makes gaming suck for bullies.


  1. Hrm — might work in a random matching based circumstance. In, say, an MMO, the bullies could use it to mute people they don’t like, thus allowing them to inflict suffering by getting their victim branded as a bully themselves. 

    If there’s no way for an evil group of bullies to organize something like that, then it’s a decent idea. 

    1.  That kind of behaviour is definite possibility, depending on the game and how the anti-harassment system is set up. For voice chat, that kind of thing is mostly only on by default in random matchmaking team games like Call of Duty, Halo, and such. For MMOs, voice chat is generally restricted to people in a small group (like within a group in WoW), whom you may have some trust in already, or it happens via third-party VOIP programs like Mumble, Ventrilo, or TeamSpeak. For text chat in any game, it’s super-easy to determine what is bad, you combine the same algorithm suggested by the show (where if you get blocked by many people it restricts you) with algorithms similar to what spam filters use, only looking for hateful speech instead of spam. A positive entry in both filters gets you restricted. Obviously, some kind of appeal process if necessary, because false positives (either via algorithm problems or concerted gaming of the system by a group of bullies) need to be remedied.

  2. Uh, trolls are trollish and creating social and anonymous networks robust against gaming is crazy difficult. see Sims mafia c. 2004 onwards

    “Hi! I see from your hub that you’re new to the area. Give me all your Simoleans or my friends and I will make it impossible to rent a house.”

    “What are you talking about?”

    “I’m a member of the Sims Mafia, and we will all mark you as untrustworthy, turning your hub solid red (with no more room for green), and no one will play with you. You have five minutes to comply. If you think I’m kidding, look at your hub-three of us have already marked you red. Don’t worry, we’ll turn it green when you pay…”

    1. “…creating social and anonymous networks robust against gaming is crazy difficult.”

      It’s hard to overstate how true this is. I have a friend who writes anti-cheat/harrassment algorithms for a large software publisher. One of the biggest problems that friend runs into is that solutions such as proposed here make the incorrect assumption that bullies/lamers/griefers are looking to extract from the game the same things other players are.

      They aren’t.

      It’s amazing…these folks will get their account banned (not just muted…banned) and turn right around and pay for another account so they can do it again. All the negative in-game or real-world consequences one could apply don’t dissuade this sort of person. They will pay real money just for the satisfaction of making your life unpleasant.

      Give them a new avenue (such as creating global hardship, like automuting or the Sims scenario) and that just hands them tools for them to enjoy the game more and make you enjoy it less. :(

      1. Well said!

        I doubt there’s a solution that will make players “behave” and not end up with a game that no one plays. People flock to MMOs because they are open-ended and non-linear. Players want to create their own kind of world and as we see in the real world, everyone has their own point of view and don’t always get along.

        1.  There’s a difference between “point of view” and “I will spew vile filth at any and all who catch my eye.”

          At some point, creating new accounts should get pretty expensive, and perhaps if too many accounts are created using the same cc details and the majority of them have been muted for this kind of behaviour, the card could simply be blacklisted by the vendor for that game.

          1. I know I wrote that it’s hard to overstate the difficulty. But maybe I should have tried. I apparently didin’t emphasize the griefer behavior pattern strongly enough.

            Accounts can cost real money, and the game publisher can block customers at the credit-card account level. Griefers will still pursue their goal, paying more real money and using new credit-card #’s (keeping in mind that current credit-card features, such as one-time card #’s for online purchases, make getting a new credit-card # way easier these days).

  3. In the interest of properly crediting the show: the name of the show is Extra Credits, it is hosted by PATV.

  4. My solution: mute all… Halo used to have it as an option, now in COD I just have to do it manually. Unless you are playing with serious gamers who know the maps and understand effective teamwork a microphone is merely a waste of time and a distraction.

    Also if you’re a parent and you’re wondering where little Johnny picked up his sailor’s tongue I’d probably keep him off online games. I’ve personally poisoned the hearts and minds of many a child whose parents have ignored the suggested game rating.

    1. Seriously this.  I haven’t played a co-op style game since Half-Life one, just about the time team speak and the like was starting to take off.  I never got into it and never saw the need for it.  Perhaps in an MMO/guild type of environment, but I never played those types of games.

      Besides we didn’t troll people by calling them names…we trolled them by memorizing the maps and killing them relentlessly and without mercy.

      1. I don’t play games much anymore, but I’ve been in random matches where myself and one or more other players form an impromptu squad and start working together – and thus gaining a big advantage over the other team. 

        This happens without communication, amazingly, but sometimes when such a squad forms someone will start using their microphone to coordinate things better and in that case I don’t mind. Normally I mute anyone who’s talking. 

        I personally have only spoken into a microphone in a game when I was playing with friends I knew in real life, but those moments where impromptu squads form are easily the most rewarding moments in online FPS games.

  5. I’m not an ML expert, but it seems you could circumvent the Sims Mafia problem fairly easily. Using fairly basic (2nd or 3rd year undergrad CS level) algorithms, you could write some code to not allow malicious mafiosi to auto-mute people — you’d look at see that positives in certain vectors (a mute by mafiosi) don’t correlate well with trusted-human troll ratings; positives by others do. So you’d ignore the mafiosi. 

  6. The show is called Extra Credits, and it’s only one of many different shows on Penny Arcade TV. Good on you for watching it though, it’s a great show!

  7. One of the best and most interesting forms of game-community policing I’ve ever seen came from the age old (and still running!) text-based fantasy roleplaying game “Dragonrealms”.

    The DR system works as follows. There are, of course, a slew of official GameMasters, people who work behind the scenes to keep things running. They have all the god powers and the flashiness, but they’re not just enforcers – they’re also community coordinators. They help build and run world events, they take on the personas of major world NPCs and actually roleplay live with the players, they help maintain a constantly evolving overarching world storyline that the playerbase can influence in dramatic ways, and they also can tinker with the world code on the fly to make the world itself change to reflect events happening in realtime.

    They’re also the last line of defense for dealing with the player interaction problems. They run programs to monitor text input, looking for certain suspicious flags like clear cases of abusive behavior, or indiscreet naughty roleplay (they don’t mind what you do behind locked doors, but if you’re in a public area of the world you should keep things relatively tame). They try to resolve conflicts between players, they weigh evidence and look at chat and command logs, and they’re generally there trying to keep the peace when all other methods fail.

    Now here’s the cool bit. Those other methods I just mentioned? They’re largely player controlled tools. The community is encouraged to be self-policing, and for people to hold each other accountable for their actions. The game is highly roleplay intensive, to the point that any out-of-character behaviors done overtly in the public portions of the world are rather doggedly frowned upon by the community as a whole.

    Now, it’s not like you have to always be in charcter. For example, the whisper system allows you to talk to one or more people in a sort of private “chat channel”, where discussions are almost entirely unrestricted. If you want to talk to your buddy about the most recent hockey game, whisper away. Or if you want to use whispering as an in-character tool for literally having your characters whisper quietly among themselves so others around them won’t overhear, that’s fine too. There are some neat touches here, such as at times being able to notice that certain people around you are whispering among one another if your perception skills are sharp enough (although of course you can’t eavesdrop on their actual conversation).

    The fluid, semi-in-character nature of the whisper system also allows it to be used to police bad behavior. If someone is being disruptive, or breaking immersion, it is expected among the community for people to whisper to the offender and inform them of their transgression and politely ask them to stop. Most of the time, it’s an innocent mistake and all it takes is that polite reminder to solve the problem pleasantly. If the offender does not comply, however, the players are allowed and expected to take on a more stern warning tone. If they purposefully persist, players are then allowed to engage in a range of options for handling the troublemaker.

    The simplest and easiest method is the THUMP command. Basically, it allows sufficiently higher level characters to temporarily mute and stun offenders by thumping them in the throat with the flat of their palm, rendering them unable to speak for several minutes. Usage of the command is logged, to prevent abuse, and misuse of the thump power has severe repurcussions, but it is a quick and effective form of community policing that typically solves all but the really worst cases. The offending character is quite visibly and strikingly thumped for all to see, and then have to stand mute and unmoving while their thumper chastizes them publicly.

    There are other, messier ways to handle things as well. Improper player behavior is grounds for PvP combat, with such combat again logged to prevent abuse. If some idiot is being a complete jackass, it is perfectly acceptable for a well respected regular community member to simply cut them down where they stand, provided they at least warn the offender by whisper or in-character challenge. A secondary system exists to compliment this, wherein you can actually formally challenge a character to combat, causing them to receive a system message which they have to choose to accept and fight, or decline in disgrace.

    There are even more systems that work similarly. The player class of “Thief” does not officially exist within the game world. The Thieves’ Guild works very hard to ensure that they are considered nothing but a myth, and indiscretion on the part of players who play as thief characters can have severe consequences. Thieves are expected to police themselves whenever possible, but there are other systems in place such as NPC thugs who show up to bust a snitch’s kneecaps in, beat them to within an inch of their life, make absolutely sure they know that this was the result of loose lips, and then dump them in a ditch or canal – literally. A character who starts getting too much heat, either among NPCs or among the playerbase, will be treated accordingly as a liability to the guild. There’s a fully operational jail system where you can get hauled off by town guards for your crimes, and then have to spend real time serving your sentence before you are let out – time that stops counting down if you’re not logged in. Oh, and don’t even bother going AFK – being inactive for too long simply logs you out.

    Of course, it’s not all punishment either! The are systems in place that actively incentivize roleplaying publicly and engagingly. If you’re out there contributing to the immersion and depth of the world, GMs hidden behind the scenes may just take note of your constructive behavior and reward you with experience bonuses, vouchers for unique item customizations, or other rewards. Even when you don’t directly draw the attention of a GM, all players are allowed to upvote other players whose contributions strike them as particularly positive. Accrue enough points from your fellow players, and you can trade them in for similar rewards.

    If you give people the means to police themselves, and reward them for using that power responsibly while punishing those who try to abuse it, you foster a community that ultimately is much healthier and positive with less need for outside intervention.

  8. I’m not a multiplayer gamer (I prefer single player games like Bioshock or Skyrim), so I’m pretty far removed from that whole culture, but I gather Penny Arcade is popular enough for this idea to be widely discussed.

    I guess my opinion isn’t worth much on this subject, but my feeling is that despite whatever problems with this notion people bring up, any sort of half-decent implementation would have to constitute an improvement.

    And my hunch is that the more oversight real people have over the process, as opposed to relying solely on algorithms, the more beneficial it’d be. And IMO that’s the way it’ll stay until the day an algorithm exists that I can have an intelligent conversation with.

    1. I’m not a multiplayer gamer

      They kept saying “this is not us” in the video when referring to tolls in online multiplayer games but the “us” they were referring to isn’t all gamers, just people who play games online with strangers.

      That’s a big demographic, sure, but with the prevalence of casual gaming, traditional singleplayer games and the fact that there must be some gamers who only play multiplayer games with people they know, it’s hard to believe the “us” in the video is even a majority of all gamers.

      1. Whether or not you play games online with strangers (I suppose with an exception for casual gaming), to the general non-gaming public you’re a gamer and part of the same community that has more than it’s fair share of vocal douche nozzles.

        1. Yeah, I’m concerned that I as a gamer get tarred with whatever brush the MSM swings at ‘us’. Particularly as someone who lives in a country where R-rated games are banned. It’s past time the idea that you can isolate gamers as a social demographic* lost currency, but in the meantime my choices as a gamer are affected by any hype that gets stirred up.

          Not to mention the primary concern here, which is adopting a systemic approach to developing online mechanisms which not only enable but strengthen peer regulation to ensure the preservation – and perhaps flourishing – of decency.

          *aside from the distinction of who has computers

  9. Maybe some form of Whuffie. To get around the Sim Mafia problem, as long as you can go around being nice and earning positive Whuffie from more people than there are Sim Mafias giving you negative Whuffie, it’s a fair approximation, I suppose. As long as Whuffie is limited to a constant per player per time period.

    1. Dan’s voice isn’t sped up, but it is pitched higher than his normal voice. They held a poll about it late last year ( ) and viewers overwhelmingly chose to keep his voice pitched higher in the video. It can be a little off-putting at first, but you get used to it pretty quickly. In any case, don’t let the sound of his voice distract you from what he’s saying.

  10. Um.. The show is called called Extra Credits…
    PATV is just the name for the part of site where they put all the video series they host.

  11. Does anybody else find it amusing that he closes the piece by saying that “In our community you shouldn’t have to hide your voice online” after dozens of episodes behind a voice modulator? Or that as horrible as the speech is he is basically advocating against free speech?

    1. Dan’s voice isn’t pitched higher for anonymity, it’s done for effect and because viewers prefer it that way. They did a poll back in November ( ), and people overwhelmingly chose to keep Dan’s voice pitched higher in the videos. He’s stated his name, where he works, etc. multiple times online, and uses his normal voice outside of the episodes, so he’s not trying to hide his identity.

Comments are closed.