Inside the kinda secret world of Facebook Community Council

I have newfound respect for online moderators who slog through potentially problematic user content all day. They get a real glimpse into the downside of humanity.

Facebook developers recently initiated me into Facebook Community Council, a secret shadow organization of vigilantes who destroy the content of ne'er-do-wells and miscreants. Our Council's blood oath: "To harness the power and intelligence of Facebook users to support us in keeping Facebook a trusted and vibrant community."

There's a whiff of McCarthyism or worse to the whole notion of people in a community reporting others for un-Facebookian activities. I signed up immediately. Immediately after I passed a tutorial and got certified, I got a long hard look at the seamy underbelly of Facebook and the nebulous concept of "community standards."

Turns out Facebook Community Council is less like vigilantism and more like beta-testing a crowdsourced tagging system where you are limited to one of eight options each time. Four are self-explanatory: Spam, Acceptable, Skip, Not English. The other four are the key problem areas, and I saw plenty of all of the specified naughtiness over time:

-Nudity (such as "visibility of pubic hair or genitalia, the display of sex toys, and solicitation of cybersex")
-Drugs (especially promotion or use of "drugs illegal in the United States... This includes depictions of marijuana plants/leaf logos. This does not include the use of alcohol or tobacco...")
-Attacking ("direct attacks on non public figures")
-Violence (such as "visible mutilation of humans (including self-harm) or sadistic violence against animals... images of urine, feces, vomit, and semen.")

Yum! Your tags are then compared to other Community Council members', and if there's enough of a match, some sort of action is apparently taken. It's strangely hypnotic, like Google Image labeler, mainly because you want to see how bad the next reported page or group is.

The majority are acceptable, reported by some overly sensitive person. The main categories of reported pages are:

• Pointwhoring for networks within the network: Mafia Wars, Farmville, etc.
• Regular whoring (escorts, nude models, sex workers, etc.)
• Middle and high school teachers who suck
• Middle and high school students who suck
• Soccer teams and players who suck
• TV talent show competitors who suck
• Bands and hip-hop artists who suck
• Religions that suck
• Hate groups masquerading as pride groups (lots of "white pride", etc.)
• Gingers
• Chavs
• Ginger chavs

Whew! That was a lot of detail! If that bugs you, I recommend joining a flagged Facebook group I marked Acceptable:

i hate it when people go into detail about everything.

• [zikman]

  sounds like the internet using the “power of the community” instead of actually hiring employees that need to be paid

  • [Anonymous]

    I am pretty sure this is what got AOL a big lawsuit.

    • [merreborn]

      No, AOL was sued because they offered partial compensation to their volunteer moderators, required them to show up to shifts on time, etc. The AOL situation resembled a poorly compensated job in more than one way.

      http://en.wikipedia.org/wiki/AOL_Community_Leader_Program#Department_of_Labor_Investigation

      If you don’t compensate them at all, and let them work or not work whenever they please, it’s not an issue.

• [Anonymous]

  One of the more irritating things are “fan” pages that are fraught with racism. There is a “Jews” page you can become a fan of which has tons of pictures of Hitler and the nazis as well as being moderated by Hazi Nitler. Nice.

• [sworm]

  As a former mod of a rather big gaming site, I also regularly used ‘the power of the community’.

  You offer them worthless points and the esteem of their nothing better to do peers.

  After a while they realize there’s no point to volunteer work unless you get to meet people you want to have sex with, which never happens online.

  It’s the basis of all charitable institutions.

• [tomslee]

  What zikman #6 said. I can absolutely see doing this kind of volunteering for a non-profit group, as Wikipedia editors do, but to do it for a billionaire-headed company? It’s like volunteering for your local Starbucks or .

• [tomslee]

  Oops. …or other big company.

• [Teufelaffe]

  @ #9: This may be overly revealing, but I’ve met at least 4 sexual partners online (one of whom I ended up marrying), so it most definitely happens. :)

• [Anonymous]

  Is it intentional that they share the acronym “FCC.” Ominous much?

• [Anonymous]

  I think Facebook needs to clean up its act on the massive fraud its ad partners perpetuate on users, personally, with full cooperation of Facebook. Personally.

  But, yeah, more of the secret police coming around and me getting more “You’ve been reported for harassment” notices because I said “fuck” or that you’re an idiot if you don’t believe in evolution, that’s totally awesome, too.

  One more good reason to get the hell off Facebook. This will not end well. Imagine Livejournal with three hundred and fifty million users, and…close enough.

  –Robert N. Lee

• [Anonymous]

  So what you’re saying is that there’s no point in referring a post for racism, bigotry, or homophobia, but any image of a sex toy is right out? The Facebook Community Council and I are not offended by anything like the same things.

• [Anonymous]

  Hi, I’m french, living in Canada and I’ve litteraly dreamed of such a kind of behavior from facebook. This will be for the best I’m sure.
  If you’re looking for french speaking persons, I’d be more than happy to join your team.

• [Teresa Nielsen Hayden]

  Andrea, thanks for saying a kind word about moderators. For the record, Boing Boing’s moderation (all hail Antinous and his staff) is based on rather more complex models of what goes right and wrong in online conversations, and what can be done about it.

  Facebook sounds to me like one of those forums where the management claims to embrace “community moderation” (which usually takes the form of a low-res tagging system) because they perceive it as the cheapest and easiest option.

  How can you tell the difference between forums that use community moderation as a cheap cop-out, and forums where they genuinely believe in it? The latter take the time and trouble to educate their users before leaving moderation in their hands. They also provide reliable backup for problems community moderation can’t address, like the occasional need to ban serious troublemakers.

  The list of Facebook tags doesn’t reassure me. It’s not a thoughtful system.

  Generally speaking, if there are only eight tags to cover all possible content, they should be carefully chosen, and they shouldn’t generate further problems. A single tag for “offensive, illegal, or threatening content” would replace half the tags on that list while covering a lot more ground and inviting a lot less trouble. Leave the precise determination of why and how something is offensive, illegal, or threatening to the people who assess the flagged content.

  It looks to me like Facebook needs a separate FAQ that explains what is and isn’t acceptable. It would be much clearer, and it would keep them from having to shoehorn that information into the tagging system. That’s a desirable state of affairs. It lowers the tone of the joint to make users read about “visible mutilation of humans or … sadistic violence against animals” or “images of urine, feces, vomit, and semen” when they’re just trying to tag some spam.

  More specific quibbles:

  Since “acceptable” is the default state of untagged content, it’s a wasted tag. Having an “acceptable” tag is also subtly dispiriting: of the possible responses the system offers you, the best you can say of something is that it fails to offend? That’s a great way to lower everyone’s expectations.

  You don’t need a tag to signify diffuse general approval. That judgement is implicit in the fact that readers have come to the site and are reading the material there. If you must have a general approval tag, “valuable” is a more useful and descriptive term, especially when combined with other descriptors: viz., “excessively abrasive, yet valuable.”

  It makes me uncomfortable to see “Not English” in what’s otherwise a list of offenses against public order. I suspect it’s there to tell Facebook they need to have that content looked at by someone who speaks that language, but if so, I wish they’d found a better way to say it.

  “Nudity” is not “visible pubic hair or genitalia, the display of sex toys, and solicitation of cybersex.” What they’re reaching for is “offensive sexual content.” Nudity is just bare skin. Equating the two is asking for trouble.

  “Drugs” is another ill-considered tag. General discussions of drugs, drug use, and drug laws are not illegal. In fact, they’re protected under the First Amendment. Privately owned sites can take them down on their own say-so, but they can’t defend that action to their users by claiming that the law made them do it. In the meantime, people who use illegal drugs can generate encoded language to talk about it in nothing flat.

  “Attacking (‘direct attacks on non public figures’)” is the tag that bothers me the most. Distinguishing between public and non-public figures has nothing to do with “keeping Facebook a trusted and vibrant community.” That distinction is purely about U.S. libel laws, which are much stricter if the person in question isn’t a public figure. (Note: it’s easy to become a public figure, even if you didn’t volunteer for it.)

  What a prohibition on attacking non-public but not public figures says to me is “Facebook doesn’t actually care how barbarically you behave, as long as they can’t get sued over it.”

  Moderation and site rulesets are always systems.

• [yuubi]

  > Violence
  > urine, feces, vomit, and semen

  I shall henceforth refer to the toilet as the “violence room”.

• [jeligula]

  @ #6 There is some of that, naturally, but by harnessing the power of the community they deflect a lot of criticism over being internet cops or censors. After all, it is users, not us, who is doing the policing.

• [Anonymous]

  I report heaps of stuff on Facebook, mainly porn links, or obvious sexual acts. I have reported profiles containing hard core sexual pics, then come back weeks later and seen them still there. Same with bestiality. Is there a huge backlog of work?
  Seems to be a huge “shadow Facebook” within Facebook, of questionable age stuff, many profiles which are private, but look to be under 18, have an open wall, and literally hundreds of friends within days of being created.
  A fella gets a bit suspicious at times….lol

• [jonathan_v]

  seeing ‘gingers, chavs, ginger chavs’ in a row made me spit my coffee out.

  hilarious. sad. hilarious again.

• [Anonymous]

  so when my video of my daughter’s preschool class dancing to a hanukkah song was removed last year from my facebook profile, due to alleged “copyright violations”, would i have the facebook community council to thank for this?

• [gastronaut]

  Lame. Censorship is not “moderation”. The guidelines FB have set out seem to be primarily geared at protecting the company legally, which shouldn’t be the concern of community members. However, I do credit them for not including copyright infringement on the list of guidelines.
  True moderation allows offensive or otherwise substandard content to be exposed as such by community members, but does not completely remove it. BB has the crude but somewhat effective technique of disemvoweling. However, Slashdot has had a sterling community moderation system for nearly 10 years and I’ve been disappointed how few sites have been able to copy it. In my experience, it’s much more gratifying to see troll posts modded down to the cellar by the community than to have them removed by some invisible overlords. The justice has kind of a “March of the Penguins” quality to it.

• [PTBartman]

  I always wondered if South Park picked Gingers because it was an anagram.

• [Anonymous]

  This is another common mistake made by moderators. Pot is not “illegal in the US”. It is illegal in some states still, controlled in others and legal for personal use in some. Of course their decision to exclude images of their choosing is their right. (should I make that BOLD, or will it be read as-is?) It’s also my right to criticize the removal and choose not to participate in their monetiz.. I mean services. It is comforting to know that at least Facebook has a multi-moderator system to cut down a bit more on abuse and corruption. But as long as said multiple moderators interact socially.. ganging-up, harassment and corruption is still very possible if not probable. No matter how very nice and level-headed you think they all seem. But you can’t take it all too seriously. It’s not like it’s THE most popu… oh. Well, it’s just the Internet. Right?

  (It is kind of amusing that it’s “just the Internet” if a user is being singled out, censored or abused, but VERY SERIOUS when it comes to copyright and threats.. make up your minds, people)

  • [Architexas]

    @#19:

    Possession of marijuana is technically illegal according to the federal government, but some states have chosen to legalize it for medicinal purposes, in contradiction of federal laws that ban its use. It is still considered a federal crime to possess it, but the “crime” is not being prosecuted in those states that allow medicinal marijuana use because there’s a whole “States’ Rights” power struggle currently under way.

• [Teresa Nielsen Hayden]

  Gastronaut @30:
  

  Lame. Censorship is not “moderation”.

  Lame. Moderation is not censorship.
  

  The guidelines FB have set out seem to be primarily geared at protecting the company legally, which shouldn’t be the concern of community members.

  The company has a right to protect itself legally. I object to that being the only thing they care about.
  

  However, I do credit them for not including copyright infringement on the list of guidelines.

  People have a right to their own work. If someone’s reposted someone else’s story or essay or illustration as if it were their own, the writers and artists have a right to complain. I’ve seen sites that entirely consisted of fraudulently relabeled content. I didn’t think they were a good thing.
  

  True moderation allows offensive or otherwise substandard content to be exposed as such by community members, but does not completely remove it.

  No. What you’re saying is that you prefer sites that preserve such content. Moderation takes many forms. It would be a tad provincial to declare any of them the One True Moderation System.

  As I said earlier, site rulesets and moderation styles are always systems. There are reasons to remove content, reasons to deprecate it, and reasons to preserve it, and all of them function differently in varying contexts.

  Do you actually want to talk about this in detail?
  

  BB has the crude but somewhat effective technique of disemvoweling.

  Disemvowelling has its own effects, some of which aren’t all that intuitive. Most of them vary with the site and the way it’s used. It’s not intended to duplicate the effects of other moderation techniques, which is good, because it doesn’t.
  

  However, Slashdot has had a sterling –

  That is, it’s your preferred system, and you’re used to it.
  

  – community moderation system for nearly 10 years and I’ve been disappointed how few sites have been able to copy it.

  In most cases, that’s because they weren’t trying.

  Haven’t you noticed? Different sites have different purposes, different rulesets, different moderation techniques, and different conversations. Slashdot isn’t Boing Boing. It also isn’t 4chan, Ars Technica, Daily Kos, the WELL, Flickr, Making Light, Firedoglake, Instructables, or the Cleveland Plain Dealer. All those sites have their own functional moderation systems, and their own distinctive sitegeists. Many of them, and quite possibly all of them if I looked hard enough, manifest interesting emergent properties.

  Moderation isn’t a single way of doing things. It’s a complex interdependent body of hacks, techniques, and rules of thumb, and it’s constantly evolving.
  

  In my experience, it’s much more gratifying to see troll posts modded down to the cellar by the community than to have them removed by some invisible overlords. The justice has kind of a “March of the Penguins” quality to it.

  I’m familiar with the effect. What you don’t understand is that there are conversations that won’t happen on Slashdot, and people who won’t hang out there, because the style of the place doesn’t suit them. All sitegeists exclude some users and conversations, and enable others.

  • [Andrea James]

    Thanks for both detailed posts, Teresa. You responded to most comments I planned to address. My main concern with the Facebook system is that it has a facade of non-hierarchical community-based organizing while remaining a top-down system. As a couple of readers noted, it’s a way to harness uncompensated labor. The reporting system gets a ton of abuse (people reporting some business page like Apple.com).

    What’s interesting with any website is that a core group of people come to care deeply about its community and wish to maintain whatever elements they like about it. They may find the compensation acceptable (placement on a high-score list, knowing they are blocking porn or racism or other things). The trick for site owners is to expand and evolve in ways that keep loyal users happy while welcoming new users. That’s a tough act to manage. Facebook is interested in upsetting as few consumers as possible, which is very difficult for a site that size.

• [Fishpaste]

  Craigslist uses a similar model except they don’t have a volunteer cadre; anyone can click the relevant button for spam etc. The fact that these are appointed begs the question as to how they are selected. That might be the real story as one can get a desired result by selecting people with a certain agenda etc. much like the U.S. Supreme court nominees.

• [gedelman]

  How come you don’t like details. Is that where God (or is it the devil?) lives? I liked your details.

  • [Teresa Nielsen Hayden]

    Gedelman, this is a set of details I like more than most people do, so I was checking to see whether I was being a bore.

    I’m glad I didn’t bore you.

• [Anonymous]

  Hah, I’ve just learned what a chav is. Thanks, Internet, for making learning fun!

• [Berandor]

  What are gingers and chavs? Is a ruler a sex toy (it can be)? Is the Council lenient on non-English pages with drug imagery?

  • [angusm]

    A ‘ginger’ is someone with red/orange hair: recently, a Facebook group called ‘National Kick a Ginger Day’ (apparently based on an idea from South Park) was blamed for inciting a series of attacks in schools.

    ‘Chav’ is British slang for a particular social group: broadly speaking, white lower-class youths, associated with particular styles of dress and behavior. The term is contemptuous and pejorative.

    • [kuanes]

      I don’t think I’ve ever seen the words ‘chav,’ ‘contemptuous,’ and ‘pejorative.’ in the same sentence before.

      God bless you, intarwebs!

    • [Hanglyman]

      I’ve always wondered what the deal is with this whole “gingers” thing. Not only the baffling, unexplained prejudice against a certain hair color, but why they’re even called gingers when actual ginger is more of a blonde color, or brown on the outside.

    • [joeposts]

      “(apparently based on an idea from South Park)”

      As a SP fan, I feel this deserves clarification. The episode (“Ginger Kids”) was about Cartman bullying the red-haired kids by comparing ‘ginger kids’ to vampires. Eventually all red-haired children in school are regarded as social outcasts. Cartman’s friends, disgusted by his (and other townspeoples’) bigotry, bleach his skin and dye his hair red while he is asleep. Cartman awakes to find himself afflicted with “gingervitis” and gets bullied by the other kids at school. Then he organizes the other ‘ginger kids’ and starts a campaign against the non-gingers, which eventually leads to a plan to kill everyone WITHOUT red hair, as gingers are the “chosen race.” Just as Cartman is ordering non-gingers to be thrown into a pit of lava, his friends reveal their trick and he has a sudden epiphany, telling his followers that we all have to get along.

      But I guess the media picked up on the idea that SP had a ‘kick-a-ginger-day’ episode and that’s where it came from, because they’ve been consistently blamed for the idea. It’s probably easier for the bullies and their parents to point a finger at a controversial TV show than to take responsibility for acting like Cartman, who is a little nazi shit.

      • [nnguyen]

        Thank you, I was about to explain before I got to your post. As foul as South Park can get, they usually have very good points to get across.

• [brinylon]

  I guess stuff like racism and sexism is just fine, why bother reporting that. PCness is so uncool. Or am I taking things too seriously now?

  • [Rob Beschizza]

    Too serious to notice that racism is actually in the list of most-reported pages, at least!

• [Antijoe]

  So I can be a cheer leader for Phillip Morris, but a picture of me wearing a shirt with a ganja leaf is morally wrong?

  Just look at those corporate values, just look at them!

  • [MrsBug]

    Antijoe, if Phillip Morris sold ganja, you’d be good. Just give it time…it’ll happen.

• [Anonymous]

  Cf Craigslist’s longstanding flagging functionality…

• [Halloween Jack]

  To paraphrase Ani DiFranco: Every tool is a sex toy if you hold it right.