Media Cloud: Watching Media Flow

Dan Gillmor is a BoingBoing guest-blogger.

MediaCloudThe Harvard Berkman Center for Internet & Society's new Media Cloud

is a system that lets you see the flow of the media. The Internet is fundamentally altering the way that news is produced and distributed, but there are few comprehensive approaches to understanding the nature of these changes. Media Cloud automatically builds an archive of news stories and blog posts from the web, applies language processing, and gives you ways to analyze and visualize the data. The system is still in early development, but we invite you to explore our current data and suggest research ideas. This is an open-source project, and we will be releasing all of the code soon. You can read more background on the project or just get started below.

(Note: I'm a Berkman Fellow, but I'd highlight this even if I wasn't. This is an important project for helping us understand what's going on in media.)


  1. For years now I’ve been meaning to take the time to write the software that takes the closed captioning from broadcast television (this is much easier with digital tuners now) and compares all news broadcast for phrase patterns — memes.

    I think it was from Stephen Jay Gould’s The Hedgehog, the Fox, and the Magister’s Pox where he complained about definitions of scientific observations or facts that were repeated rote from textbook to textbook, and thus uncritically examined, such as whether the definition was even a very relatable explanation to its audience.

    But my aim is to automate a phenomenon that I find myself doing semi-regularly when watching television news (that isn’t The Daily Show, although come to think of it, their battery of TiVos and army of interns seem to do something quite similar): Mapping across time the adoption particular key phrases used to describe complex issues in concise and framed manner, seemingly intended to keep the audience thinking about the issue in a consistent and uniform way.

    I suspect this phenomenon has become particularly exacerbated due to the “24 news cycle” ala CNN et. al.

    I don’t think it’s necessarily a “conspiracy”, just that everybody in the news industry is too busy to figure out for themselves what’s going on, so they all just parrot each other instead.

    The worst of this were the memes originating from video news releases that mass-media parroted as actual news.

    The result is a kind of memetic echo chamber, and those “busy people” includes the people watching the daily news at home struggling to keep informed about the world.

    So, a meme mapper / phrase cloud generator for closed captioned text of television news would be useful to interpret the news more critically.

    Rather like Wikiscanner does for Wikipedia.

  2. This looks great!

    Advanced visualization technology and natural language processing are fascinating.

    Maybe I’ll get involved. My friend has been pestering/encouraging me to join an Open Source project.

  3. I recall during the massive Vietnam era protests that what I was seeing on the streets of San Francisco and what was being reported by the wire services weren’t in sync. The crowd numbers were always knocked down by the wire services.
    Given the number of still and video cameras in the hands of the public, I would hope there would be a way for the public to upload what really happened versus what was said to have happened.
    While the fearless reporter is a staple of fiction, the reality is that he’s beholden to the advertising department and a lapdog to politicians. So please collect from the blogs and the alternative press too.

  4. “While the fearless reporter is a staple of fiction, the reality is that he’s beholden to the advertising department and a lapdog to politicians”

    Yes. Given the possibility that your perception isn’t capable of coming up with detailed counts via the eyeball method or that reporters are all soulless sellouts who are just giving lip service to all those ethics they go on about, I think the latter is the most likely option.

  5. Woo-hoo! So many times when I think up a wisp of a cool “new” idea, I soon find out that Berkman has been working on it for months. Also great to see it covered on BB immediately.

    To build on Zuzu’s mention of VNRs (Video News Releases), see PRWatch’s extremely comprehensive Fake TV News piece. Also, see Sentiment analysis.

Comments are closed.