Comments on: How many tweets are possible?

By: Erich Morisse

Erich Morisse — Thu, 28 Feb 2013 22:19:00 +0000

Long tail is fine, as Shannon’s 1.1-1.2 bits per letter is average encoding length, with more frequent letters encoded with fewer bits.

You bring up a great point though, it looks like the encoding is based off of a first order character frequency. Once you bring in word frequency and word transition frequencies (both also discussed by Shannon), the average encoding bit count changes.

Any way you cut it, this is my favorite “What If?” yet.

By: redesigned

redesigned — Thu, 28 Feb 2013 01:34:00 +0000

that finally explains twitter.

monkeys…keyboards…shakespeare…something or other.

By: chris rogers

chris rogers — Wed, 27 Feb 2013 21:29:00 +0000

Pretty sure ASCII is 7 bits and lots of those are control charcters like a linefeed. So really 26 letters, 10 digits and lets say 20 symbols. 56 letters could be encoded in 6 bits…

By: Matthew Miller

Matthew Miller — Wed, 27 Feb 2013 19:38:00 +0000

Wow! It’s almost like you didn’t read the article!

By: KWillets

KWillets — Wed, 27 Feb 2013 17:25:00 +0000

Using the Shannon entropy of common English gives a low estimate for this, because all the sentences that are uncommon are added back into the corpus. Shannon measured the ability of humans to guess the next character in a string, but the guesses relied on the fact that, say, “I am reading a book” is more common that “I am reading a boot”. In an “all possible sentences” corpus, both are equally likely; we lose the ability to down-rate the “long tail” of correct but unlikely sentences.

By: Eddie K

Eddie K — Wed, 27 Feb 2013 14:28:00 +0000

In a similar vein, I’ve had to design a number of small 16×16 pixel icons used for web site favicons, application drop down menus, file lists and the like.

It’s quite an art to make a 16×16 icon that’s colorful, attractive and meaningful that doesn’t look like something out of a broken video card or a child’s stick figure drawing.

How many such icons are possible? Assuming 24 bit pixels (8 bits for each of red, blue and green) gives 2^24 or 16,777,216 colors per pixel. With 256 pixels per icon you get (2^24)^256. Essentially a 256 digit, base 16,777,216 number or 2^6,144 different icons. We shouldn’t run out for a while.

I’ve always wondered what kind of fabulous icon a real artist such as a Picasso, Warhol or Pollack could produce on such a tiny canvas.

By: oasisob1

oasisob1 — Wed, 27 Feb 2013 13:45:00 +0000

1: Isn’t it ‘Munroe’?
2: i’ve been tweeting md5 hashes for ages, fuck the 140 char limit.

By: Roose_Bolton

Roose_Bolton — Wed, 27 Feb 2013 12:43:00 +0000

Of these, 25% are dadboner’s “Really looking forward to the weekend you guys” and another 25% are “Feel like I drank a thousand beers last night you guys”.

By: Matt Jacobson

Matt Jacobson — Wed, 27 Feb 2013 12:34:00 +0000

Arthur C. Clarke’s story needs an upgrade: “The Nine Trillion Tweets of God.”

By: Nathan Hornby

Nathan Hornby — Wed, 27 Feb 2013 08:04:00 +0000

Clever stuff!

By: Κατασκευή ιστοσελίδων

Κατασκευή ιστοσελίδων — Wed, 27 Feb 2013 07:11:00 +0000

It’s really huge number,if it’s real

By: corollax

corollax — Wed, 27 Feb 2013 06:29:00 +0000

You’ve just described a very simplistic, if computationally expensive, pre-computed compression algorithm. Because after all, that’s what you get when you run a compression program: a binary number for which there exists a corresponding piece of data.
You recover that data through a process called decompression. Which in the case of your index isn’t very computationally expensive, but is instead absurdly costly in storage space.

By: BarBarSeven

BarBarSeven — Wed, 27 Feb 2013 05:25:00 +0000

…they won’t be able to make a meaningful dent in the list in the lifetime of the Earth.

Which is why to make an impact on Twitter one needs to grab the reader by the neck & kick them in the balls by calling a 9 year old African American girl a “see you next Tuesday” in a Tweet. AMIRIGHT!

By: timquinn

timquinn — Wed, 27 Feb 2013 03:54:00 +0000

. . . but don’t take it personally.

By: timquinn

timquinn — Wed, 27 Feb 2013 03:53:00 +0000

100000000000000000000000000000000000000000000005

By: Aaron Griffith

Aaron Griffith — Wed, 27 Feb 2013 03:39:00 +0000

Randall used the concept of informational entropy, which is basically how many bits you need in order to fully specify something under a given set of assumptions. Two coin flips has two bits of entropy, but two coin flips where you know at least one is heads has about 1.6 bits. This amount also happens to be the theoretical lower bound on how small you can compress something.

He cited a fairly well-known paper by the father of information theory that says, given the assumption that a string of characters is a grammatical english sentence, each character has an entropy of around 1.1 bits. This is how he accounted for the fact that most ASCII strings are not grammatical english: normally, a random ASCII string will have about 7 bits per character.

By: Gorgonzola

Gorgonzola — Wed, 27 Feb 2013 03:31:00 +0000

It should take a supercomputer that long to generate them all and give them an index number. Then instead of tweeting, people can just reference the number of the tweet they were going to make.

By: Just_Ok

Just_Ok — Wed, 27 Feb 2013 02:50:00 +0000

tl;dr

By: timquinn

timquinn — Wed, 27 Feb 2013 02:36:00 +0000

Each utterance (tweet) has a context and therefore each one is unique and has a unique meaning.

By: OtherMichael

OtherMichael — Wed, 27 Feb 2013 02:13:00 +0000

@CPAN

By: Nathan Hornby

Nathan Hornby — Wed, 27 Feb 2013 01:12:00 +0000

Admittedly he likely considered this – if we thought of it so did he.

By: noah django

noah django — Wed, 27 Feb 2013 00:54:00 +0000

yeah, y’all’s thought was my thought. i was wondering how Monroe was arriving at “meaningful English tweets” using his methodology versus what you and chaircrusher suggest. Monroe’s idea sounds more like Borges’ Library of Babel to me.

By: SamSam

SamSam — Wed, 27 Feb 2013 00:43:00 +0000

I’m quite certain you didn’t read the post.

He dismisses the answer of “the number of possible 1[40] character ascii messages” in about the third sentence.

By: Nathan Hornby

Nathan Hornby — Wed, 27 Feb 2013 00:38:00 +0000

Especially when you consider that would include millions of combinations of just punctuation and mathematical symbols.

I don’t think that would be a very fun novelty account.

By: chaircrusher

chaircrusher — Wed, 27 Feb 2013 00:30:00 +0000

The number of possible 160 character ascii messages is not the # of possible tweets in English. If you constrain it to the actual vocabulary of English including slang and misspelled words, you cut down the possibilities enormously. Still a rough go to read it all, but I suspect a much lower number of messages that are arguably english sentences.

By: Drew_Gehringer

Drew_Gehringer — Tue, 26 Feb 2013 23:53:00 +0000

”Drop your panties, Sir William; I cannot wait until lunchtime!”

By: GawainLavers

GawainLavers — Tue, 26 Feb 2013 23:51:00 +0000

Reading all the tweets takes you ten thousand eternal years.

At least I’ve been given a reason to cheer the heat death of the universe.

By: Matthew Petty

Matthew Petty — Tue, 26 Feb 2013 23:41:00 +0000

“Hold the newsreader’s nose squarely, waiter, or friendly milk will countermand my trousers”

By: Boundegar

Boundegar — Tue, 26 Feb 2013 23:36:00 +0000

I caught an error in the math. The number of meaningfully different English tweets is actually zero.