Neatorama posted the origins of several common symbols, reprinted from a book titled Uncle John's Supremely Satisfying Bathroom Reader.
Exclamation Point Origin: Like the question mark, the exclamation point was invented by stacking letters. The mark comes from the Latin word io, meaning "exclamation of joy." Written vertically, with the i above the o, it forms the exclamation point we use today.Link
David Pescovitz is Boing Boing's co-editor/managing partner. He's also a research director at Institute for the Future. On Instagram, he's @pesco.