Philips has released a "keychain camcorder," which PC Magazine describes as a "thumb drive with a lens." It stores and plays MP3s and shoots stills and videos (admittedly, it doesn't do a stellar job at any of these tasks), and it's cheap(-ish) and you can keep it in the change-pocket of your jeans.
(Thanks, Reid!)