Windell at Evil Mad Scientist Labs has dredged up an amazing project book from the Internet Archive: How to Build a Working Digital Computer (1967) (by Edward Alcosser, James P. Phillips, and Allen M. Wolk) contains a full set of instructions for building a working computer out of paperclips and various bits and bobs from the local hardware store. You can even use paperclips for switches (though, as Windell notes, "Arrays of paperclip logic gates can get pretty big, pretty fast.")
The instructions include a read-only drum memory for storing the computer program (much like a player piano roll), made from a juice can, with read heads made from bent paper clips. A separate manually-operated “core” memory (made of paper-clip switches) is used for storing data.
So can this “paper clip” computer actually built, and if so, would it work? Apparently yes, on both counts. Cleveland youngsters Mark Rosenstein and Kenny Antonelli built one named “Emmerack” in 1972 (albeit substituting Radio Shack slide switches for most of the paper clips), and another was built in 1975 by the Wickenburg High School Math Club in Arizona. And, at least one modern build has been completed, as you can see on YouTube.