In The Atlantic, Alexis Madrigal recalls the otherworldly howl of a 56K modem negotiating its connection, and quotes our pal GlennF in his 1998 explanation of that characteristic anthem of the heroic bronze age of the Internet:
This is a choreographed sequence that allowed these digital devices to piggyback on an analog telephone network. "A phone line carries only the small range of frequencies in which most human conversation takes place: about 300 to 3,300 hertz," Glenn Fleishman explained in the Times back in 1998. "The modem works within these limits in creating sound waves to carry data across phone lines." What you're hearing is the way 20th century technology tunneled through a 19th century network; what you're hearing is how a network designed to send the noises made by your muscles as they pushed around air came to transmit anything, or the almost-anything that can be coded in 0s and 1s.
The frequencies of the modem's sounds represent parameters for further communication. In the early going, for example, the modem that's been dialed up will play a note that says, "I can go this fast." As a wonderful old 1997 website explained, "Depending on the speed the modem is trying to talk at, this tone will have a different pitch."
That is to say, the sounds weren't a sign that data was being transferred: they were the data being transferred. This noise was the analog world being bridged by the digital. If you are old enough to remember it, you still knew a world that was analog-first.
The sound clip comes from The Museum of Endangered Sounds, where you can find "the mechanical noise a VHS tape made when it entered the VCR or the way a portable CD player sounded when it skipped."