Stupid Unix trick: why you shouldn't pipe yes into yes

Update: M Tang's explanation for this is wrong, but Seth Schoen sent us a great correction

There's a GNU-coreutils program called yes whose function is to "output a string repeatedly until killed." M Tang tried piping the output of one yes command into another. It ended badly:

Taking a look at the source code for yes, it looks like the single argument is being stored in a char array, then, in a while(true) and for loop, each character is printed to the stdout, followed by a new line (\n) character.
So when we use the output of one yes command as the argument for another, the outer yes command fills up the computer’s memory with the output of the inner yes command. Then I have to restart my computer and feel stupid.

yes 'yes no' (via Hacker News)

I write books. My latest is a YA science fiction novel called Homeland (it's the sequel to Little Brother). More books: Rapture of the Nerds (a novel, with Charlie Stross); With a Little Help (short stories); and The Great Big Beautiful Tomorrow (novella and nonfic). I speak all over the place and I tweet and tumble, too.

MORE: computer science • Funny

More at Boing Boing

Ants and Stars: Bruce Sterling and Jasmina Tesanovic visit the Sardinia Radio Telescope in Italy

The Snowden Principle

relawson

so, fork bomb? lol
- eldritch
  
  Not quite, technically speaking, but it has the same net effect.
- Symbiote
  
  To be a fork bomb it has to make many, many processes (“forking” is the way new processes are made).
  
  The classic example, which you should only try if you have saved all work, is:
  : () { :|:& }; :
  
  It’s perhaps more obvious what’s going on if you replace : with a word and indent:
  bomb () { bomb | bomb & }; bomb
  
  On Linux (I’m not sure about anything else), it is safe (ish) to run only if you limit the maximum number of processes first, with ulimit -u 100.
http://twitter.com/agger_modspil Carsten Agger

I tried yes $(yes no) in a bash terminal, and it didn’t take down the computer – it failed to allocate memory and took the terminal window down with it. YMMV.
- eldritch
  
  Hooray, memory protection!
- billstewart
  
  More precisely, it crashed the shell that was running in the terminal window, and when the shell exited, the terminal window closed down cleanly, just as it would if you’d typed “exit” in the shell. You can demonstrate this by typing “sh” or “bash” into your shell, and then doing the dangerous stuff in that shell, which can crash and still leave you a terminal window with the outer shell where you can see the error message.
  
  Pro tip: try several different shells, such as bash and /bin/sh (which seems to be “dash” under Ubuntu), and run ntop in another window to get some feedback on what’s happening.
JT Montreal

Actually, I don’t think the “first” (on the command line) ‘yes’ will ever be executed. The shell will try to interpret the command in the backquotes first, to generate the argument list – so `yes no` gets run, but since it never quits, the shell never finishes building the “real” command line, eventually just running out of memory.

It is not a fork bomb. sh/bash/csh/tcsh/zsh will just execute yes once.

Remember kids: your shell is a programming language.
- KWillets
  
  Yes, it’s just an argument expansion. The author doesn’t understand how the shell works. The shell has an argument limit too, which precludes running the first program even if the expansion completes. That’s why we have xargs.
Bob pelorson

So this is just another poorly implemented GNU + Linux problem? When are people going to learn and switch to FreeBSD?
- wysinwyg
  
  Umm, you’d have the same exact behavior on FreeBSD or any other Unix-like in which you run the “yes” program from bash. It’s a “badly-written program” — apparently on purpose.
  - http://www.iandicomputing.com Clifton
    
    The “yes” program does a fine job of exactly what it’s intended to do, generate an infinite stream of output. It is not the fault of “yes” if you point that stream somewhere it will cause trouble.
    
    That’s like saying “a circular saw is a badly designed tool” because if you put your hand onto the running blade, it cuts your fingers off, or a fire hose is a badly designed tool because you can’t drink from it.
    - wysinwyg
      
      Sorry, I didn’t realize that the problem was with the shell invocation and not the program itself. The point I was actually making — that this behavior can easily be reproduced on FreeBSD — stands, however.
    - http://glitch.tl/ Michael Smith
      
      That you, Neal Stephenson?
- billstewart
  
  No, it’s a user who doesn’t understand how the shell works; the operating system is behaving just fine. the `yes no` generates an infinite amount of output, and hands it to an application (the shell) which needs to stash some of it somewhere before rejecting it or using it for the next step. It’s doomed to fail; the question is whether it does so gracefully.
  
  Older Unix shells had fairly tight limits on command length, e.g. 1023, while bash and dash seem to keep allocating memory until the operating system won’t give them any more, and then produce error messages and crash. What do your favorite FreeBSD shells do?
  
  I don’t see why the user had to restart his system just because the shell crashed, but I don’t know what environment he was running his shell in. Ubuntu deals adequately with applications that want to hog all the memory and a bit of extra CPU, but maybe he filled up his disk drive with a core file or something?
- Forkboy
  
  Don’t know about FreeBSD but it sure knocked my Solaris 10 workstation on its ass.
  - http://glitch.tl/ Michael Smith
    
    Yo should know not to run scripts whjich people post on the internet. There was that clever script which one guy had in his /. sig. I had to give it a go and fortunately my home directory was backed up.
http://barak.a.pearlmutter.myopenid.com/ Barak A. Pearlmutter

The command: yes $(yes) is not “piping yes into yes”. That would be yes|yes, which is harmless as the pipe will fill and then block the writer. Which is the whole point of pipes instead of using temporary files!
http://profile.yahoo.com/2C6ZLMTICDOZHQGW66SUEDARLQ Murphy

Ugh. Just set limits/ulimit to stop this dumbness.
Raphael Clancy

So, yeah… I tried this on xUbuntu 12.10 and let it run. Eventually the terminal crashed. But, it never affected the rest of the computer even slightly.
- c0rtana
  
  Now be manly and do it from console.
  - Raphael Clancy
    
    I’m too lazy to be a real man. ;) But, in my defense, since he’s on a mac, I’m pretty sure he ran it from a terminal too. Can you even boot a Mac to a CLI?
    - Raphael Clancy
      
      … and why would you want to?
      - allium
        
        (raises hand)
        
        Note: I know next to nix about Unix.
        
        A month or so ago I was visiting my brother’s family when my Macbook froze. Force quitting through the keyboard didn’t work, so I powercycled it. Turned it on again, and the main partition was gone – only the rescue and Windows partitions showed. I started up through the rescue partition, and its disk utility indicated the hard drive itself was sound, but the main partition wouldn’t mount. Trying to start up the main partition in command line gave me more specific error codes that indicated its file structure was okay, but its journal was corrupted. The journal can be turned off, but to do so you have to mount the partition, and you can’t mount the partition while its journal is corrupted! :/
        
        After cursing the heavens and doing some research on Google on another computer, I found that going into Terminal, entering “diskutil disableJournal force (partition name)”, ignoring the failure message, and entering “diskutil mount (partition name)” might work. Reset again, the main partition came back, and my niece & nephew got to play WoW again.
    - Cormacolinde
      
      CMD-S at startup will boot into single-user mode.
    - Chas. Owens
      
      Ah, the Mac explains everything. Macs don’t run out of memory until they run out of disk space (which will never happen as the machine begins to crawl long before that).
      - http://www.facebook.com/harrkev Kevin Harrelson
        
        This is called “virtual memory” and has been a feature of every major operating system since Windows 95. Using too much is a bad idea, however. As you use more, your system slows down. I would rather have limits so that my machine just dies rather than get slower and slower until the hard drive is filled.
- billstewart
  
  The terminal didn’t crash. The shell (typically bash) running inside the terminal did. When the shell exited, the terminal knew to go away and clean up after itself. Try typing “bash” or “sh” from the shell in your terminal window and then running the yes `yes no` stuff inside that – the inner shell will crash, but the outer one will be there and you’ll be able to see the error message.
bzishi

I don’t get it. Why didn’t he just kill the process (or wait until memory protection kills it for you)? Even if he was logged on as root, I don’t see how it would kill the system. And the command

yes `yes no`

isn’t a pipe, unless I’m confused by the shell. The pipe would be

yes no | yes

which would just run the first yes command until memory is exhausted (it wouldn’t get to the second one because the first one hasn’t terminated).

Update: I did a quick reference, and a backquote is a way for command substitution that I wasn’t familiar with. In any case, my critique still stands, but with the backquoted ‘yes’ being the one that causes the problem.
- http://www.iandicomputing.com Clifton
  
  To get pedantic, which appears to be the point of this thread, on a Unix or Unix-alike system running your example of “yes no | yes”, both “yes” programs would be started essentially simultaneously.
  
  The second one would run indefinitely because it’s not checking its input, and as somebody noted upstream, the first one would run for a little bit but would get suspended once the pipe buffer filled, waiting for the second to begin reading it. The net output of the pipeline would be the same as if you had just run “yes”.
  
  It’s good to have a clear mental model of what’s going on when you write shell scripts, and I have written a bunch.
W. Hogue

And don’t type “Google” into Google.
- mongo
  
  But you can search for “Google” on Yahoo and vice-versa.
  They’re used to it by now.
- chaopoiesis
  
  … and don’t beam the planet up in Star Trek.
Aurvondel

I’m surprised ‘yes’ even reads stdin… why does it need to?
- http://www.iandicomputing.com Clifton
  
  It doesn’t; the author is new to Unix – as he admits up front, so I would cut him some slack – and doesn’t yet understand the distinction between …|foo and `foo` or $(foo). The result of `yes no` that the shell tries to build a command line argument consisting of an unlimited number of reps of “nonono…” which (may) run the currently executing shell out of memory, depending on what shell you’re using, but it won’t run the whole computer out of memory.
phuzz

Running it through an SSH shell into my xubuntu VM eats 25% cpu and so far, several Gb of memory.
Ulp, no, my putty session just died and it looks like the system just killed it.
http://twitter.com/dargaud Guillaume Dargaud

My preferred way to crash a linux PC is to type the following ‘smiley’:
:(){ :&:& };:
- http://www.google.com/profiles/moneta.mace Mace Moneta
  
  Which doesn’t work on any distribution with sane limits.
http://www.iandicomputing.com Clifton

To sum up:
“yes” doesn’t kill processes, people kill processes.
Stephen Dennis

I recently found this – do not try it:

Command line Russian roulette

[ $[ $RANDOM % 6 ] == 0 ] && rm -rf / || echo *Click*
Dana Nonsense

This is only dangerous if the user has root. Users should almost never be running anything as root.
http://www.facebook.com/people/Maj-Variola/100003091129531 Maj Variola

Make a file called “a”
In that file put the text “a &”
chmod +x a
./a
Profit.
This would kill a sparcstation
- Chris Wright
  
  Why would you put . in $PATH? That gives me the screaming heebie-jeebies, if you know what I mean.
  - L_Mariachi
    
    ./ isn’t in $PATH, it is a path.
    - lebel
      
      then use $0 instead of ./a. It’ll work from anywhere…
http://www.facebook.com/greg.ercolano Greg Ercolano

Haven’t looked into this, but wondering why the ‘sender’ yes(1)’s write operation isn’t blocking when the pipe’s buffer is full.

It’s my understanding (and a common situation) that if the ‘receiver’ end of a pipe is unable to “keep up”, the sender’s write operation should be blocked for I/O (put to sleep by the kernel) allowing the receiver to catch up and empty the buffer. If the receiver never catches up (ie. second yes(1) not even reading stdin), sender should remain blocked indefinitely.

No special code should be needed for this; write(2), printf(3), fwrite(3) should all behave this way IIRC, as this sounds like something the kernel should be doing during the low level write(2) operation..
- http://www.facebook.com/greg.ercolano Greg Ercolano
  
  Oh wait, I see, this is a command line expansion thing with backquotes.. nevermind! Yeah, I’ll bet that breaks some shells.
  
  Some versions of unix (e.g. many recent releases of OSX) don’t even enforce ulimit maximums for ram, which means you can crash a mac with a ram eating process, even if the admin set a ram limit for users.. which is a pretty grave omission:
  https://discussions.apple.com/thread/3293075?start=0&tstart=0
  http://lists.apple.com/archives/unix-porting/2005/Jun/msg00115.html
- billstewart
  
  This isn’t yes|yes which would do that. This is
  $ yes `yes no`
  where your shell runs the right-hand “yes” command and uses its output to build the command string for the left-hand “yes” command
  yes no no no no no no no [ad nauseum]
  An old-fashioned shell would have a limited amount of memory available to build the command string, and once it filled it up it would either fail or “succeed” or otherwise stop reading more input, and it’d all be fine, and might or might not try to run the created command.
  
  Bash and dash and probably some other popular shells will let you create unlimited-length command strings, so they’ll keep malloc()ing more memory until that fails. The rest of the user’s system presumably reacted the way it would to any command that tried to hog all the memory, but since it was probably the user’s main shell, he interpreted its lack of responsiveness to his input as a system problem.
http://www.last.fm/user/kraniac/ Shazbot

yes no | head -256 | say
http://www.facebook.com/profile.php?id=1158369073 John McCormick

That’s why I use ‘maybe’. Not as reliable but then i don’t risk big issues with recursion.