I/O Redirection and Pipes

One of the true powers of the shell is the ability to pipe and redirect input and output. With these two tools, the huge amounts of small tools that bash provides can be combined together into complex programs that would be hard to create otherwise.

In the simplest terms, a pipe is moving data from the output of one program to the input of another. In order to fully appreciate this concept, I must introduce two new commands: echo and grep. Echo simply prints its argument out on the command line, and grep is a search utility.

onionchesse@hunger:~$ echo -e "Hi!\nKittens are fun\nI like to eat onions\nAnd chesse"
Hi
Kittens are fun
I like to eat onions
And chesse

In this command there are two new concepts, arguments and newlines. The ‘-e’ on the command is the argument. In this case, the presence of the -e tells the command to enable ‘escape characters’ which allow us to put tabs and other hidden characters in the output. The ‘\n’ in the command is a ‘newline’, a character that ‘moves the cursor’ down a line. It causes the output to be printed out on multiple lines!

Now that we see how echo works, lets see how to use grep, to search for ‘chesse’

onionchesse@hunger:~$ echo -e "Hi\nKittens are fun\nI like to eat onions\nAnd chesse" | grep "chesse"
And chesse

Hey look! only the ‘And chesse’ line got printed out! Lets examine this  a little closer… The ‘|’ character is a ‘pipe’, it will send the output of the ‘echo’ program to the input of the ‘grep’ program. The grep program will then block any lines not containing ‘chesse’, as that is what we told it to search for. What if we only want it to print out ‘chesse’, instead of the entire line? We can use an argument to the grep command to do that!

onionchesse@hunger:~$ echo -e "Hi\nKittens are fun\nI like to eat onions\nAnd chesse" | grep -o "chesse"
chesse

The ‘-o’ argument tells grep to print ‘only matching’ strings, which is why we only got the ‘chesse’ this time!

However, we still have no idea how to write to files! The simplest way to do that is with output redirection, or the idea of sending the output of a program somewhere besides your terminal window. Lets see if we can put the output of the echo command into a file!

onionchesse@hunger:~$ echo -e "Hi\nKittens are fun\nI like to eat onions\nAnd chesse" > foodies.txt

The ‘>’ symbol redirects the output of the echo command to the file ‘foodies.txt’. Notice how there is no output on the screen! That is because the output got sent to a file instead. Lets ‘ls’ and try to find it!

onionchesse@hunger:~$ ls
Desktop Documents Downloads Dropbox foodies.txt Minecraft Music Pictures 
onionchesse@hunger:~$ cat foodies.txt
Hi
Kittens are fun
I like to eat onions
And chesse

The ls command clearly shows our new file, foodies.txt! We can ‘cat’ the file to find out the contents of it, and we find out what echo printed out earlier! However, the ‘>’ operator is dangerous, in that it overwrites any files already with the same name. We can prevent this with the ‘>>’ operator, which will append to the file instead! Lets go ahead and try it out on foodies.txt!

onionchesse@hunger:~$ echo "Do I like pie?" >> foodies.txt
onionchesse@hunger:~$ cat foodies.txt
Hi
Kittens are fun
I like to eat onions
And chesse
Do I like pie?

And we see clearly, that our output has been appended to the end of the file! Very useful, especially when you want to send debug output of a program to someone else.

This is only a tiny fraction of what is possible with bash, and by combining more tools using pipes and redirection, you can build up complicated programs extremely easily! This takes a lot of practice, however, so play around with redirection and see what happens!

– [onion] chesse

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s