Month: September 2014

I/O Redirection and Pipes

One of the true powers of the shell is the ability to pipe and redirect input and output. With these two tools, the huge amounts of small tools that bash provides can be combined together into complex programs that would be hard to create otherwise.

In the simplest terms, a pipe is moving data from the output of one program to the input of another. In order to fully appreciate this concept, I must introduce two new commands: echo and grep. Echo simply prints its argument out on the command line, and grep is a search utility.

onionchesse@hunger:~$ echo -e "Hi!\nKittens are fun\nI like to eat onions\nAnd chesse"
Kittens are fun
I like to eat onions
And chesse

In this command there are two new concepts, arguments and newlines. The ‘-e’ on the command is the argument. In this case, the presence of the -e tells the command to enable ‘escape characters’ which allow us to put tabs and other hidden characters in the output. The ‘\n’ in the command is a ‘newline’, a character that ‘moves the cursor’ down a line. It causes the output to be printed out on multiple lines!

Now that we see how echo works, lets see how to use grep, to search for ‘chesse’

onionchesse@hunger:~$ echo -e "Hi\nKittens are fun\nI like to eat onions\nAnd chesse" | grep "chesse"
And chesse

Hey look! only the ‘And chesse’ line got printed out! Lets examine this  a little closer… The ‘|’ character is a ‘pipe’, it will send the output of the ‘echo’ program to the input of the ‘grep’ program. The grep program will then block any lines not containing ‘chesse’, as that is what we told it to search for. What if we only want it to print out ‘chesse’, instead of the entire line? We can use an argument to the grep command to do that!

onionchesse@hunger:~$ echo -e "Hi\nKittens are fun\nI like to eat onions\nAnd chesse" | grep -o "chesse"

The ‘-o’ argument tells grep to print ‘only matching’ strings, which is why we only got the ‘chesse’ this time!

However, we still have no idea how to write to files! The simplest way to do that is with output redirection, or the idea of sending the output of a program somewhere besides your terminal window. Lets see if we can put the output of the echo command into a file!

onionchesse@hunger:~$ echo -e "Hi\nKittens are fun\nI like to eat onions\nAnd chesse" > foodies.txt

The ‘>’ symbol redirects the output of the echo command to the file ‘foodies.txt’. Notice how there is no output on the screen! That is because the output got sent to a file instead. Lets ‘ls’ and try to find it!

onionchesse@hunger:~$ ls
Desktop Documents Downloads Dropbox foodies.txt Minecraft Music Pictures 
onionchesse@hunger:~$ cat foodies.txt
Kittens are fun
I like to eat onions
And chesse

The ls command clearly shows our new file, foodies.txt! We can ‘cat’ the file to find out the contents of it, and we find out what echo printed out earlier! However, the ‘>’ operator is dangerous, in that it overwrites any files already with the same name. We can prevent this with the ‘>>’ operator, which will append to the file instead! Lets go ahead and try it out on foodies.txt!

onionchesse@hunger:~$ echo "Do I like pie?" >> foodies.txt
onionchesse@hunger:~$ cat foodies.txt
Kittens are fun
I like to eat onions
And chesse
Do I like pie?

And we see clearly, that our output has been appended to the end of the file! Very useful, especially when you want to send debug output of a program to someone else.

This is only a tiny fraction of what is possible with bash, and by combining more tools using pipes and redirection, you can build up complicated programs extremely easily! This takes a lot of practice, however, so play around with redirection and see what happens!

– [onion] chesse


The Shell

When I say ‘computer’, most people think of this:

However, when I say ‘computer’ to a computer expert, they think about this:

Vastly different right? Every computer expert a least has the basic knowledge of command line usage. The reason for this is that the command line is the most basic, raw, and primal way of interfacing with a computer, that allows for more complete control and higher productivity. To run firefox in a command line, you type:

onionchesse@hunger:$ firefox

As opposed for looking for an icon or searching a large list of programs (i’m looking at you windows 8!). This simplicity and powerful nature of the command line leads to huge increases in productivity. By the way, these blocks indicate snippets of command line ‘code’, which do not include the stuff before the ‘$’. So in this case, you would type ‘firefox’ into your shell.

Because of the major inconsistencies between the original (unix) and the skewed (windows) command lines, I will be talking the unix shell, which is ‘bash’ by default on most Linux, and osX machines. (Once using both enough, you will see why). Bash is a free and opensource derivative of the original command line interface, the shell (or sh). Bash stands for ‘bourne-again shell’, which is extremely appropriate, given its history. This article will cover most of the basics of using a shell.

To start your shell, just open up a ‘terminal’ program on your computer. If you are running windows, you need to install cygwin to get this functionality.

When you start a shell, you are ‘placed’ in a directory on your computer. Think about opening a file explorer in windows, or finder on a mac, you are pointed at a location on your computer, like my documents or my programs (in windows). To find out where you are on your computer, type ‘pwd’:

onionchesse@hunger:~$ pwd

That output, the ‘/home/onionchesse’ is the current directory, or where I currently ‘am’. I can find out what files are in this location by using the ‘ls’ command:

onionchesse@hunger:~$ ls
bin Copy Desktop Documents Downloads Dropbox Minecraft Music Pictures Public Templates Videos

Hey, there are all my folders which are inside that ‘current directory’. Lets now try moving around! To move around, we use the ‘cd’ or change directory command:

onionchesse@hunger:~$ cd Minecraft
onionchesse@hunger:~/Minecraft$ ls
MagicLauncher_1.2.5.jar Minecraft.jar Mods TechnicLauncher.jar

The cd command moved us into the Minecraft folder, and ls listed the contents of the Minecraft folder, which has some cool jar files! (Pro Tip: You can press tab when typing out a directory name to autocomplete it if the name is unabiguous! Ex: ‘cd Mine<tab>’ completes to ‘cd Minecraft/’). Typing out ‘cd ..’ will take you back ‘up’ a directory, and typing ‘cd ~’ will take you to where you started! (This is called the ‘home directory’ on unix machines, and it is represented by the ‘~’). These little shortcuts add up until you can save yourself a ton of keystrokes.

Lets now create a text file! The ‘touch’ command creates a text file:

onionchesse@hunger:~$ touch MinecraftIsAwsome.txt
onionchesse@hunger:~$ ls
MagicLauncher_1.2.5.jar MinecraftIsAwsome.txt Minecraft.jar Mods TechnicLauncher.jar

Hey! MinecraftIsAwsome.txt showed up! Lets see what inside it! The ‘cat’ or concatenate command will print out files (as text) to the command line:

onionchesse@hunger:~$ cat MinecraftIsAwsome.txt

Oh no! Nothing got printed out! This is because there was nothing in the text file to begin with… We need to fill that text file with information to print out. Next week, i’ll talk about i/o redirection, which will allow us to easily write to files from the command line! In the meantime, open up Notepad, Word, or Microsoft Paint and edit that file the way you’re used to. Just make sure to save as a text file!

-[onion] chesse

A Tale of Software

Due to the apple unveiling event taking place tomorrow, I have decided to stray from my usual topic and talk a about the commercialization in software! This will also give insights into my future posts.

Last post, I talked about how computers had become tainted with overuse, and how all computers became poorly maintained over time. This is mainly due to the commercialization of software. In the beginning, only experts could use computers. One of the main reasons for this was the command line, a prompt where you would type commands to do everything on a computer. To use it, one had to remember commands, parameters to those commands, and pay a penalty for any mistyped commands. It was easy to ‘brick’ an entire system with a typo on the command line. With the invent of the graphical user interface (or GUI), a new era of computing started.

Now, suddenly, it was extremely easy to use a computer! One could simply click on a icon to run a program, rather than having to memorize commands. One could check boxes instead of memorize command line arguments! And most importantly, confirmations were added, to reduce the possibility of breaking your system.

Naturally, this kind of computing exploded, as literally anyone had access to the power of a computer, with no training. The corporations that developed these GUI’s exploded into the giants that we know today: Apple, and Microsoft. The command line, at least in windows, was a shunned tool, thrown away like a old sock. Computer ‘Experts’ became those who knew what all the little buttons do, instead of those who had spent years learning about the internals of a computer. The ‘stupid user’ became someone who had literally no prior experience with technology, maybe forced into a computing job. All possible because of the GUI.

However, the GUI gave companies the power to lock its users in. All the users had no idea about the internals of a computer, they ‘just wanted to get work done’. So naturally, the GUI’s constricted, giving less and less of those options that were needed by the true professionals, as the target audience was those who had no idea how computers work. Recently, the corporations have taken it even farther, by essentially forcing certain mandates on its users. Windows has many ‘features’ that limit a user’s freedom, from having the power to delete programs on its computers, to sending information about usage back to Microsoft. Apple actively fights users who are trying to gain freedom, by voiding warranty on ‘jailbroken’ products, limiting apps on the app-store, and placing backdoors in products available to Apple itself, giving them almost full control over every apple product. Of course, the users don’t care… They just want to ‘use a computer’. The professionals who were around before this revolution are warning of the consequences to accepting these products, but very few people are listening, as no one simply cares. All who ‘disagree’ are either ignorant to these changes being ‘marketed’ to them in plain sight, or are willing to trade usability and ease of use for their freedom.

In order to truly learn about computers, you need to bypass the restrictions that corporations put on you, so you can have full access to the wonderful machine you own, and the only way to do that is by utilizing free software. Because of this, I will mainly be showing you how to use free software, rather than proprietary alternatives, because proprietary software simply limits what is possible for a computer to do.

When you go down to the apple store and attempt to get that iPhone 6, think about how much freedom are you willing to give up. Do you even have the freedom to abstain from buying the new iPhone? Or are you just digging yourself a hole of reliance from which you have no possibility of escaping…

-[onion] chesse


The Computer

Once, Computers were a tool reserved for the very elite. They cost many thousands of dollars, were the size of a room, and required constant care to run properly. To use a Computer you would need to learn about every aspect of how a Computer would work. This fact, in combination with how catastrophic a simple error could be, lead to every Computer being extremely well maintained. Now, the only Computers that are maintained as well as those from our past are those of companies, who pay system administrators to take care of, use, and oversee use of a server…

With the rise of ‘easy computing’ came the rise of the famous, at least in IT, stupid user. Nowadays, anyone from any background can ‘use a Computer’, in fact, many of us spend most of our waking lives using one, be it for social media, gaming, or typing a report. Little ‘previous knowledge’ is required, unlike any other high level tool. Every tractor driver requires basic training, every pianist must learn how to play, and every machinist must learn how to use a milling machine and other tools, all processes which take years to accomplish. However, today we see even 1st graders being brought over to the ‘Computer lab’, and being told to type up essays, with little to no prior knowledge about Computers. The Computer, nowadays, is treated as a basic tool, on par with a pen, or a crowbar, but its internals are more complicated than any other gadget on the planet.

Almost every system in our age is one maintained by a person who is not a Computer expert, which means they are blind to the true power and glory of computing. They would say, “Oh, my Computer is getting slow, time to buy a new one”, or, “Any file that ends with a .txt is a text file”, or, “I need to go download some more ram“, all comments that are blissfully ignorant of the true power and capabilities of a Computer. In order to use any tool effectively, you need to learn the most effective way to use it, and how the tool works ‘under the hood’.

This blog is not for the ‘stupid users‘, nor is it for the ‘wizards‘. It is for those users who are amazed by the technology of the Computer, and willing to put in time, make sacrifices, and fix their own problems, in order to LEARN how a Computer actually works. If these people encounter an error they have never seen before, they will take the time to figure out what has gone wrong, why it went wrong, and the PROPER way to fix it.

Let us go forth, and find out what it REALLY means to check that ‘advanced users only’ box.

-[onion] chesse

DISCLAIMER: While it may appear that I know a lot about Computers, I really don’t. I just want to make it easier for people to go through what I struggled through for 4 long years: beginning to learn how to truly use a Computer. My journey is nowhere near complete, but I hope to help the few of you that will stick with me to get started and broaden everyone’s mind about The Computer!