ashchristopher said:
Another useful redirection is stderr to stdout.
For example:
someScriptThatOutputsToSTDERR.rb 2> /tmp/all.output
or
someScriptThatOutputsToSTDERR.rb 2> /dev/null (*very useful for cronjobs)
The Shell - Advanced Commands
Wildcards
Some characters (called wildcards) mean special things to the shell. For example, * matches zero or more characters, so ls bio/*.txt lists all the text files in the bio directory:
$ ls bio/*.txt
bio/albus.txt bio/ginny.txt bio/harry.txt bio/hermione.txt
The wildcard ? matches any single character, so ls jan-??.txt lists text files whose names start with “jan-” followed by two characters. You can probably guess what ls jan-??.* does.
Note that the shell is the program that expands wildcards, not individual applications, so ls can't tell whether it was invoked as ls *.txt or as ls earth.txt venus.txt. Also, wildcards only work with filenames, not with command names, so ta* does not find the tabulate command.
Redirecting Input and Output
When a program is running, it is called a process. Every process automatically has three connections to the outside world:
• Standard input (stdin): connected to the keyboard
• Standard output (stdout): connected to the screen
• Standard error (stderr): also connected to the screen (but used for error messages)
To change the program behaviour, you can tell the shell to connect standard input and standard output to files instead. The command < input_file reads from input_file instead of from the keyboard. However, you don't need to use this very often, since most Unix commands let you specify the input file (or files) as command-line arguments.
The command > output_file writes to output_file instead of to the screen. Only “normal” output goes to the file, not error messages. You can also use both redirection operators simultaneously:
command < input_file > output_file
Note that redirection takes effect command-by-command, rather than permanently.
Redirection Examples
Save the number of words in all text files to a file called words.len:
$ cd bio
$ wc *.txt > words.len
Nothing appears on the screen because the output is being sent to the file words.len. We can check the contents using cat:
$ cat words.len
7 66 468 albus.txt
5 46 311 ginny.txt
5 49 342 harry.txt
5 49 331 hermione.txt
6 54 364 ron.txt
28 264 1816 total
Try typing cat > junk.txt. No input file is specified, so cat reads from the keyboard, and output is sent to a file -- the world's dumbest text editor. When you're done, use rm junk.txt to get rid of the file. Don't type rm * unless you're really, really sure that's what you want to do…
Don't redirect out to the same file you are reading from, e.g. sort words > words. Because the shell sets up redirection before running the command, redirecting out to an existing file truncates it and makes it empty. sort then goes and reads the empty file, and the contents of words are lost.
Pipes
Suppose you want to use the output of one program as the input of another. For example, if you want to use wc -w *.txt to count the words in some files, then sort -n to sort them numerically. The obvious solution is to send output of the first command to a temporary file, then read from that file:
$ wc -w *.txt > words.tmp
$ sort -n words.tmp
46 ginny.txt
49 harry.txt
49 hermione.txt
54 ron.txt
66 albus.txt
264 total
$ rm words.tmp
The right way is to use a pipe, written as "|". This tells the operating system to connect the standard output of the first program to the standard input of the second. This gives a solution that is both more convenient and less error prone than temporary files.
$ wc -w *.txt | sort -n
46 ginny.txt
49 harry.txt
49 hermione.txt
54 ron.txt
66 albus.txt
264 total
You can chain any number of commands together. Or you can combine pipes with input and output redirection:
$ grep 'Title' spells.txt | sort | uniq -c | sort -n -r | head -10 > popular_spells.txt
Any program that reads from standard input and writes to standard output can use redirection and pipes. Programs that do this are often called filters. If your programs work like filters, you (and other people) can combine them with standard tools.
