In my very first Linux Learning Series post, I outlined one of the guiding philosophies of UNIX:
Write programs to handle text streams, because that is a universal interface.
This post will cover how to work with text streams of different forms.
Text Editors
The first and most obvious form of text is a file with a text stream saved inside. This is the form most of us are used to, so we will start here.
Basic Text Viewing
First, open a terminal and enter the following:
devil@ubuntuVM:~$ sudo apt install libtext-lorem-perl
devil@ubuntuVM:~$ lorem -p 3 > sample.txt
The first command installs a small Perl module called lorem, which will output a bunch of Latin-looking text called Lorem Ipsum. The \> symbol is known as an I/O redirection operator, which I will cover in greater detail later. For now, you only need to know that lorem -p 3 will generate three paragraphs of Lorem Ipsum, and the > operator will save the output to a new file named sample.txt.
The file extension .txt is not required, but it’s what many of you are familiar with, so I have reused it here. Now, with this sample.txt file full of text, we can work with it.
There are a handful of basic CLI utilities for viewing text. The simplest is cat, which will print all of the text within a file. For example:
devil@ubuntuVM:~$ cat sample.txt
Molestiae ipsa delectus nulla. Quam laudantium dolores et voluptate rerum id dolor veritatis. Est fugiat quisquam similique rerum totam.
Ut consequatur aut ab molestias est ut. Et quaerat optio consequatur aliquid iure. Sit temporibus dolores laboriosam.
Eum delectus aliquam nesciunt a consequatur reprehenderit. Est aut dolores sit et quia. Rerum incidunt earum vel in consequatur aspernatur dolorum. Nihil vel deleniti omnis recusandae et. Veniam nobis officiis blanditiis est ex fuga omnis. Incidunt et voluptatem quod tempora voluptates quasi qui.
The file was randomly generated, so yours likely looks different. That’s OK, this is just for demonstration purposes.
Another useful tool is less, which performs similarly to cat except with a scrolling feature.
devil@ubuntuVM:~$ less sample.txt
This will display a footer with sample.txt (END) if the file is short, or a number of lines if the file is long. By default, it will display up to a full terminal’s height worth of text and wait for a keyboard input (up/down, pgup/pgdown, home/end, etc.).
To display the beginning of a file, use head. To display a certain number of lines, use the -n option.
To display the end of the file, use tail. tail also has the useful -f option which will “follow” a file and display updates as they occur. Very useful for watching log files as they fill.
Basic Text Editing
The simplest CLI tool for text editing is nano.
It is installed by default with Ubuntu, so you can use it straight away.
Open the sample.txt file using nano and look around:
devil@ubuntuVM:~$ nano sample.txt
nano displays a footer with some common shortcuts. For example, “^X” will exit the program. By default, the CTRL key is represented by ^, so CTRL+X will exit. Use CTRL+G to see the built-in help and other shortcuts.
If you want to get really in-depth with CLI text editing, you can look up vim and emacs. You may even choose to join the holy war and choose a side! I personally use vim and am ready to do battle — you can find me on Twitter.
Basic Text Searching
To search within text, use grep.
devil@ubuntuVM:~$ grep -i ipsum sample.txt
Quo et eveniet illo molestiae illo enim est. Ipsum atque modi libero est id tenetur. Sed sit dolores ut rerum odit error quo. Aut sint sint dolorum quidem aut dolorem tenetur. Eligendi sunt perferendis molestias. Beatae sed et exercitationem dolores doloremque qui beatae.
Text Streams
Most CLI programs in Linux are built around text streams. They are built to read from a special device called stdin (standard input) and write to a special device called stdout (standard output). Any errors are written to stderr (standard error).
Since these programs use a standard input and output setup, they can be connected together using redirection operators.
Redirection Operators
The basic redirection operators are:
|(pipe)<(input)>(output)2>(output to error)<<(append to input)>>(append to output)2>>(append to error)
That’s a lot of weird looking stuff, so we’ll go one-by-one.
Pipe Operator
The pipe operator | is entered with SHIFT+\ on a Standard US keyboard. It should be right beneath the Backspace key.
| allows you to send the output of a program (using stdout) to the input of another program (using stdin). Here’s an example:
devil@ubuntuVM:~$ ls | less
You will see something similar to:
Desktop
Documents
Downloads
Music
Pictures
Public
sample.txt
Templates
Videos
(END)
Press “Q” to close this and return to the terminal.
What happened?
First, the ls program ran, which read and printed the contents of the current directory to stdout. Then the resulting text stream was redirected to the less program’s input (using stdin). So essentially, ls gave a stream of text data to less using the | operator. Then less ran normally using that stream, displaying it in your terminal and letting you use it.
You could do a similar thing using ls | nano - if you wanted to save or manipulate that stream.
(Autist’s note) — Some programs (like nano) default to reading a file instead of stdin, so they require a special option. - is the common shorthand for stdin or stdout depending on the program. In the previous example, ls | nano - allows nano to read from stdin, which receives the piped output from ls.
Input Operator
Instead of specifying a file for cat, we can use the input operator <.
devil@ubuntuVM:~$ cat < sample.txt
Et saepe laudantium cupiditate nobis optio mollitia. Eos eum rem vero similique ipsa aut. Numquam nesciunt aut error et nisi iste et.
Qui voluptas accusamus nam consequuntur recusandae dolorum dolore quia. Nihil ipsam autem ut quas facere rerum. Praesentium dolores enim animi assumenda quo voluptate deserunt. Amet sit at perferendis perferendis eaque. Autem corporis est dolores.
Quo et eveniet illo molestiae illo enim est. Ipsum atque modi libero est id tenetur. Sed sit dolores ut rerum odit error quo. Aut sint sint dolorum quidem aut dolorem tenetur. Eligendi sunt perferendis molestias. Beatae sed et exercitationem dolores doloremque qui beatae.
Output Operator
The output operator can redirect an output stream to a file. Let’s say I wanted to get the first 2 lines from sample.txt, and save it to a new file. I could edit it with nano and save to a new filename, or do it entirely in one CLI command using redirection.
devil@ubuntuVM:~$ head -n2 sample.txt > new.txt
devil@ubuntuVM:~$ cat new.txt
Et saepe laudantium cupiditate nobis optio mollitia. Eos eum rem vero similique ipsa aut. Numquam nesciunt aut error et nisi iste et.
Error Operator
This is an advanced topic that we usually don’t need to know. It’s useful if you want to capture or redirect errors via /dev/stderr.
Append Operators
I rarely use the input append operator (<<), but the output append operator (>>) is very useful.
The output operator > will overwrite all text in a file each time it is used. The append operator >> will add the text to the end of the existing file. This is very useful if you want to store the results of multiple commands to a single file without juggling multiple temporary files.
Example — Let’s say I want to store the current contents of a directory and a date stamp:
devil@ubuntuVM:~$ date >> temp.txt
devil@ubuntuVM:~$ ls >> temp.txt
devil@ubuntuVM:~$ cat temp.txt
Fri 25 Jun 2021 11:30:05 AM PDT
Desktop
Documents
Downloads
Music
new.txt
Pictures
Public
sample.txt
Templates
temp.txt
Videos
date runs, then the output is redirected to temp.txt. Then ls runs, and its output is redirected to the end of temp.txt.
Note that you can use >> to append to a new file, even if it doesn’t exist. It behaves just like > in that case, you can use it freely without having to keep track of the first/second commands.
Turbo Advanced Topic — Multiple Redirections
You can string these redirections together, and it’s quite common to do so. Here’s a fun but fairly useless example.
devil@ubuntuVM:~$ ls
Desktop Downloads Pictures sample.txt temp.txt Documents
Music new.txt Public Templates Videos
devil@ubuntuVM:~$ ls | grep txt
new.txt
sample.txt
temp.txt
devil@ubuntuVM:~$ ls | grep txt | tr txt TXT
new.TXT
sample.TXT
Temp.TXT
devil@ubuntuVM:~$ ls | grep txt | tr txt TXT >> crazy.txt
The only new thing here is the tr command, which will replace all instances of “txt” with “TXT”. Useful if you’re trying to replace words or change capitalization, etc.