{"id":198,"date":"2018-10-10T08:44:00","date_gmt":"2018-10-10T00:44:00","guid":{"rendered":"https:\/\/yeslq.com\/201810198.html"},"modified":"2019-05-07T10:43:19","modified_gmt":"2019-05-07T02:43:19","slug":"the-linux-command-line-expansion-2","status":"publish","type":"post","link":"https:\/\/yeslq.com\/?p=198","title":{"rendered":"The Linux Command Line—Expansion"},"content":{"rendered":"

\nThe Linux Command Line <\/a><\/i>by William Shotts<\/span><\/h1>\n

<\/p>\n

\nEach time you type a command line and press the enter key, bash performs several processes upon the text before it carries out your command. We have seen a couple of cases of how a simple character sequence, for example \u201c*\u201d, can have a lot of meaning to the shell. The process that makes this happen is called expansion<\/i>. With expansion, you type something and it is expanded into something else before the shell acts upon it. To demonstrate what we mean by this, let’s take a look at the echo<\/a> command. echo is a shell builtin that performs a very simple task. It prints out its text arguments on standard output:<\/div>\n
\n[me@linuxbox me]$ echo this is a test
\nthis is a test<\/div>\n
\nThat’s pretty straightforward. Any argument passed to echo gets displayed. Let’s try another example:<\/div>\n
\n[me@linuxbox me]$ echo *
\nDesktop Documents ls-output.txt Music Pictures Public Templates Videos<\/div>\n
\nSo what just happened? Why didn’t echo print \u201c*\u201d? As you recall from our work with wildcards, the \u201c*\u201d character means match any characters in a filename, but what we didn’t see in our original discussion was how the shell does that. The simple answer is that the shell expands the \u201c*\u201d into something else (in this instance, the names of the files in the current working directory) before the echo command is executed. When the enter key is pressed, the shell automatically expands any qualifying characters on the command line before the command is carried out, so the echo command never saw the \u201c*\u201d, only its expanded result. Knowing this, we can see that echo behaved as expected.<\/div>\n

\nPathname Expansion<\/span><\/h2>\n
\nThe mechanism by which wildcards work is called pathname expansion<\/i>. If we try some of the techniques that we employed in our earlier lessons, we will see that they are really expansions. Given a home directory that looks like this:<\/div>\n
\n[me@linuxbox me]$ ls<\/p>\n
Desktop\nls-output.txt\nDocuments Music\nPictures\nPublic\nTemplates\nVideos<\/pre>\n<\/div>\n
\nwe could carry out the following expansions:<\/div>\n
\n[me@linuxbox me]$ echo D*
\nDesktop Documents<\/div>\n
\nand:<\/div>\n
\n[me@linuxbox me]$ echo *s
\nDocuments Pictures Templates Videos<\/div>\n
\nor even:<\/div>\n
\n[me@linuxbox me]$ echo [[:upper:]]*
\nDesktop Documents Music Pictures Public Templates Videos<\/div>\n
\nand looking beyond our home directory:<\/div>\n
\n[me@linuxbox me]$ echo \/usr\/*\/share
\n\/usr\/kerberos\/share \/usr\/local\/share<\/div>\n
\nTilde Expansion<\/span><\/h2>\n
\nAs you may recall from our introduction to the cd command, the tilde character (\u201c~\u201d) has a special meaning. When used at the beginning of a word, it expands into the name of the home directory of the named user, or if no user is named, the home directory of the current user:<\/div>\n
\n[me@linuxbox me]$ echo ~
\n\/home\/me<\/div>\n
\nIf user \u201cfoo\u201d has an account, then:<\/div>\n
\n[me@linuxbox me]$ echo ~foo
\n\/home\/foo<\/div>\n
\nArithmetic Expansion<\/span><\/h2>\n
\nThe shell allows arithmetic to be performed by expansion. This allow us to use the shell prompt as a calculator:<\/div>\n
\n[me@linuxbox me]$ echo $((2 + 2))
\n4<\/div>\n
\nArithmetic expansion uses the form:<\/div>\n
 $((expression))<\/pre>\n
\nwhere expression is an arithmetic expression consisting of values and arithmetic operators.<\/div>\n
\nArithmetic expansion only supports integers (whole numbers, no decimals), but can perform quite a number of different operations.<\/div>\n
\nSpaces are not significant in arithmetic expressions and expressions may be nested. For example, to multiply five squared by three:<\/div>\n
\n[me@linuxbox me]$ echo $(($((5**2)) * 3))
\n75<\/div>\n
\nSingle parentheses may be used to group multiple subexpressions. With this technique, we can rewrite the example above and get the same result using a single expansion instead of two:<\/div>\n
\n[me@linuxbox me]$ echo $(((5**2) * 3))
\n75<\/div>\n
\nHere is an example using the division and remainder operators. Notice the effect of integer division:<\/div>\n
\n[me@linuxbox me]$ echo Five divided by two equals $((5\/2))
\nFive divided by two equals 2
\n[me@linuxbox me]$ echo with $((5%2)) left over.
\nwith 1 left over.<\/div>\n
\nBrace Expansion<\/span><\/h2>\n
\nPerhaps the strangest expansion is called brace expansion<\/i>. With it, you can create multiple text strings from a pattern containing braces. Here’s an example:<\/div>\n
\n[me@linuxbox me]$ echo Front-{A,B,C}-Back
\nFront-A-Back Front-B-Back Front-C-Back<\/div>\n
\nPatterns to be brace expanded may contain a leading portion called a preamble<\/i> and a trailing portion called a postscript<\/i>. The brace expression itself may contain either a comma-separated list of strings, or a range of integers or single characters. The pattern may not contain embedded whitespace. Here is an example using a range of integers:<\/div>\n
\n[me@linuxbox me]$ echo Number_{1..5}
\nNumber_1 Number_2 Number_3 Number_4 Number_5<\/div>\n
\nA range of letters in reverse order:<\/div>\n
\n[me@linuxbox me]$ echo {Z..A}
\nZ Y X W V U T S R Q P O N M L K J I H G F E D C B A<\/div>\n
\nBrace expansions may be nested:<\/div>\n
\n[me@linuxbox me]$ echo a{A{1,2},B{3,4}}b
\naA1b aA2b aB3b aB4b<\/div>\n
\nSo what is this good for? The most common application is to make lists of files or directories to be created. For example, if you were a photographer and had a large collection of images you wanted to organize into years and months, the first thing you might do is create a series of directories named in numeric \u201cYear-Month\u201d format. This way, the directory names will sort in chronological order. You could type out a complete list of directories, but that’s a lot of work and it’s error-prone too. Instead, you could do this:<\/div>\n
\n[me@linuxbox me]$ mkdir Photos
\n[me@linuxbox me]$ cd Photos
\n[me@linuxbox Photos]$ mkdir {2007..2009}-0{1..9} {2007..2009}-{10..12}
\n[me@linuxbox Photos]$ ls<\/p>\n
2007-01 2007-07 2008-01 2008-07 2009-01 2009-07\n2007-02 2007-08 2008-02 2008-08 2009-02 2009-08\n2007-03 2007-09 2008-03 2008-09 2009-03 2009-09\n2007-04 2007-10 2008-04 2008-10 2009-04 2009-10\n2007-05 2007-11 2008-05 2008-11 2009-05 2009-11\n2007-06 2007-12 2008-06 2008-12 2009-06 2009-12<\/pre>\n<\/div>\n
\nPretty slick!<\/div>\n
\nParameter Expansion<\/span><\/h2>\n
\nWe’re only going to touch briefly on parameter expansion<\/i> in this lesson, but we’ll be covering it more later. It’s a feature that is more useful in shell scripts than directly on the command line. Many of its capabilities have to do with the system’s ability to store small chunks of data and to give each chunk a name. Many such chunks, more properly called variables<\/i>, are available for your examination. For example, the variable named \u201cUSER\u201d contains your user name. To invoke parameter expansion and reveal the contents of USER you would do this:<\/div>\n
\n[me@linuxbox me]$ echo $USER
\nme<\/div>\n
\nTo see a list of available variables, try this:<\/div>\n
\n[me@linuxbox me]$ printenv | less<\/div>\n
\nYou may have noticed that with other types of expansion, if you mistype a pattern, the expansion will not take place and the echo command will simply display the mistyped pattern. With parameter expansion, if you misspell the name of a variable, the expansion will still take place, but will result in an empty string:<\/div>\n
\n[me@linuxbox me]$ echo $SUER
\n[me@linuxbox ~]$<\/div>\n
\nCommand Substitution<\/span><\/h2>\n
\nCommand substitution<\/i> allows us to use the output of a command as an expansion:<\/div>\n
\n[me@linuxbox me]$ echo $(ls)
\nDesktop Documents ls-output.txt Music Pictures Public Templates Videos<\/div>\n
\nOne of my favorites goes something like this:<\/div>\n
\n[me@linuxbox me]$ ls -l $(which cp)
\n-rwxr-xr-x 1 root root 71516 2007-12-05 08:58 \/bin\/cp<\/div>\n
\nHere we passed the results of which cp as an argument to the ls command, thereby getting the listing of of the cp program without having to know its full pathname. We are not limited to just simple commands. Entire pipelines can be used (only partial output shown):<\/div>\n
\n[me@linuxbox me]$ file $(ls \/usr\/bin\/* | grep bin\/zip)<\/p>\n
\/usr\/bin\/bunzip2:\n\/usr\/bin\/zip:      ELF 32-bit LSB executable, Intel 80386, version 1 \n(SYSV), dynamically linked (uses shared libs), for GNU\/Linux 2.6.15, stripped\n\/usr\/bin\/zipcloak: ELF 32-bit LSB executable, Intel 80386, version 1\n(SYSV), dynamically linked (uses shared libs), for GNU\/Linux 2.6.15, stripped\n\/usr\/bin\/zipgrep:  POSIX shell script text executable\n\/usr\/bin\/zipinfo:  ELF 32-bit LSB executable, Intel 80386, version 1\n(SYSV), dynamically linked (uses shared libs), for GNU\/Linux 2.6.15, stripped\n\/usr\/bin\/zipnote:  ELF 32-bit LSB executable, Intel 80386, version 1\n(SYSV), dynamically linked (uses shared libs), for GNU\/Linux 2.6.15, stripped\n\/usr\/bin\/zipsplit: ELF 32-bit LSB executable, Intel 80386, version 1\n(SYSV), dynamically linked (uses shared libs), for GNU\/Linux 2.6.15, stripped<\/pre>\n<\/div>\n
\nIn this example, the results of the pipeline became the argument list of the file command. There is an alternate syntax for command substitution in older shell programs which is also supported in bash. It uses back-quotes instead of the dollar sign and parentheses:<\/div>\n
\n[me@linuxbox me]$ ls -l `which cp`
\n-rwxr-xr-x 1 root root 71516 2007-12-05 08:58 \/bin\/cp<\/div>\n
\nQuoting<\/span><\/h2>\n
\nNow that we’ve seen how many ways the shell can perform expansions, it’s time to learn how we can control it. Take for example:<\/div>\n
\n[me@linuxbox me]$ echo this is a     test
\nthis is a test<\/div>\n
\nor:<\/div>\n
\n[me@linuxbox me]$ [me@linuxbox ~]$ echo The total is $100.00
\nThe total is 00.00<\/div>\n
\nIn the first example, word-splitting by the shell removed extra whitespace from the echo command’s list of arguments. In the second example, parameter expansion substituted an empty string for the value of \u201c$1\u201d because it was an undefined variable. The shell provides a mechanism called quoting to selectively suppress unwanted expansions.<\/div>\n
\nDouble Quotes<\/span><\/h2>\n
\nThe first type of quoting we will look at is double quotes. If you place text inside double quotes, all the special characters used by the shell lose their special meaning and are treated as ordinary characters. The exceptions are \u201c$\u201d, \u201c\u201d (backslash), and \u201c`\u201d (back- quote). This means that word-splitting, pathname expansion, tilde expansion, and brace expansion are suppressed, but parameter expansion, arithmetic expansion, and command substitution are still carried out. Using double quotes, we can cope with filenames containing embedded spaces. Say you were the unfortunate victim of a file called two words.txt. If you tried to use this on the command line, word-splitting would cause this to be treated as two separate arguments rather than the desired single argument:<\/div>\n
\n[me@linuxbox me]$ ls -l two words.txt<\/p>\n
ls: cannot access two: No such file or directory\nls: cannot access words.txt: No such file or directory<\/pre>\n<\/div>\n
\nBy using double quotes, you can stop the word-splitting and get the desired result; further, you can even repair the damage:<\/div>\n
\n[me@linuxbox me]$ ls -l “two words.txt”
\n-rw-rw-r– 1 me me 18 2008-02-20 13:03 two words.txt
\n[me@linuxbox me]$ mv “two words.txt” two_words.txt<\/div>\n
\nThere! Now we don’t have to keep typing those pesky double quotes. Remember, parameter expansion, arithmetic expansion, and command substitution still take place within double quotes:<\/div>\n
\n[me@linuxbox me]$ echo “$USER $((2+2)) $(cal)”<\/p>\n
me 4\nFebruary 2008\nSu Mo Tu We Th Fr Sa\n                1  2\n 3  4  5  6  7  8  9\n10 11 12 13 14 15 16\n17 18 19 20 21 22 23\n24 25 26 27 28 29<\/pre>\n<\/div>\n
\nWe should take a moment to look at the effect of double quotes on command substitution. First let’s look a little deeper at how word splitting works. In our earlier example, we saw how word-splitting appears to remove extra spaces in our text:<\/div>\n
\n[me@linuxbox me]$ echo this is a     test
\nthis is a test<\/div>\n
\nBy default, word-splitting looks for the presence of spaces, tabs, and newlines (linefeed characters) and treats them as delimiters between words. This means that unquoted spaces, tabs, and newlines are not considered to be part of the text. They only serve as separators. Since they separate the words into different arguments, our example command line contains a command followed by four distinct arguments. If we add double quotes:<\/div>\n
\n[me@linuxbox me]$ echo “this is a     test”
\nthis is a     test<\/div>\n
\nword-splitting is suppressed and the embedded spaces are not treated as delimiters, rather they become part of the argument. Once the double quotes are added, our command line contains a command followed by a single argument. The fact that newlines are considered delimiters by the word-splitting mechanism causes an interesting, albeit subtle, effect on command substitution. Consider the following:<\/div>\n
\n[me@linuxbox me]$ echo $(cal)
\nFebruary 2008 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
\n[me@linuxbox me]$ echo “$(cal)”<\/p>\n
February 2008\nSu Mo Tu We Th Fr Sa\n                1  2\n 3  4  5  6  7  8  9\n10 11 12 13 14 15 16\n17 18 19 20 21 22 23\n24 25 26 27 28 29\n<\/pre>\n<\/div>\n
\nIn the first instance, the unquoted command substitution resulted in a command line containing thirty-eight arguments. In the second, a command line with one argument that includes the embedded spaces and newlines.<\/div>\n
\nSingle Quotes<\/span><\/h2>\n
\nIf you need to suppress all expansions, you use single quotes. Here is a comparison of unquoted, double quotes, and single quotes:<\/div>\n
\n[me@linuxbox me]$ echo text ~\/*.txt {a,b} $(echo foo) $((2+2)) $USER
\ntext \/home\/me\/ls-output.txt a b foo 4 me
\n[me@linuxbox me]$ echo “text ~\/*.txt {a,b} $(echo foo) $((2+2)) $USER”
\ntext ~\/*.txt {a,b} foo 4 me
\n[me@linuxbox me]$ echo ‘text ~\/*.txt {a,b} $(echo foo) $((2+2)) $USER’
\ntext ~\/*.txt {a,b} $(echo foo) $((2+2)) $USER<\/div>\n
\nAs you can see, with each succeeding level of quoting, more and more of the expansions are suppressed.<\/div>\n
\nEscaping Characters<\/span><\/h2>\n
\nSometimes you only want to quote a single character. To do this, you can precede a character with a backslash, which in this context is called the escape character<\/i>. Often this is done inside double quotes to selectively prevent an expansion:<\/div>\n
\n[me@linuxbox me]$ echo “The balance for user $USER is: $5.00”
\nThe balance for user me is: $5.00<\/div>\n
\nIt is also common to use escaping to eliminate the special meaning of a character in a filename. For example, it is possible to use characters in filenames that normally have special meaning to the shell. These would include \u201c$\u201d, \u201c!\u201d, \u201c&\u201d, \u201c \u201c, and others. To include a special character in a filename you can to this:<\/div>\n
\n[me@linuxbox me]$ mv bad&filename good_filename<\/div>\n
\nTo allow a backslash character to appear, escape it by typing \u201c\u201d. Note that within single quotes, the backslash loses its special meaning and is treated as an ordinary character.<\/div>\n
\nMore Backslash Tricks<\/span><\/h2>\n
\nIf you look at the man pages for any program written by the GNU project<\/a>, you will notice that in addition to command line options consisting of a dash and a single letter, there are also long option names that begin with two dashes. For example, the following are equivalent:<\/div>\n
\n
ls -r\nls --reverse\n       \n<\/pre>\n<\/div>\n
\nWhy do they support both? The short form is for lazy typists on the command line and the long form is mostly for scripts though some options may only be long form. I sometimes use obscure options, and I find the long form useful if I have to review a script again months after I wrote it. Seeing the long form helps me understand what the option does, saving me a trip to the man page. A little more typing now, a lot less work later. Laziness is maintained.<\/div>\n
\nAs you might suspect, using the long form options can make a single command line very long. To combat this problem, you can use a backslash to get the shell to ignore a newline character like this:<\/div>\n
\n
ls -l \n   --reverse \n   --human-readable \n   --full-time\n       \n<\/pre>\n<\/div>\n
\nUsing the backslash in this way allows us to embed newlines in our command. Note that for this trick to work, the newline must be typed immediately after the backslash. If you put a space after the backslash, the space will be ignored, not the newline. Backslashes are also used to insert special characters into our text. These are called backslash escape characters<\/i>. Here are the common ones:<\/div>\n\n\n\n\n\n\n\n\n
Escape Character<\/strong><\/td>\nName<\/strong><\/td>\nPossible Uses<\/strong><\/td>\n<\/tr>\n
n<\/td>\nnewline<\/td>\nAdding blank lines to text<\/td>\n<\/tr>\n
t<\/td>\ntab<\/td>\nInserting horizontal tabs to text<\/td>\n<\/tr>\n
a<\/td>\nalert<\/td>\nMakes your terminal beep<\/td>\n<\/tr>\n
<\/td>\nbackslash<\/td>\nInserts a backslash<\/td>\n<\/tr>\n
f<\/td>\nformfeed<\/td>\nSending this to your printer ejects the page<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
\nThe use of the backslash escape characters is very common. This idea first appeared in the C programming language. Today, the shell, C++, perl, python, awk, tcl, and many other programming languages use this concept. Using the echo command with the -e option will allow us to demonstrate:<\/div>\n
\n[me@linuxbox me]$ echo -e “Inserting several blank linesnnn”
\nInserting several blank lines<\/p>\n
[me@linuxbox me]$ echo -e “Wordstseparatedtbythorizontalttabs.”<\/p>\n
Words separated   by  horizontal  tabs\n<\/pre>\n
[me@linuxbox me]$ echo -e “aMy computer went “beep”.”<\/p>\n
My computer went “beep”.<\/p>\n
[me@linuxbox me]$ echo -e “DEL C:WIN2KLEGACY_OS.EXE”<\/p>\n
DEL C:WIN2KLEGACY_OS.EXE<\/p><\/div>\n
\n
\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"
The Linux Command Line by William Shotts Each time … \u9605\u8bfb\u66f4\u591a<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[51,52,53],"class_list":["post-198","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-linux","tag-shell","tag-the-linux-command-line"],"_links":{"self":[{"href":"https:\/\/yeslq.com\/index.php?rest_route=\/wp\/v2\/posts\/198","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/yeslq.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/yeslq.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/yeslq.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/yeslq.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=198"}],"version-history":[{"count":1,"href":"https:\/\/yeslq.com\/index.php?rest_route=\/wp\/v2\/posts\/198\/revisions"}],"predecessor-version":[{"id":305,"href":"https:\/\/yeslq.com\/index.php?rest_route=\/wp\/v2\/posts\/198\/revisions\/305"}],"wp:attachment":[{"href":"https:\/\/yeslq.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=198"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/yeslq.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=198"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/yeslq.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=198"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}