Chapter 10. Commands in a Pipe

A selection of commands can be connected together with a pipe. This means the output of one command is the input for the next command. It is useful for filtering text from a variable or file and writing the result in a variable or file.

Changing a timestamp

This example shows how you can change the timestamp in a file. It is done in-place.

    all:
        :print Setting date in foobar.txt.
        :cat foobar.txt
              | :eval re.sub('Last Change: .*\n', 'Last Change: ' + DATESTR + '\n', stdin)
              >! foobar.txt

Lets see how this works:

    % cat foobar.txt
    This is example text for the A-A-P tutorial.
    Last Change: 2002 Feb 29
    The useful contents would start here.
    % aap
    Setting date in foobar.txt.
    % cat foobar.txt
    This is example text for the A-A-P tutorial.
    Last Change: 2002 Oct 21
    The useful contents would start here.
    %

The last command in the example consists of three parts. First comes the :cat command. It reads the "foobar.txt" file and passes it throught the pipe to the next command. "cat" is short for "concatenate". This is one of the good-old Unix commands that actually does much more than the name suggests. In this example nothing is concatenated. Below you will see examples where it does.

The second part of the example is the :eval command. This is used to read the text coming in through the pipe and modify it with a Python expression. In this case the expression is a "re.sub()" function call. This Python function takes three arguments: A pattern, a replacement string and the text to operate on. All occurences of the pattern in the text are changed to the replacement string. The pattern "Last Change: .*\n" matches a line with the date that was inserted previously. The replacement string contains DATESTR, which is an Aap variable that contains today's date as a string, e.g., "2002 Oct 19". The text to operate on is stdin. This is the variable that holds the text that is coming in through the pipe.

The third and last part >! foobar.txt redirects the output of the :eval command back to the file "foobar.txt". Using just ">" would cause an error, since the file already exists.

Note that in a Unix shell command this pipe would not work: The "foobar.txt" would be overwritten before it was read. In Aap this does not happen, the commands in the pipe are executed one by one. That makes it easier to use, but it does mean the text is kept in memory. Don't use pipes for a file that is bigger than half the memory you have available.

Changing a file in-place has the disadvantage that the normal dependencies don't work, since there is no separate source and target file. Often it is better to use a file "foobar.txt.in" as source, change it like in the example above and write it as a new file. The recipe would be:

    foobar.txt: foobar.txt.in
        :print Setting date in $target.
        :cat $source
                | :eval re.sub('Last Change: .*\n', 'Last Change: ' + DATESTR + '\n', stdin)
                >! $target

Creating a file from pieces

Sometimes you need to generate a file from several pieces. Here is an example that concatenates two files and puts a generated text line in between.

    manual.html: body.html footer.html
        @import time
        :eval time.strftime("%A %d %B %Y", time.localtime(time.time()))
                 | :print $(lt)BR$(gt)Last updated: $stdin$BR
                 | :cat body.html - footer.html >! $target

There are quite a few items here that need to be explained. First of all, the "@import time" line. This is a Python command to load the "time" module. So far we used modules that Aap has already loaded for you. This one isn't, and since we use the "time" module in the next :eval command it needs to be loaded explicitly.

The Python function "strftime()" formats the date and time in a specified format. See the Python documentation for the details. In this case the resulting string looks like "Monday 21 October 2002".

The output of the :eval command is piped into a :print command. The variable stdin contains the output of the previous command. Note that "$(lt)" is used instead of "$lt". The meaning is exactly the same: the value of the lt variable. Without the extra parenthesis it would read "$ltBR", which would be the value of the "ltBR" variable.

The resulting text is:

        <BR>Last updated: Monday 21 October 2002\n

Note that the first "BR" is the HTML code for a line break, while the "$Br" at the end is the Aap variable that contains a line break (here displayed as "\n").

Finally, the :cat command concatenates the file "body.html", the output of the :print command and the file "footer.html". Thus the "-" stands for where the pipe input is used. The result is redirected to target, which is "manual.html".

Pipe output in a variable

The generated date in the previous example could be used elsewhere in the recipe. Since we don't want to repeat a complicated expression the result of the :eval command should be redirected to a variable, like this:

    @import time
    :eval time.strftime("%A %d %B %Y", time.localtime(time.time()))
                | :assign Datestamp

    manual.html: body.html footer.html
           :print $(lt)BR$(gt)Last updated: $Datestamp$Br
                 | :cat body.html - footer.html >! $target

The :assign command takes the input from the pipe and puts it in the variable mentioned as its argument, which is "Datestamp" here. Actually, the same can be done with a normal assignment and a Python expression in backticks, but we intentionally wanted to show using a pipe here.

Creating a file from scratch

It is also possible to completely generate a file from scratch. Here is an example that generates a C header file:

1    :include config.aap
2    pathdef.c: config.aap
3       :print Creating $target
4       :print >! $target /* pathdef.c */
5       :print >> $target /* This file is automatically created by main.aap */
6       :print >> $target /* DO NOT EDIT!  Change main.aap only. */
7       :print >> $target $#include "vim.h"
8       :print >> $target char_u *default_vim_dir = (char_u *)"$VIMRCLOC";
9       :print >> $target char_u *all_cflags = (char_u *)"$CC -c -I$srcdir $CFLAGS";

The first :print command displays a message, so that it's clear "pathdef.c" is being generated. The next line contains ">!" to overwrite an existing file. It doesn't matter if the file already existed or not, it now only contains the line "/* pathdef.c */". The third and following lines contain ">>". This will cause each line to be appended to "pathdef.c".

In the example the VIMRCLOC and srcdir variables are defined in the recipe "config.aap". That is why this file is used as a source in the dependency. Also note the use of "$#" in line 7. Since "#" normally starts a comment it cannot be used directly here. "$#" is a special item that results in a "#" in the :print output. This is the resulting file:

       /* pathdef.c */
       /* This file is automatically created by main.aap */
       /* DO NOT EDIT!  Change main.aap only. */
       #include "vim.h"
       char_u *default_vim_dir = (char_u *)"/usr/local/share/vim61";
       char_u *all_cflags = (char_u *)"cc -c -I. -g -O2";

The list of ">>" redirections is quite verbose. Fortunately there is a shorter way:

1    :include config.aap
2    pathdef.c: config.aap
3        :print Creating $target
4        text << EOF
5            /* pathdef.c */
6            /* This file is automatically created by main.aap */
7            /* DO NOT EDIT!  Change main.aap only. */
8            $#include "vim.h"
9            char_u *default_vim_dir = (char_u *)"$VIMRCLOC";
10           char_u *all_cflags = (char_u *)"$CC -c -I$srcdir $CFLAGS";
11               EOF
12       :print $text >! $target

In line 4 "text << EOF" is used. This is called a block assignment. The following lines, up to the matching "EOF" line, are assigned to the variable text. You can use something else than "EOF" if you want to. It must be a word that does not appear inside of the text as a line on its own. White space before and after the word is ignored.

The indent of the text in the block assignment is removed. The indent of the first line is used, the same amount of indent is removed from the following lines. Thus if the second line has two more spaces worth of indent than the first line, it will have an indent of two spaces in the result. Half a tab is replace with four spaces when necessary (a tab always counts for up to eight spaces).