A-A-P logo
NLnet logo

Documentation - Recipe file

A-A-P is still in an early phase. Therefore this document can only be used to get an idea of the current ideas of the recipe file format. Everything may still change!

The "hello world" example
The basics
A top level recipe
A ported application
Building multiple variants
Using Python in recipes
More about attributes
More about commands
Pipe commands
Automatic dependency checking
Multiple targets
Publishing a web site
Version control with CVS
Recipe syntax definition

The "hello world" example

Most programming languages start with a short example that prints a "hello world" message. This is how to compile such a program with A-A-P:
    SOURCE = hello.c
    TARGET = hello
This is all that A-A-P needs to know to compile "hello.c" into the program "hello". A-A-P will figure out what compiler is to be used for a ".c" file and run it with the default options.


The basics

When you have several files with source code you can specify them as a list:
    SOURCE =
    TARGET = myprogram
Notice that it is not necessary to use a line continuation character, as you would have to do in a Makefile. A-A-P knows that when a line is encountered with the same indent as the line the assignment started with, that must be the end of the list. The amount of indent for the continuation lines is irrelevant, so long as it's more than the indent of the first line.

Indents are very important, just like in a Python script. Make sure your tabstop is always set to the standard value of eight, otherwise you might run into trouble when mixing tabs and spaces.

Someone who sees this recipe would like to know what it's for. This requires adding comments. These start with a "#" character and extend until the end of the line (like in Makefiles and Python scripts).
It is also possible to associate a comment with a specific item:
    # A-A-P file to compile myprogram
    SOURCE =
   	    main.c	# startup stuff
	    version.c	# just the date stamp
	    help.c	# display help stuff

    TARGET = myprogram { comment = "build my program" }
Now run A-A-P with:

   aap comment
And the output will be:
   comments for main.aap:
	target myprogram: build my program
This way of associating a comment with an item, using curly braces, is called an attribute. Various attributes can be associated with items. Another example is to force compiling the "version.c" file each time, so that the date stamp it contains is always up-to-date:
    SOURCE =
	    version.c {force}
    TARGET = myprogram { comment = "build my program" }

On MS-Windows an executable ends in ".exe". Adding this to "myprogram" would result in a recipe that only works on MS-Windows. Better is to use the EXESUF variable, which contains ".exe" only when appropriate.
    SOURCE = main.c
    TARGET = myprogram$EXESUF
You can see that it's very simple to use a variable in an assignment. The dollar sign indicates a variable name is following.
If you might want to use a dollar in a file name, double the dollar sign. For example, to use the "$temp.c" file as a source file:
    SOURCE = $$temp.c
A variable name consists of ASCII letters, numbers and the underscore. If you want to append one of these after a variable name you must put parenthesis around the variable name, so that A-A-P knows where it ends:
    MAKENAME = Make


A top level recipe

A program usually requires different methods for compiling and installation, depending on the Operating System. A good way to handle this is to use one recipe at the top level, which decides what OS-specific recipe to use next. Here is an example that we will explain:
1	FTPBASE = ftp://ftp.vim.org/pub/vim  ftp://ftp.us.vim.org/pub/vim
2	CVSROOT = cvs://:pserver:anonymous@cvs.vim.org/vim {path = aap}
4	:recipe {origin = $FTPBASE/aap/main.aap $CVSROOT}
6	@if OSTYPE == "posix":
7	    :child unix.aap {origin = $FTPBASE/unix/vim.aap.gz $CVSROOT}
8	@elif OSTYPE == "mswin":
9	    :child pc.aap {origin = $FTPBASE/pc/vim.zip//vim.aap $CVSROOT}
10	@else:
11	    :error OS not supported: $OSTYPE
line 1: The URL of two ftp sites is specified where the files for this program can be found. Nothing special is added, thus anonymous ftp is assumed.
line 2: For CVS a semi-URL is used, starting with "cvs://". This will be explained below.

line 4: A line starting with ":" indicates an A-A-P command. This clearly separates them from Python lines, assignments, etc.
The ":recipe" command specifies the origin of the recipe itself. This is used to check if an updated version is available. A new version is only obtained when specifically asked for, not each time the recipe is executed.
The origin attribute specifies three locations. That is because the variable $FTPBASE contains two items. Using this variable with text appended results in two items, each with the text appended. After the expansion the command looks like this:
   :recipe {origin = ftp://ftp.vim.org/pub/vim/aap/main.aap
		     cvs://:pserver:anonymous@cvs.vim.org/vim {path = aap}}
Only one of the three URLs will be used. Normally it's the first one, but when it doesn't work the next one is used. Note that it is very unusual to mix using CVS and ftp, this is just for the example.
The first two URLs are ftp sites. These are standard URLs. For CVS the part after "cvs://" uses the standard CVSROOT syntax. This is the same as what the cvs command uses after the "-d" option. A-A-P also needs to know from which directory of the cvs repository the recipe has to be obtained. This is specified with the "path" attribute. Note that this is a nested attribute: it applies to an item inside the "origin" attribute. For CVS the file name is not specified. That is because CVS requires that the local name is the same as the name in the repository.

line 6: A line starting with "@" indicates this is a Python script line. All Python commands can be used this way. Here an "if" statement checks if the recipe is being executed on a Unix system.
Note that in Python lines the same variables are available as in non-Python lines, but without the "$" sign. A convention is that variables that are used in non-Python lines are uppercase. This makes the recipe easier to understand.

The ":child" command tells A-A-P to use another recipe. In this case an "origin" attribute is specified, which means the recipe can be refreshed, the latest version can be obtained.
Notice that the recipe name after $FTPBASE is different from the local name and ends in ".aap.gz". A-A-P knows this means the recipe was compressed with gzip and will automatically uncompress it before using it. For CVS the name in the repository must be equal to the local name, therefore the recipe name is not specified.

line 8: An else-if check for a MS-Windows system.

line 9: The ":child" command that specifies the recipe to use for MS-Windows. The name after $FTPBASE actually specifies a zip archive in which the recipe is contained. A-A-P knows how to extract the file from the archive.

line 10-11: There is no recipe for other systems. The ":error" command is used to produce an error message and abort. Note that "$OSTYPE" is the same variable as "OSTYPE" used in the Python lines.


A ported application

When an application already exists but for your system it requires a few tweaks, a port recipe can do the work. This can also be used for applications that work fine but you want to apply a number of patches on to add a feature.

Since A-A-P is prepared for doing all the work, usually you only need to specify in the recipe where to find the files involved. Here is an example:
1    PORT_TYPE = autoconf
3    MASTER_SITE = ftp://ftp.download.net/pub/mirror/  http://sources.org/files/
4    MASTER_SUBDIR = foo/myprog
5    MASTER_FILE = myprog_src-1.3.tar.gz {md5 = 12341234}
6 		   docs/myprog_doc.tar.gz {md5 = 56785678}
8    PATCH_SITE = ftp://ftp.ucs.edu/~mike/myprog/
9    PATCH_SUBDIR = files/patches/
10   PATCH_FILE = myprog.001.gz myprog.002.add
12   DEPEND_BUILD = cproto {version = "1.[3-9]|[2-9].[0-9]"}
13   DEPEND_RUN = ghostscript

This indicates this is a port and not everything is done with A-A-P recipes. The type indicates what to do to build the port. "autoconf" means the "configure" script will be run.
List of sites where the original files can be downloaded from. This goes down to the common directory for all downloadable files.
Path at $MASTER_SITE below which all required files can be found. This is optional, the path can also be included in the files themselves. It's also possible to use a full URL here, in case on some site a different path is required.
The list of files to download and unpack. Note that all the files are used, while for $MASTER_SITE only the first one that works is used.
Like the MASTER_ variables, but specifies the location where to get patch files.
A list of applications that are required for building the target. In the example the "cproto" program is required. The attribute specifies a pattern to check for the right version.
A list of applications that are required for running the target. It's like DEPEND_BUILD, but getting these applications can be postponed until after building the target.
Note that all names are singular. This avoids having to think about which is plural and which is not.

Building multiple variants

Quite often you want to compile an application for release with maximal optimizing. But the optimizer confuses the debugger, thus when stepping through the program to locate a problem, you want to recompile without optimizing.

A-A-P provides a way to build several variants of the same application. You just need to specify the variants and what is different about them. A-A-P will then take care of putting the resulting files in a different directory, so that you don't have to recompile everything when you select another variant. Here is an example:
1    SOURCE = main.c version.c gui.c
3    :variant BUILD
4       release
5          CFLAGS = -O4
6          TARGET = myprog
7       debug
8          CFLAGS = -g
9          TARGET = myprogd
11   OFILES = $BDIR/main.o $BDIR/version.o
13   GUI ?= console
14   :variant GUI
15      console
16      motif
17          SOURCE += motif.c
18      gtk
19          SOURCE += gtk.c
This example contains quite a few interesting items. The essential is the use of the ":variant" statement (line 3 and 14). It specifies a variable name and a list of alternatives. The first one has "release" and "debug" variants, the second "console", "motif" and "gtk". These can be selected independently, thus there are six combinations: release-console, debug-console, release-motif, debug-motif, release-gtk and debug-gtk.

If the "BUILD" variable was not assigned a value, the first alternative "release" is selected. The user can select the debug variant by giving aap the command line argument "BUILD=debug". In line 13 a similar thing is done for the "GUI" variable. Here the default is to build a console version, unless "GUI" has been set to "gtk" or "motif". An configure check for the required GUI libraries might be used to set the "GUI" variable.

The assignment to "OFILES" in line 11 uses $BDIR in between the two variants. $BDIR is the directory used to store intermediate results for a specific variant. When switching between "debug" and "release" the ":variant" command will change the value of $BDIR. The "main.c" and "version.c" files don't depend on the value of GUI. Therefore their object files can be shared between the GUI variants. This is done here by using the value of $BDIR before the GUI variant changes it. (Later this will be done in a nicer way.)


Using Python in recipes

In various places in the recipe Python commands and expressions can be used for complicated things. You have already seen lines that start with "@". When a few Python lines are used, it's best to line up the "@" characters:
      @if OSTYPE == "unix"
      @   for i in [ "solaris", "hpux" ]
      @       if OSNAME == i
      @           COMMERCIAL = "yes"
When the number of lines gets longer, it's easier to put them in a block:
         if OSTYPE == "unix"
            for i in [ "solaris", "hpux" ]
               if OSNAME == i
                  COMMERCIAL = "yes"
The ":end" must appear on a line by itself, only a comment may follow. An alternate terminator string may be specified, although it's very unlikely that ":end" appears in the Python block.

In assignments and rules a Python expression can be used. Example:
   @from glob import glob
   SOURCE = `glob("*.c")`
Note that this uses a Python expression, not a Python command. The resulting string or list of strings is inserted in-place before the assignment is evaluated. In this example calling glob() results in a list of all files matching "*.c", which is turned into a space-separated string.
This is actually not a very good example, because a "test.c" file that you temporarily used will accidentally be included. It's better to list the source files explicitly.

A common use of a Python expression is to change a list of source file names to a list of target file names. Example:
   OBJECT = `aap_sufreplace(".c", OBJSUF, SOURCE)`
The "aap_sufreplace()" function takes three arguments. The last argument is the name of a variable that is a list of names (with attributes). The first argument is the suffix which is to be replaced. The middle argument is the replacement suffix. In this example each name in $SOURCE ending in ".c" will be changed to end in $OBJSUF, which is ".o" or ".obj".

A variable is often used for a list of items. Parsing this string and turning it into a list can be done with the aap_var2list() function. Example:
    for s in aap_var2list(SOURCE):
	print "source file " + s
If you also want to access the item attributes, you can turn the variable into a list of dictonaries. Each dictionary contains a "name" entry that is the name of the item and other entries are the attribute values. Entries starting with an underscore are for internal use, the others are attributes. This is how to list them all:
    for d in aap_var2dictlist(SOURCE):
	print "source item " + d["name"]
	for a in d.keys():
	    if a != "name" and a[0] != '_':
	    	print "     attribute " + a + ": " + d[a]
There are several other functions supplied by A-A-P, so that you hardly ever have to use the low level access in a recipe.


More about attributes

Most A-A-P variables are a list of items, and each item can have attributes. These attributes are used for various purposes. One of them is to decide when a source file has actually changed, such that it has to be compiled again. A-A-P avoids using timestamps, because these are often unreliable (e.g., when putting back an older version of a file, make doesn't notice the change).

The default method for source files is to use a signature. For C files, for example, comments and changes in white space are ignored. This reduces the number of compilations when the actual code didn't change. Especially useful when adding a comment for a global variable. However, when using a tool to extract documentation from the comments, another signature must be used. Example:
   myprog.hlp : $SOURCE {check = md5}
       :sys getfunclist $SOURCE > myprog.hlp
To set default attributes these two commands can be used:
   :depend_attribute "*.c" {check = c_signature}
   :target_attribute 'all|clean' {type = notafile}
The first command sets attributes for the "depend" side of a rule, the files right of the ":". The argument is a pattern that is matched against the file names. In this case "*.c" matches with all C files. The check for a changed file is then done with the "c_signature" method.
The second command sets attributes for the target side of a rule, the files on the left of the ":". The pattern "all|clean" matches the targets "all" and "clean" and gives them the attribute "notafile", which means they are not file targets.
These examples are actually the defaults, thus these commands doesn't change anything useful.


More about commands

The commands starting with a colon are special A-A-P commands. Here are a few useful ones.
:rule target-pattern : source-pattern
Define a build rule for building "target" from "source". This can overrule or add to the default build rules. For example, to filter php files to obtain plain html, using the imaginary "unphp" command:
   :rule %.html : %.php
       :sys unphp <$source >$target
Attributes can be added to the source pattern to specify the default check to be used. When nothing is given, the default is used (md5).
:refresh filename
Get a file. Download it when needed.
:unpack archive [file]
Unpack an archive. When no specific file is mentioned, all files are extracted. This is a portable way to unpack an archive.
:update filename
An explicit invocation for the rule to make sure filename is up-to-date. This can be used when the normal dependencies are not sufficient.
:require item
Require the precense of a tool. When the tool is not available this will fail. A try block can then be used to fall back to another tool. Example:
    myprog: $OBJECT
           :require xyx-linker
           LINKER = xyz-linker
     	      :require abc-linker
     	      LINKER = abc-linker
              :error "can't find a useful linker"
        $LINKER $OBJECT -o myprog

Pipe commands

A selection of commands can be connected together with a pipe. This means the output of one command is the input for the next command. It is useful for filtering text from a variable or file and writing the result in a variable or file.
:cat file1 file2 | :filter re.sub('this', 'that', %s) > file3
This example concatenates the files "file1" and "file2", replaces all occurrences of "this" with "that" and writes the result in "file3".

The following commands can be used in a pipe:

:print [redir] text
Evaluate $VAR things in the argument and print the result. When redirection is specified the result is written to a file. When a pipe follows, the result is passed on to the next command.
   var = something
   :print var is now $var | :assign var_text
This example assigns "var is now something" to the variable "var_text".
:print [redir]
Reads the input from a pipe and prints it. When redirection is specified the result is written to a file. When a pipe follows, the result is passed on to the next command (not very useful!).
:cat [redir] filename ...
Concatenate all the files mentioned in the argument. Where an argument is "-" the output of the previous command in the pipe is used. When redirection is specified the result is written to a file. When a pipe follows, the result is passed on to the next command.
   :print $var | :cat file1 - file2 >! file3
This example concatenates "file1", the value of variable "var" and "file2" and writes the result in "file3".
:assign varname
Assign the output of the previous command in the pipe to variable "varname".
:filter python-expression
Filter the output of the previous command in the pipe through a Python expression. The expression must include "%s" somewhere, it is replaced by the name of the variable that holds the text resulting from the previous command. When redirection is specified the result is written to a file. When a pipe follows, the result is passed on to the next command.
:tee [redir] filename ...
Write the output of the previous command in the pipe to each file mentioned as an argument. When redirection is specified the same text is written to a file. When a pipe follows, the same text is passed on to the next command.
   :cat file1 file2 | :tee file3 | :assign var
This example concatenates files "file1" and "file2", writes the result in "file3" and assigns the same text to variable "var"

The output of these commands can be redirected to a file instead of written through a pipe to the next command. There are three variants:

> filename
Write a new file. If the file exists this is an error.
>! filename
Write a new file or overwrite an existing file.
>> filename
Append to a file. If it doesn't exist yet it is created.

Automatic dependency checking

When a source file includes other files, the targets that depend on it need to be rebuild. Thus when "foo.c" includes "foo.h" and "foo.h" is changed, the build commands to produce "foo.o" from "foo.c" must be executed, even though "foo.c" itself didn't change.

A-A-P detects these implied dependencies automatically for the types it knows about. Currently that is C and C++. For other types of files you can add your own dependency checker. For example, this is how to define a checker for the "tt" file type:
   foo : foo.tt
	:sys tt_compiler $source > $target
   :autodepend tt
   	:sys tt_checker $source > $target
The "tt_checker" command reads the file "$source" and writes a dependency line in the file "$target". This is a dependency like it is used in a recipe. In a makefile this has the same syntax, thus tools that produce dependencies for "make" will work. Here is an example:
   foo.o : foo.tt  foo.hh  include/common.hh
This is interpreted as a dependency on "foo.hh" and "include/common.hh". Note that "foo.o" and "foo.tt" are ignored. Tools designed for "make" produce these but they are irrelevant for A-A-P.

Since the build commands for ":autodepend" are ordinary build commands, you can use Python commands, system commands or a mix of both to do the dependency checking.


Multiple targets

When a dependency has more than one target, this means that the build commands will produce all these targets. This makes it possible to specify build commands that produce several files at the same time. Example that compiles a file and at the same time produces a documentation file:
   foo.o foo.html : foo.src
	:sys srcit $source -o $(target[0]) --html $(target[1])
People used to "make" must be careful, they might expect the build commands to be executed once for each target. A-A-P doesn't work that way, because the above example would be impossible. To run commands on each target this must be explicitly specified. Example:
   dir1 dir2 dir3 :
   	@for item in target_list:
	     :mkdir $item
The variable "target_list" is a Python list of the target items. "source_list" is the list of source items. An extreme example of executing build commands for each combination of sources and targets:
   	@for trg in target_list:
	    :print start of file  >! $trg
	    @for src in source_list:
		:sys foofilter -D$trg $src >> $trg


Publishing a web site

If you are maintaining a web site it is often a good idea to edit the files on your local system. After trying out the changes you then need to upload the changed files to the web server. A-A-P can be used to keep track of which files changed and upload these files. This is called publishing.

Here is an example of a recipe:
   FILES = `glob("*.html")` `glob("images/*.jpg")`
   :attr {publish = scp://user@ftp.foo.org/public_html/%file%} $FILES
That's all. You just need to specify the files you want to be published and the URL that says how and where to upload them to. Now "aap publish" will find out which files have changed and upload them. The first time all files will be uploaded (use "aap publish --touch" to avoid that).

The ":attr" command adds attributes to its arguments. In this case the "publish" attribute is added, which specifies the URL where to upload a file to. In the example the "scp" protocol is used, which is a good method for uploading files to a public server.

If the same recipe produces some of the html files, you need to specify a dependency for them. The ":publishall" command can then take care of the publishing:
   FILES = index.html images/logo.gif
   :attr {publish = scp://user@ftp.foo.org/public_html/%file%} $FILES

   all: $FILES

   publish: $FILES

   index.html: header.part body.part footer.part
      :cat $source >! $target
With this recipe, executing "aap" without arguments will produce the "index.html" file when needed. "aap publish" will do the same and publish updated files.


Version control with CVS

CVS is often used for development of Open Source Software. A-A-P provides facilities to obtain the latest version of an application, and for checking in changes you made. For downloading you only need to specify the location of the CVS server and the name of the module. Here is an example that obtains the Recipe Executive:
   CVSROOT = :pserver:anonymous@cvs.a-a-p.sf.net:/cvsroot/a-a-p
	:refresh {refresh = cvs://$CVSROOT} Exec
Now do "aap refresh" and the directory "Exec" will be created and all files obtained from the CVS server.

When you are working on a project and want to checkin your changes to the CVS server, you need to specify which files you want to checkin. This is done by supplying a "commit" attribute. Only these files will appear in the CVS repository, other files will be removed. Example:
   CVSROOT = :ext:$USERNAME@cvs.myproject.sf.net:/cvsroot/myproject
   FILES = main.c common.h version.c
   :attr {commit = cvs://$CVSROOT} $FILES
Doing "aap revise" will commit all your changes and remove files that appear in CVS but don't have a "commit" attribute.

All this can also be done by using cvs commands. But it's easy to make a mistake in typing the server name or forgetting one of the actions required. After putting the magic words in a recipe once you know it will keep working. And a user doesn't even have to know that CVS is being used. For another version control system the commands he uses will be the same.


Recipe syntax definition

This defines the recipe syntax (more or less). NOTE: this may still change quite a bit.

|separates alternatives
()grouping a sequence of items or alternatives
[]optional items (also does grouping)
""contains literal text; """ is one double quote
...indicates the preceding item or group can be repeated
EOLan end-of-line, optionally preceded by a comment
INDENT  an amount of white space
A comment starts with "#" and continues until the end-of-line. It may appear in many places.

Line continuation is done with a backslash immediately before an end-of-line. It's not needed for an assignment and other commands that don't have a build_block: the item continues if the following line has more indent than the first line.
aapfile ::= ( [ aap_item ] EOL ) ...
aap_item ::= dependency | rule | variant | otherfile | build_command
dependency ::= targets ":" [ dependencies ] [ EOL build_block ]
targets ::= item_list
dependencies ::= item_list
build_block ::= INDENT build_command [ EOL [ INDENT build_command ] ] ...
rule ::= ":rule" pattern ... ":" pattern ... [ EOL build_block ]
variant ::= ":variant" name ( EOL variant_item ) ...
variant_item ::= INDENT1 varvalue EOL INDENT2 build_block
otherfile ::= includefile | childfile
includefile ::= ":include" item_list
childfile ::= ":child" item_list
build_command ::= assignment | script_item | generic_command | message
assignment ::= assign_set | assign_append | assign_cond
assign_set ::= name "=" item_list
assign_append ::= name "+=" item_list
assign_cond ::= name "?=" item_list
script_item ::= script_line | script_block
script_line ::= "@" script_command
script_block ::= ":python" EOL ( script_command EOL ) ... ":end"
generic_command ::= ":" command_name [ command_argument ]
message ::= info_msg | error_msg
info_msg ::= ":print" expression ...
error_msg ::= ":error" expression ...
# A list can contain white-separated items and Python style expressions.
item_list ::= item [ white_space item ] ...
item ::= ( simple_item | "(" item_list ")" ) [ attribute ... ]
simple_item ::= ( expression | non_white_item ) ...
attribute ::= "{" name [ "=" expression ] "}"
expression ::= string_expr | python_expr
string_expr ::= """ text """ | "'" text "'"
python_expr ::= "`" python-expression "`"
non_white_item ::= non-white-text | "$" [ "(" ] name [ ")" ]



The A-A-P pages:

     recipe file
     Aap manual
     use cases
     design decisions
tools overview
     script languages
     build tools
     Install tools
     issue tracking
     version control
     browse tools
     various tools
SourceForge project page

Send comments on this page to Webmaster AT a-a-p.org.            Hosted by SourceForge Logo