Documentation - Recipe file

A-A-P is still in an early phase. Therefore this document can only be used to get an idea of the current ideas of the recipe file format. Everything may still change!

The "hello world" example
The basics
A top level recipe
A ported application
Building multiple variants
Using Python in recipes
More about attributes
More about commands
Pipe commands
Automatic dependency checking
Multiple targets
Publishing a web site
Version control with CVS
Recipe syntax definition

The "hello world" example

    SOURCE = hello.c
    TARGET = hello

The basics

    SOURCE =
           main.c
           version.c
           help.c
    TARGET = myprogram

Indents are very important, just like in a Python script. Make sure your tabstop is always set to the standard value of eight, otherwise you might run into trouble when mixing tabs and spaces.

Someone who sees this recipe would like to know what it's for. This requires adding comments. These start with a "#" character and extend until the end of the line (like in Makefiles and Python scripts).
It is also possible to associate a comment with a specific item:

# A-A-P file to compile myprogram SOURCE = main.c # startup stuff version.c # just the date stamp help.c # display help stuff TARGET = myprogram { comment = "build my program" }

Now run A-A-P with:

   aap comment

   comments for main.aap:
	target myprogram: build my program

    SOURCE =
   	    main.c
	    version.c {force}
	    help.c
    TARGET = myprogram { comment = "build my program" }

On MS-Windows an executable ends in ".exe". Adding this to "myprogram" would result in a recipe that only works on MS-Windows. Better is to use the EXESUF variable, which contains ".exe" only when appropriate.

SOURCE = main.c TARGET = myprogram$EXESUF

You can see that it's very simple to use a variable in an assignment. The dollar sign indicates a variable name is following.
If you might want to use a dollar in a file name, double the dollar sign. For example, to use the "$temp.c" file as a source file:

SOURCE = $$temp.c

A variable name consists of ASCII letters, numbers and the underscore. If you want to append one of these after a variable name you must put parenthesis around the variable name, so that A-A-P knows where it ends:

MAKENAME = Make BUILDFILE = $(MAKENAME)file

A top level recipe

1	FTPBASE = ftp://ftp.vim.org/pub/vim  ftp://ftp.us.vim.org/pub/vim
2	CVSROOT = cvs://:pserver:anonymous@cvs.vim.org/vim {path = aap}
3
4	:recipe {origin = $FTPBASE/aap/main.aap $CVSROOT}
5	
6	@if OSTYPE == "posix":
7	    :child unix.aap {origin = $FTPBASE/unix/vim.aap.gz $CVSROOT}
8	@elif OSTYPE == "mswin":
9	    :child pc.aap {origin = $FTPBASE/pc/vim.zip//vim.aap $CVSROOT}
10	@else:
11	    :error OS not supported: $OSTYPE

line 4: A line starting with ":" indicates an A-A-P command. This clearly separates them from Python lines, assignments, etc.
The ":recipe" command specifies the origin of the recipe itself. This is used to check if an updated version is available. A new version is only obtained when specifically asked for, not each time the recipe is executed.
The origin attribute specifies three locations. That is because the variable $FTPBASE contains two items. Using this variable with text appended results in two items, each with the text appended. After the expansion the command looks like this:

:recipe {origin = ftp://ftp.vim.org/pub/vim/aap/main.aap ftp://ftp.us.vim.org/pub/vim/aap/main.aap cvs://:pserver:anonymous@cvs.vim.org/vim {path = aap}}

Only one of the three URLs will be used. Normally it's the first one, but when it doesn't work the next one is used. Note that it is very unusual to mix using CVS and ftp, this is just for the example.
The first two URLs are ftp sites. These are standard URLs. For CVS the part after "cvs://" uses the standard CVSROOT syntax. This is the same as what the cvs command uses after the "-d" option. A-A-P also needs to know from which directory of the cvs repository the recipe has to be obtained. This is specified with the "path" attribute. Note that this is a nested attribute: it applies to an item inside the "origin" attribute. For CVS the file name is not specified. That is because CVS requires that the local name is the same as the name in the repository.

line 6: A line starting with "@" indicates this is a Python script line. All Python commands can be used this way. Here an "if" statement checks if the recipe is being executed on a Unix system.
Note that in Python lines the same variables are available as in non-Python lines, but without the "$" sign. A convention is that variables that are used in non-Python lines are uppercase. This makes the recipe easier to understand.

The ":child" command tells A-A-P to use another recipe. In this case an "origin" attribute is specified, which means the recipe can be refreshed, the latest version can be obtained.
Notice that the recipe name after $FTPBASE is different from the local name and ends in ".aap.gz". A-A-P knows this means the recipe was compressed with gzip and will automatically uncompress it before using it. For CVS the name in the repository must be equal to the local name, therefore the recipe name is not specified.

line 8: An else-if check for a MS-Windows system.

line 9: The ":child" command that specifies the recipe to use for MS-Windows. The name after $FTPBASE actually specifies a zip archive in which the recipe is contained. A-A-P knows how to extract the file from the archive.

line 10-11: There is no recipe for other systems. The ":error" command is used to produce an error message and abort. Note that "$OSTYPE" is the same variable as "OSTYPE" used in the Python lines.

A ported application

Since A-A-P is prepared for doing all the work, usually you only need to specify in the recipe where to find the files involved. Here is an example:

1 PORT_TYPE = autoconf 2 3 MASTER_SITE = ftp://ftp.download.net/pub/mirror/ http://sources.org/files/ 4 MASTER_SUBDIR = foo/myprog 5 MASTER_FILE = myprog_src-1.3.tar.gz {md5 = 12341234} 6 docs/myprog_doc.tar.gz {md5 = 56785678} 7 8 PATCH_SITE = ftp://ftp.ucs.edu/~mike/myprog/ 9 PATCH_SUBDIR = files/patches/ 10 PATCH_FILE = myprog.001.gz myprog.002.add 11 12 DEPEND_BUILD = cproto {version = "1.[3-9]|[2-9].[0-9]"} 13 DEPEND_RUN = ghostscript

PORT_TYPE

This indicates this is a port and not everything is done with A-A-P recipes. The type indicates what to do to build the port. "autoconf" means the "configure" script will be run.

MASTER_SITE

List of sites where the original files can be downloaded from. This goes down to the common directory for all downloadable files.

MASTER_SUBDIR

Path at $MASTER_SITE below which all required files can be found. This is optional, the path can also be included in the files themselves. It's also possible to use a full URL here, in case on some site a different path is required.

MASTER_FILE

The list of files to download and unpack. Note that all the files are used, while for $MASTER_SITE only the first one that works is used.

PATCH_SITE, PATCH_SUBDIR, PATCH_FILE

Like the MASTER_ variables, but specifies the location where to get patch files.

DEPEND_BUILD

A list of applications that are required for building the target. In the example the "cproto" program is required. The attribute specifies a pattern to check for the right version.

DEPEND_RUN

A list of applications that are required for running the target. It's like DEPEND_BUILD, but getting these applications can be postponed until after building the target.

Building multiple variants

A-A-P provides a way to build several variants of the same application. You just need to specify the variants and what is different about them. A-A-P will then take care of putting the resulting files in a different directory, so that you don't have to recompile everything when you select another variant. Here is an example:

1 SOURCE = main.c version.c gui.c 2 3 :variant BUILD 4 release 5 CFLAGS = -O4 6 TARGET = myprog 7 debug 8 CFLAGS = -g 9 TARGET = myprogd 10 11 OFILES = $BDIR/main.o $BDIR/version.o 12 13 GUI ?= console 14 :variant GUI 15 console 16 motif 17 SOURCE += motif.c 18 gtk 19 SOURCE += gtk.c

This example contains quite a few interesting items. The essential is the use of the ":variant" statement (line 3 and 14). It specifies a variable name and a list of alternatives. The first one has "release" and "debug" variants, the second "console", "motif" and "gtk". These can be selected independently, thus there are six combinations: release-console, debug-console, release-motif, debug-motif, release-gtk and debug-gtk.

If the "BUILD" variable was not assigned a value, the first alternative "release" is selected. The user can select the debug variant by giving aap the command line argument "BUILD=debug". In line 13 a similar thing is done for the "GUI" variable. Here the default is to build a console version, unless "GUI" has been set to "gtk" or "motif". An configure check for the required GUI libraries might be used to set the "GUI" variable.

The assignment to "OFILES" in line 11 uses $BDIR in between the two variants. $BDIR is the directory used to store intermediate results for a specific variant. When switching between "debug" and "release" the ":variant" command will change the value of $BDIR. The "main.c" and "version.c" files don't depend on the value of GUI. Therefore their object files can be shared between the GUI variants. This is done here by using the value of $BDIR before the GUI variant changes it. (Later this will be done in a nicer way.)

Using Python in recipes

      @if OSTYPE == "unix"
      @   for i in [ "solaris", "hpux" ]
      @       if OSNAME == i
      @           COMMERCIAL = "yes"

      :python
         if OSTYPE == "unix"
            for i in [ "solaris", "hpux" ]
               if OSNAME == i
                  COMMERCIAL = "yes"
      :end

In assignments and rules a Python expression can be used. Example:

@from glob import glob SOURCE = `glob("*.c")`

Note that this uses a Python expression, not a Python command. The resulting string or list of strings is inserted in-place before the assignment is evaluated. In this example calling glob() results in a list of all files matching "*.c", which is turned into a space-separated string.
This is actually not a very good example, because a "test.c" file that you temporarily used will accidentally be included. It's better to list the source files explicitly.

A common use of a Python expression is to change a list of source file names to a list of target file names. Example:

OBJECT = `aap_sufreplace(".c", OBJSUF, SOURCE)`

The "aap_sufreplace()" function takes three arguments. The last argument is the name of a variable that is a list of names (with attributes). The first argument is the suffix which is to be replaced. The middle argument is the replacement suffix. In this example each name in $SOURCE ending in ".c" will be changed to end in $OBJSUF, which is ".o" or ".obj".

A variable is often used for a list of items. Parsing this string and turning it into a list can be done with the aap_var2list() function. Example:

for s in aap_var2list(SOURCE): print "source file " + s

If you also want to access the item attributes, you can turn the variable into a list of dictonaries. Each dictionary contains a "name" entry that is the name of the item and other entries are the attribute values. Entries starting with an underscore are for internal use, the others are attributes. This is how to list them all:

for d in aap_var2dictlist(SOURCE): print "source item " + d["name"] for a in d.keys(): if a != "name" and a[0] != '_': print " attribute " + a + ": " + d[a]

There are several other functions supplied by A-A-P, so that you hardly ever have to use the low level access in a recipe.

More about attributes

The default method for source files is to use a signature. For C files, for example, comments and changes in white space are ignored. This reduces the number of compilations when the actual code didn't change. Especially useful when adding a comment for a global variable. However, when using a tool to extract documentation from the comments, another signature must be used. Example:

myprog.hlp : $SOURCE {check = md5} :sys getfunclist $SOURCE > myprog.hlp

To set default attributes these two commands can be used:

:depend_attribute "*.c" {check = c_signature} :target_attribute 'all|clean' {type = notafile}

The first command sets attributes for the "depend" side of a rule, the files right of the ":". The argument is a pattern that is matched against the file names. In this case "*.c" matches with all C files. The check for a changed file is then done with the "c_signature" method.
The second command sets attributes for the target side of a rule, the files on the left of the ":". The pattern "all|clean" matches the targets "all" and "clean" and gives them the attribute "notafile", which means they are not file targets.
These examples are actually the defaults, thus these commands doesn't change anything useful.

More about commands

:rule target-pattern : source-pattern

Define a build rule for building "target" from "source". This can overrule or add to the default build rules. For example, to filter php files to obtain plain html, using the imaginary "unphp" command:

:rule %.html : %.php :sys unphp <$source >$target

Attributes can be added to the source pattern to specify the default check to be used. When nothing is given, the default is used (md5).

:refresh filename

Get a file. Download it when needed.

:unpack archive [file]

Unpack an archive. When no specific file is mentioned, all files are extracted. This is a portable way to unpack an archive.

:update filename

An explicit invocation for the rule to make sure filename is up-to-date. This can be used when the normal dependencies are not sufficient.

:require item

Require the precense of a tool. When the tool is not available this will fail. A try block can then be used to fall back to another tool. Example:

myprog: $OBJECT @try: :require xyx-linker LINKER = xyz-linker @except: @try: :require abc-linker LINKER = abc-linker @except: :error "can't find a useful linker" $LINKER $OBJECT -o myprog

Pipe commands

:cat file1 file2 | :filter re.sub('this', 'that', %s) > file3

The following commands can be used in a pipe:

:print [redir] text

Evaluate $VAR things in the argument and print the result. When redirection is specified the result is written to a file. When a pipe follows, the result is passed on to the next command.

var = something :print var is now $var | :assign var_text

This example assigns "var is now something" to the variable "var_text".

:print [redir]

Reads the input from a pipe and prints it. When redirection is specified the result is written to a file. When a pipe follows, the result is passed on to the next command (not very useful!).

:cat [redir] filename ...

Concatenate all the files mentioned in the argument. Where an argument is "-" the output of the previous command in the pipe is used. When redirection is specified the result is written to a file. When a pipe follows, the result is passed on to the next command.

:print $var | :cat file1 - file2 >! file3

This example concatenates "file1", the value of variable "var" and "file2" and writes the result in "file3".

:assign varname

Assign the output of the previous command in the pipe to variable "varname".

:filter python-expression

Filter the output of the previous command in the pipe through a Python expression. The expression must include "%s" somewhere, it is replaced by the name of the variable that holds the text resulting from the previous command. When redirection is specified the result is written to a file. When a pipe follows, the result is passed on to the next command.

:tee [redir] filename ...

Write the output of the previous command in the pipe to each file mentioned as an argument. When redirection is specified the same text is written to a file. When a pipe follows, the same text is passed on to the next command.

:cat file1 file2 | :tee file3 | :assign var

This example concatenates files "file1" and "file2", writes the result in "file3" and assigns the same text to variable "var"

The output of these commands can be redirected to a file instead of written through a pipe to the next command. There are three variants:

> filename

Write a new file. If the file exists this is an error.

>! filename

Write a new file or overwrite an existing file.

>> filename

Append to a file. If it doesn't exist yet it is created.

Automatic dependency checking

A-A-P detects these implied dependencies automatically for the types it knows about. Currently that is C and C++. For other types of files you can add your own dependency checker. For example, this is how to define a checker for the "tt" file type:

foo : foo.tt :sys tt_compiler $source > $target :autodepend tt :sys tt_checker $source > $target

The "tt_checker" command reads the file "$source" and writes a dependency line in the file "$target". This is a dependency like it is used in a recipe. In a makefile this has the same syntax, thus tools that produce dependencies for "make" will work. Here is an example:

foo.o : foo.tt foo.hh include/common.hh

This is interpreted as a dependency on "foo.hh" and "include/common.hh". Note that "foo.o" and "foo.tt" are ignored. Tools designed for "make" produce these but they are irrelevant for A-A-P.

Since the build commands for ":autodepend" are ordinary build commands, you can use Python commands, system commands or a mix of both to do the dependency checking.

Multiple targets

   foo.o foo.html : foo.src
	:sys srcit $source -o $(target[0]) --html $(target[1])

   dir1 dir2 dir3 :
   	@for item in target_list:
	     :mkdir $item

   $OUTPUTFILES : $INPUTFILES
   	@for trg in target_list:
	    :print start of file  >! $trg
	    @for src in source_list:
		:sys foofilter -D$trg $src >> $trg

Publishing a web site

Here is an example of a recipe:

FILES = `glob("*.html")` `glob("images/*.jpg")` :attr {publish = scp://user@ftp.foo.org/public_html/%file%} $FILES

That's all. You just need to specify the files you want to be published and the URL that says how and where to upload them to. Now "aap publish" will find out which files have changed and upload them. The first time all files will be uploaded (use "aap publish --touch" to avoid that).

The ":attr" command adds attributes to its arguments. In this case the "publish" attribute is added, which specifies the URL where to upload a file to. In the example the "scp" protocol is used, which is a good method for uploading files to a public server.

If the same recipe produces some of the html files, you need to specify a dependency for them. The ":publishall" command can then take care of the publishing:

FILES = index.html images/logo.gif :attr {publish = scp://user@ftp.foo.org/public_html/%file%} $FILES all: $FILES publish: $FILES :publishall index.html: header.part body.part footer.part :cat $source >! $target

With this recipe, executing "aap" without arguments will produce the "index.html" file when needed. "aap publish" will do the same and publish updated files.

Version control with CVS

   CVSROOT = :pserver:anonymous@cvs.a-a-p.sf.net:/cvsroot/a-a-p
   refresh:
	:refresh {refresh = cvs://$CVSROOT} Exec

When you are working on a project and want to checkin your changes to the CVS server, you need to specify which files you want to checkin. This is done by supplying a "commit" attribute. Only these files will appear in the CVS repository, other files will be removed. Example:

CVSROOT = :ext:$USERNAME@cvs.myproject.sf.net:/cvsroot/myproject FILES = main.c common.h version.c :attr {commit = cvs://$CVSROOT} $FILES

Doing "aap revise" will commit all your changes and remove files that appear in CVS but don't have a "commit" attribute.

All this can also be done by using cvs commands. But it's easy to make a mistake in typing the server name or forgetting one of the actions required. After putting the magic words in a recipe once you know it will keep working. And a user doesn't even have to know that CVS is being used. For another version control system the commands he uses will be the same.

Recipe syntax definition

Notation:

|	separates alternatives
()	grouping a sequence of items or alternatives
[]	optional items (also does grouping)
""	contains literal text; """ is one double quote
...	indicates the preceding item or group can be repeated
EOL	an end-of-line, optionally preceded by a comment
INDENT	an amount of white space

A comment starts with "#" and continues until the end-of-line. It may appear in many places.

Line continuation is done with a backslash immediately before an end-of-line. It's not needed for an assignment and other commands that don't have a build_block: the item continues if the following line has more indent than the first line.

aapfile	::=	( [ aap_item ] EOL ) ...
aap_item	::=	dependency | rule | variant | otherfile | build_command
dependency	::=	targets ":" [ dependencies ] [ EOL build_block ]
targets	::=	item_list
dependencies	::=	item_list
build_block	::=	INDENT build_command [ EOL [ INDENT build_command ] ] ...
rule	::=	":rule" pattern ... ":" pattern ... [ EOL build_block ]
variant	::=	":variant" name ( EOL variant_item ) ...
variant_item	::=	INDENT1 varvalue EOL INDENT2 build_block
otherfile	::=	includefile | childfile
includefile	::=	":include" item_list
childfile	::=	":child" item_list
build_command	::=	assignment | script_item | generic_command | message
assignment	::=	assign_set | assign_append | assign_cond
assign_set	::=	name "=" item_list
assign_append	::=	name "+=" item_list
assign_cond	::=	name "?=" item_list
script_item	::=	script_line | script_block
script_line	::=	"@" script_command
script_block	::=	":python" EOL ( script_command EOL ) ... ":end"
generic_command	::=	":" command_name [ command_argument ]
message	::=	info_msg | error_msg
info_msg	::=	":print" expression ...
error_msg	::=	":error" expression ...
# A list can contain white-separated items and Python style expressions.
item_list	::=	item [ white_space item ] ...
item	::=	( simple_item | "(" item_list ")" ) [ attribute ... ]
simple_item	::=	( expression | non_white_item ) ...
attribute	::=	"{" name [ "=" expression ] "}"
expression	::=	string_expr | python_expr
string_expr	::=	""" text """ | "'" text "'"
python_expr	::=	"`" python-expression "`"
non_white_item	::=	non-white-text | "$" [ "(" ] name [ ")" ]