|
Documentation - Recipe file
A-A-P is still in an early phase.
Therefore this document can only be used to get an idea of the current ideas
of the recipe file format. Everything may still change!
The "hello world" example
The basics
A top level recipe
A ported application
Building multiple variants
Using Python in recipes
More about attributes
More about commands
Pipe commands
Automatic dependency checking
Multiple targets
Publishing a web site
Version control with CVS
Recipe syntax definition
The "hello world" example
Most programming languages start with a short example that prints a "hello
world" message. This is how to compile such a program with A-A-P:
SOURCE = hello.c
TARGET = hello
|
This is all that A-A-P needs to know to compile "hello.c" into the program
"hello". A-A-P will figure out what compiler is to be used for a ".c" file
and run it with the default options.
index
The basics
When you have several files with source code you can specify them as a list:
SOURCE =
main.c
version.c
help.c
TARGET = myprogram
|
Notice that it is not necessary to use a line continuation character, as you
would have to do in a Makefile. A-A-P knows that when a line is encountered
with the same indent as the line the assignment started with, that must be the
end of the list. The amount of indent for the continuation lines is
irrelevant, so long as it's more than the indent of the first line.
Indents are very important, just like in a Python script. Make sure your
tabstop is always set to the standard value of eight, otherwise you might run
into trouble when mixing tabs and spaces.
Someone who sees this recipe would like to know what it's for. This requires
adding comments. These start with a "#" character and extend until the end of
the line (like in Makefiles and Python scripts).
It is also possible to associate a comment with a specific item:
# A-A-P file to compile myprogram
SOURCE =
main.c # startup stuff
version.c # just the date stamp
help.c # display help stuff
TARGET = myprogram { comment = "build my program" }
|
Now run A-A-P with:
aap comment
And the output will be:
comments for main.aap:
target myprogram: build my program
This way of associating a comment with an item, using curly braces, is called
an attribute.
Various attributes can be associated with items. Another example is to force
compiling the "version.c" file each time, so that the date stamp it contains
is always up-to-date:
SOURCE =
main.c
version.c {force}
help.c
TARGET = myprogram { comment = "build my program" }
|
On MS-Windows an executable ends in ".exe". Adding this to "myprogram" would
result in a recipe that only works on MS-Windows. Better is to use the EXESUF
variable, which contains ".exe" only when appropriate.
SOURCE = main.c
TARGET = myprogram$EXESUF
|
You can see that it's very simple to use a variable in an assignment. The
dollar sign indicates a variable name is following.
If you might want to use a dollar in a file name, double the dollar sign. For
example, to use the "$temp.c" file as a source file:
A variable name consists of ASCII letters, numbers and the underscore. If
you want to append one of these after a variable name you must put parenthesis
around the variable name, so that A-A-P knows where it ends:
MAKENAME = Make
BUILDFILE = $(MAKENAME)file
|
index
A top level recipe
A program usually requires different methods for compiling and installation,
depending on the Operating System. A good way to handle this is to use one
recipe at the top level, which decides what OS-specific recipe to use next.
Here is an example that we will explain:
1 FTPBASE = ftp://ftp.vim.org/pub/vim ftp://ftp.us.vim.org/pub/vim
2 CVSROOT = cvs://:pserver:anonymous@cvs.vim.org/vim {path = aap}
3
4 :recipe {origin = $FTPBASE/aap/main.aap $CVSROOT}
5
6 @if OSTYPE == "posix":
7 :child unix.aap {origin = $FTPBASE/unix/vim.aap.gz $CVSROOT}
8 @elif OSTYPE == "mswin":
9 :child pc.aap {origin = $FTPBASE/pc/vim.zip//vim.aap $CVSROOT}
10 @else:
11 :error OS not supported: $OSTYPE
|
line 1: The URL of two ftp sites is specified where the files for this program
can be found. Nothing special is added, thus anonymous ftp is assumed.
line 2: For CVS a semi-URL is used, starting with "cvs://".
This will be explained below.
line 4: A line starting with ":" indicates an A-A-P command. This clearly
separates them from Python lines, assignments, etc.
The ":recipe" command specifies the origin of the recipe itself.
This is used to check if an updated version is available.
A new version is only obtained when specifically asked for, not each time the
recipe is executed.
The origin attribute specifies three locations.
That is because the variable $FTPBASE contains two items.
Using this variable with text appended results in two items, each with the
text appended.
After the expansion the command looks like this:
:recipe {origin = ftp://ftp.vim.org/pub/vim/aap/main.aap
ftp://ftp.us.vim.org/pub/vim/aap/main.aap
cvs://:pserver:anonymous@cvs.vim.org/vim {path = aap}}
|
Only one of the three URLs will be used. Normally it's the first one, but when
it doesn't work the next one is used. Note that it is very unusual to mix
using CVS and ftp, this is just for the example.
The first two URLs are ftp sites.
These are standard URLs.
For CVS the part after "cvs://" uses the standard CVSROOT syntax.
This is the same as what the cvs command uses after the "-d" option.
A-A-P also needs to know from which directory of the cvs repository the recipe
has to be obtained.
This is specified with the "path" attribute.
Note that this is a nested attribute: it applies to an item inside the
"origin" attribute.
For CVS the file name is not specified.
That is because CVS requires that the local name is the same as the name in
the repository.
line 6: A line starting with "@" indicates this is a Python script line. All
Python commands can be used this way. Here an "if" statement checks if the
recipe is being executed on a Unix system.
Note that in Python lines the same variables are available as in non-Python
lines, but without the "$" sign. A convention is that variables that are used
in non-Python lines are uppercase. This makes the recipe easier to understand.
The ":child" command tells A-A-P to use another recipe.
In this case an "origin" attribute is specified, which means the recipe can be
refreshed, the latest version can be obtained.
Notice that the recipe name after $FTPBASE is different from the local name
and ends in ".aap.gz".
A-A-P knows this means the recipe was compressed with gzip and will
automatically uncompress it before using it.
For CVS the name in the repository must be equal to the local name, therefore
the recipe name is not specified.
line 8: An else-if check for a MS-Windows system.
line 9: The ":child" command that specifies the recipe to use for MS-Windows.
The name after $FTPBASE actually specifies a zip archive in which the recipe is
contained. A-A-P knows how to extract the file from the archive.
line 10-11: There is no recipe for other systems. The ":error" command is used
to produce an error message and abort. Note that "$OSTYPE" is the same
variable as "OSTYPE" used in the Python lines.
index
A ported application
When an application already exists but for your system it requires a few
tweaks, a port recipe can do the work. This can also be used for applications
that work fine but you want to apply a number of patches on to add a feature.
Since A-A-P is prepared for doing all the work, usually you only need to
specify in the recipe where to find the files involved. Here is an example:
1 PORT_TYPE = autoconf
2
3 MASTER_SITE = ftp://ftp.download.net/pub/mirror/ http://sources.org/files/
4 MASTER_SUBDIR = foo/myprog
5 MASTER_FILE = myprog_src-1.3.tar.gz {md5 = 12341234}
6 docs/myprog_doc.tar.gz {md5 = 56785678}
7
8 PATCH_SITE = ftp://ftp.ucs.edu/~mike/myprog/
9 PATCH_SUBDIR = files/patches/
10 PATCH_FILE = myprog.001.gz myprog.002.add
11
12 DEPEND_BUILD = cproto {version = "1.[3-9]|[2-9].[0-9]"}
13 DEPEND_RUN = ghostscript
|
- PORT_TYPE
- This indicates this is a port and not everything is done with A-A-P
recipes. The type indicates what to do to build the port. "autoconf"
means the "configure" script will be run.
- MASTER_SITE
- List of sites where the original files can be downloaded from. This
goes down to the common directory for all downloadable files.
- MASTER_SUBDIR
- Path at $MASTER_SITE below which all required files can be found. This
is optional, the path can also be included in the files themselves. It's
also possible to use a full URL here, in case on some site a different path
is required.
- MASTER_FILE
- The list of files to download and unpack. Note that all the files are
used, while for $MASTER_SITE only the first one that works is used.
- PATCH_SITE, PATCH_SUBDIR, PATCH_FILE
- Like the MASTER_ variables, but specifies the location where to get
patch files.
- DEPEND_BUILD
- A list of applications that are required for building the target.
In the example the "cproto" program is required. The attribute specifies
a pattern to check for the right version.
- DEPEND_RUN
- A list of applications that are required for running the target.
It's like DEPEND_BUILD, but getting these applications can be postponed
until after building the target.
Note that all names are singular. This avoids having to think about which is
plural and which is not.
index
Building multiple variants
Quite often you want to compile an application for release with maximal
optimizing. But the optimizer confuses the debugger, thus when stepping
through the program to locate a problem, you want to recompile without
optimizing.
A-A-P provides a way to build several variants of the same application. You
just need to specify the variants and what is different about them. A-A-P
will then take care of putting the resulting files in a different directory,
so that you don't have to recompile everything when you select another
variant. Here is an example:
1 SOURCE = main.c version.c gui.c
2
3 :variant BUILD
4 release
5 CFLAGS = -O4
6 TARGET = myprog
7 debug
8 CFLAGS = -g
9 TARGET = myprogd
10
11 OFILES = $BDIR/main.o $BDIR/version.o
12
13 GUI ?= console
14 :variant GUI
15 console
16 motif
17 SOURCE += motif.c
18 gtk
19 SOURCE += gtk.c
|
This example contains quite a few interesting items. The essential is the use
of the ":variant" statement (line 3 and 14). It specifies a variable name and
a list of alternatives. The first one has "release" and "debug" variants,
the second "console", "motif" and "gtk". These can be selected independently,
thus there are six combinations: release-console, debug-console,
release-motif, debug-motif, release-gtk and debug-gtk.
If the "BUILD" variable was not assigned a value, the first alternative
"release" is selected. The user can select the debug variant by giving aap
the command line argument "BUILD=debug".
In line 13 a similar thing is done for the "GUI" variable. Here the default
is to build a console version, unless "GUI" has been set to "gtk" or "motif".
An configure check for the required GUI libraries might be used to set the
"GUI" variable.
The assignment to "OFILES" in line 11 uses $BDIR in between the two variants.
$BDIR is the directory used to store intermediate results for a specific
variant.
When switching between "debug" and "release" the ":variant" command will
change the value of $BDIR.
The "main.c" and "version.c" files don't depend on the value of GUI.
Therefore their object files can be shared between the GUI variants. This is
done here by using the value of $BDIR before the GUI variant changes it.
(Later this will be done in a nicer way.)
index
Using Python in recipes
In various places in the recipe Python commands and expressions can be used
for complicated things. You have already seen lines that start with "@".
When a few Python lines are used, it's best to line up the "@" characters:
@if OSTYPE == "unix"
@ for i in [ "solaris", "hpux" ]
@ if OSNAME == i
@ COMMERCIAL = "yes"
|
When the number of lines gets longer, it's easier to put them in a block:
:python
if OSTYPE == "unix"
for i in [ "solaris", "hpux" ]
if OSNAME == i
COMMERCIAL = "yes"
:end
|
The ":end" must appear on a line by itself, only a comment may follow.
An alternate terminator string may be specified, although it's very unlikely
that ":end" appears in the Python block.
In assignments and rules a Python expression can be used. Example:
@from glob import glob
SOURCE = `glob("*.c")`
|
Note that this uses a Python expression, not a Python command.
The resulting string or list of strings is inserted in-place before the
assignment is evaluated. In this example calling glob() results in a list of
all files matching "*.c", which is turned into a space-separated string.
This is actually not a very good example, because a "test.c" file that you
temporarily used will accidentally be included. It's better to list the
source files explicitly.
A common use of a Python expression is to change a list of source file names
to a list of target file names. Example:
OBJECT = `aap_sufreplace(".c", OBJSUF, SOURCE)`
|
The "aap_sufreplace()" function takes three arguments. The last argument is
the name of a variable that is a list of names (with attributes).
The first argument is the suffix which is to be replaced.
The middle argument is the replacement suffix.
In this example each name in $SOURCE ending in ".c" will be changed to end in
$OBJSUF, which is ".o" or ".obj".
A variable is often used for a list of items.
Parsing this string and turning it into a list can be done with the
aap_var2list() function. Example:
for s in aap_var2list(SOURCE):
print "source file " + s
|
If you also want to access the item attributes, you can turn the variable into
a list of dictonaries. Each dictionary contains a "name" entry that is the
name of the item and other entries are the attribute values. Entries starting
with an underscore are for internal use, the others are attributes.
This is how to list them all:
for d in aap_var2dictlist(SOURCE):
print "source item " + d["name"]
for a in d.keys():
if a != "name" and a[0] != '_':
print " attribute " + a + ": " + d[a]
|
There are several other functions supplied by A-A-P, so that you hardly ever
have to use the low level access in a recipe.
index
More about attributes
Most A-A-P variables are a list of items, and each item can have attributes.
These attributes are used for various purposes. One of them is to decide when
a source file has actually changed, such that it has to be compiled again.
A-A-P avoids using timestamps, because these are often unreliable (e.g., when
putting back an older version of a file, make doesn't notice the change).
The default method for source files is to use a signature. For C files, for
example, comments and changes in white space are ignored. This reduces the
number of compilations when the actual code didn't change. Especially useful
when adding a comment for a global variable. However, when using a tool to
extract documentation from the comments, another signature must be used.
Example:
myprog.hlp : $SOURCE {check = md5}
:sys getfunclist $SOURCE > myprog.hlp
|
To set default attributes these two commands can be used:
:depend_attribute "*.c" {check = c_signature}
:target_attribute 'all|clean' {type = notafile}
|
The first command sets attributes for the "depend" side of a rule, the files
right of the ":". The argument is a pattern that is matched against the file
names. In this case "*.c" matches with all C files. The check for a changed
file is then done with the "c_signature" method.
The second command sets attributes for the target side of a rule, the files on
the left of the ":". The pattern "all|clean" matches the targets "all" and
"clean" and gives them the attribute "notafile", which means they are not file
targets.
These examples are actually the defaults, thus these commands doesn't change
anything useful.
index
More about commands
The commands starting with a colon are special A-A-P commands. Here are a few
useful ones.
- :rule target-pattern : source-pattern
- Define a build rule for building "target" from "source".
This can overrule or add to the default build rules.
For example, to filter php files to obtain plain html, using the imaginary
"unphp" command:
:rule %.html : %.php
:sys unphp <$source >$target
|
Attributes can be added to the source pattern to specify the default check
to be used. When nothing is given, the default is used (md5).
- :refresh filename
- Get a file. Download it when needed.
- :unpack archive [file]
- Unpack an archive. When no specific file is mentioned, all files are
extracted. This is a portable way to unpack an archive.
- :update filename
- An explicit invocation for the rule to make sure filename is up-to-date.
This can be used when the normal dependencies are not sufficient.
- :require item
- Require the precense of a tool. When the tool is not available this
will fail. A try block can then be used to fall back to another tool.
Example:
myprog: $OBJECT
@try:
:require xyx-linker
LINKER = xyz-linker
@except:
@try:
:require abc-linker
LINKER = abc-linker
@except:
:error "can't find a useful linker"
$LINKER $OBJECT -o myprog
|
index
Pipe commands
A selection of commands can be connected together with a pipe.
This means the output of one command is the input for the next command.
It is useful for filtering text from a variable or file and writing the result
in a variable or file.
:cat file1 file2 | :filter re.sub('this', 'that', %s) > file3
|
This example concatenates the files "file1" and "file2", replaces all
occurrences of "this" with "that" and writes the result in "file3".
The following commands can be used in a pipe:
- :print [redir] text
- Evaluate $VAR things in the argument and print the result.
When redirection is specified the result is written to a file.
When a pipe follows, the result is passed on to the next command.
var = something
:print var is now $var | :assign var_text
|
This example assigns "var is now something" to the variable "var_text".
- :print [redir]
- Reads the input from a pipe and prints it.
When redirection is specified the result is written to a file.
When a pipe follows, the result is passed on to the next command (not very
useful!).
- :cat [redir] filename ...
- Concatenate all the files mentioned in the argument.
Where an argument is "-" the output of the previous command in the pipe is
used.
When redirection is specified the result is written to a file.
When a pipe follows, the result is passed on to the next command.
:print $var | :cat file1 - file2 >! file3
|
This example concatenates "file1", the value of variable "var" and "file2"
and writes the result in "file3".
- :assign varname
- Assign the output of the previous command in the pipe to variable
"varname".
- :filter python-expression
- Filter the output of the previous command in the pipe through a Python
expression. The expression must include "%s" somewhere, it is replaced by
the name of the variable that holds the text resulting from the previous
command.
When redirection is specified the result is written to a file.
When a pipe follows, the result is passed on to the next command.
- :tee [redir] filename ...
- Write the output of the previous command in the pipe to each file
mentioned as an argument.
When redirection is specified the same text is written to a file.
When a pipe follows, the same text is passed on to the next command.
:cat file1 file2 | :tee file3 | :assign var
|
This example concatenates files "file1" and "file2", writes the result in
"file3" and assigns the same text to variable "var"
The output of these commands can be redirected to a file instead of written
through a pipe to the next command. There are three variants:
- > filename
- Write a new file. If the file exists this is an error.
- >! filename
- Write a new file or overwrite an existing file.
- >> filename
- Append to a file. If it doesn't exist yet it is created.
index
Automatic dependency checking
When a source file includes other files, the targets that depend on it need to
be rebuild. Thus when "foo.c" includes "foo.h" and "foo.h" is changed, the
build commands to produce "foo.o" from "foo.c" must be executed, even though
"foo.c" itself didn't change.
A-A-P detects these implied dependencies automatically for the types it knows
about. Currently that is C and C++. For other types of files you can add
your own dependency checker. For example, this is how to define a checker for
the "tt" file type:
foo : foo.tt
:sys tt_compiler $source > $target
:autodepend tt
:sys tt_checker $source > $target
|
The "tt_checker" command reads the file "$source" and writes a dependency line
in the file "$target". This is a dependency like it is used in a recipe.
In a makefile this has the same syntax, thus tools that produce dependencies
for "make" will work. Here is an example:
foo.o : foo.tt foo.hh include/common.hh
|
This is interpreted as a dependency on "foo.hh" and "include/common.hh".
Note that "foo.o" and "foo.tt" are ignored. Tools designed for "make" produce
these but they are irrelevant for A-A-P.
Since the build commands for ":autodepend" are ordinary build commands, you
can use Python commands, system commands or a mix of both to do the dependency
checking.
index
Multiple targets
When a dependency has more than one target, this means that the build commands
will produce all these targets. This makes it possible to specify build
commands that produce several files at the same time. Example that compiles a
file and at the same time produces a documentation file:
foo.o foo.html : foo.src
:sys srcit $source -o $(target[0]) --html $(target[1])
|
People used to "make" must be careful, they might expect the build commands to
be executed once for each target. A-A-P doesn't work that way, because the
above example would be impossible. To run commands on each target this must
be explicitly specified. Example:
dir1 dir2 dir3 :
@for item in target_list:
:mkdir $item
|
The variable "target_list" is a Python list of the target items. "source_list"
is the list of source items. An extreme example of executing build commands
for each combination of sources and targets:
$OUTPUTFILES : $INPUTFILES
@for trg in target_list:
:print start of file >! $trg
@for src in source_list:
:sys foofilter -D$trg $src >> $trg
|
index
Publishing a web site
If you are maintaining a web site it is often a good idea to edit the files
on your local system. After trying out the changes you then need to upload
the changed files to the web server. A-A-P can be used to keep track of which
files changed and upload these files. This is called publishing.
Here is an example of a recipe:
FILES = `glob("*.html")` `glob("images/*.jpg")`
:attr {publish = scp://user@ftp.foo.org/public_html/%file%} $FILES
|
That's all. You just need to specify the files you want to be published and
the URL that says how and where to upload them to. Now "aap publish" will
find out which files have changed and upload them. The first time all files
will be uploaded (use "aap publish --touch" to avoid that).
The ":attr" command adds attributes to its arguments. In this case the
"publish" attribute is added, which specifies the URL where to upload a file
to. In the example the "scp" protocol is used, which is a good method for
uploading files to a public server.
If the same recipe produces some of the html files, you need to specify a
dependency for them. The ":publishall" command can then take care of the
publishing:
FILES = index.html images/logo.gif
:attr {publish = scp://user@ftp.foo.org/public_html/%file%} $FILES
all: $FILES
publish: $FILES
:publishall
index.html: header.part body.part footer.part
:cat $source >! $target
|
With this recipe, executing "aap" without arguments will produce the
"index.html" file when needed. "aap publish" will do the same and publish
updated files.
index
Version control with CVS
CVS is often used for development of Open Source Software.
A-A-P provides facilities to obtain the latest version of an application, and
for checking in changes you made.
For downloading you only need to specify the location of the CVS server and
the name of the module. Here is an example that obtains the Recipe Executive:
CVSROOT = :pserver:anonymous@cvs.a-a-p.sf.net:/cvsroot/a-a-p
refresh:
:refresh {refresh = cvs://$CVSROOT} Exec
|
Now do "aap refresh" and the directory "Exec" will be created and all files
obtained from the CVS server.
When you are working on a project and want to checkin your changes to the CVS
server, you need to specify which files you want to checkin. This is done by
supplying a "commit" attribute. Only these files will appear in the CVS
repository, other files will be removed. Example:
CVSROOT = :ext:$USERNAME@cvs.myproject.sf.net:/cvsroot/myproject
FILES = main.c common.h version.c
:attr {commit = cvs://$CVSROOT} $FILES
|
Doing "aap revise" will commit all your changes and remove files that appear
in CVS but don't have a "commit" attribute.
All this can also be done by using cvs commands. But it's easy to make a
mistake in typing the server name or forgetting one of the actions required.
After putting the magic words in a recipe once you know it will keep working.
And a user doesn't even have to know that CVS is being used. For another
version control system the commands he uses will be the same.
index
Recipe syntax definition
This defines the recipe syntax (more or less). NOTE: this may still change
quite a bit.
Notation:
| | separates alternatives |
() | grouping a sequence of items or alternatives |
[] | optional items (also does grouping) |
"" | contains literal text; """ is one double quote |
... | indicates the preceding item or group can be repeated |
EOL | an end-of-line, optionally preceded by a comment |
INDENT | an amount of white space |
A comment starts with "#" and continues until the end-of-line.
It may appear in many places.
Line continuation is done with a backslash immediately before an end-of-line.
It's not needed for an assignment and other commands that don't have a
build_block: the item continues if the following line has more indent than the
first line.
aapfile |
::= |
( [ aap_item ] EOL ) ... |
aap_item |
::= |
dependency | rule |
variant | otherfile | build_command |
|
dependency |
::= |
targets ":" [ dependencies ] [ EOL build_block ] |
targets |
::= |
item_list |
dependencies |
::= |
item_list |
build_block |
::= |
INDENT build_command [ EOL [ INDENT build_command ] ] ... |
|
rule |
::= |
":rule" pattern ... ":" pattern ... [ EOL build_block ] |
|
variant |
::= |
":variant" name ( EOL variant_item ) ... |
variant_item |
::= |
INDENT1 varvalue EOL INDENT2 build_block |
|
otherfile |
::= |
includefile | childfile |
includefile |
::= |
":include" item_list |
childfile |
::= |
":child" item_list |
|
build_command |
::= |
assignment | script_item | generic_command |
message |
|
assignment |
::= |
assign_set | assign_append | assign_cond |
assign_set |
::= |
name "=" item_list |
assign_append |
::= |
name "+=" item_list |
assign_cond |
::= |
name "?=" item_list |
|
script_item |
::= |
script_line | script_block |
script_line |
::= |
"@" script_command |
script_block |
::= |
":python" EOL ( script_command EOL ) ... ":end" |
|
generic_command |
::= |
":" command_name [ command_argument ] |
|
message |
::= |
info_msg | error_msg |
info_msg |
::= |
":print" expression ... |
error_msg |
::= |
":error" expression ... |
|
# A list can contain white-separated items and Python style expressions. |
|
item_list |
::= |
item [ white_space item ] ... |
item |
::= |
( simple_item | "(" item_list ")" ) [ attribute ... ] |
simple_item |
::= |
( expression | non_white_item ) ... |
attribute |
::= |
"{" name [ "=" expression ] "}" |
expression |
::= |
string_expr | python_expr |
string_expr |
::= |
""" text """ | "'" text "'" |
python_expr |
::= |
"`" python-expression "`" |
non_white_item |
::= |
non-white-text | "$" [ "(" ] name [ ")" ] |
index
|
|