Chapter 28. Customizing Filetype Detection and Actions

See Chapter 8, Filetypes and Actions for a simple example how to define a new filetype.

Filetype Detection

A-A-P detects the type of a file automatically. This is used to decide what tools can be used for a certain file.

The detection often uses the name of the file, especially the suffix. Extra suffixes like ".in" and ".gz" are ignored. Sometimes the contents of the file is inspected. for example, on Unix a shell script is recognized by "#!/bin/sh" in the first line.

To manually set the file type of an item add the "filetype" attribute. This overrules the automatic detection. Example:

        foo.o : foo.x {filetype = cpp}

Most detection is already built in. If this is not sufficient for your work, filetype detection instructions can be used to change the way file type detection works. These instructions can either be in a file or directly in the recipe:

        :filetype  filename
        :filetype
            suffix p pascal
            script .*bash$ sh

The advantage of using a file is that it will also be possible to use it when running the filetype detection as a separate program. The advantage of using the instructions directly in the recipe is that you don't have to create another file.

For the syntax of the file see Chapter 38, Filetype Detection.

It is sometimes desired to handle a group of files in a different way. For example, to use a different optimizer setting when compiling C files. An easy way to do this is to give these files a different filetype. Then define a compile action specifically for this filetype. Example:

        :attr {filetype = c_opt} bar.c foo.c
        :action compile c_opt
            OPTIMIZE = 3
            :do compile $source

When you define an action (or a route), Aap checks that the filetypes you use are known filetypes, i.e. mentioned somewhere in a :filetype command. If you just make up filetypes and use them in actions, Aap will give you a warning. This helps detect misspellings and the like. However, for the "optimized C" filetype above, this leads to a warning where you do not want one: c_opt is a proper filetype in this context. In order to declare a filetype without giving any rules to detect files of that type, use declare in a :filetype command:

    :filetype
	declare c_opt

The detected filetypes never contain an underscore. A-A-P knows that the underscore separates the normal filetype from the additional part. When no action is found for the whole filetype, it is tried by removing the "_" and what comes after it. (This isn't fully implemented yet!)

Care should be taken that an action should not invoke itself. For example, to always compile "version.c" when linking a program:

        :attr {filetype = my_prog} $TARGET
        :action build my_prog object
            :do compile {target = version$OBJSUF} version.c
            :do build {target = $target} {filetype = program}
                                        $source version$OBJSUF

Without "{filetype = program}" on the :do build" command, the action would invoke itself, since the filetype for $TARGET is "my_prog".

Attributes on source arguments of the :do that start with "var_" are passed on to the executed action as variables. Examples:

          :do build foo.c {var_OPTIMIZE = 2}

          :attr {var_OPTIMIZE = 2} foo.c
          :do build foo.c

In the same way an attribute starting with "add_" is added to a variable. Example:

          :do build foo.c {add_DEFINE = -DBIG}

The value of the attribute is not added when it already appears in the variable. This avoids adding it two or more times.

Variable values for the executed action can also be passed by giving an attribute just after the action:

          :do build {OPTIMIZE = 2} foo.c

And yet another method is to define a user scope and use this for the action:

          s_foo.OPTIMIZE = 4
          :do build {scope = s_foo} foo.c

Executing Actions

The user can select how to perform an action for each type of file. For example, the user may specify that the "view" action on a file of type "html" uses Netscape, and the "edit" action on a "html" file uses Vim.

To start an application:

        :do actionname filename

This starts the action "actionname" for file "filename". Variables to be used by the commands can be specified after the action as attributes:

        :action convert default
            :sys convertcmd $?arg $source >$target
        ...
        :do convert {arg = -r} filename

The filetype is automatically detected from the file name(s) and possibly its contents. To overrule the automatic filetype detection, specify the filetype as an attribute on the file:

        :do convert filename {filetype = text}

This changes the filetype of the source file, the input of the action. To change the filetype of the output, the target of the action, use a "filetype" attribute on the action name:

  :do convert {filetype = html} filename {filetype = text}

Multiple filename arguments can be used:

        :do action file1 file2 file3

For some actions a target can or must be specified. This is done as an attribute on the action name:

        :do convert {target = outfile} infile1 infile2

Attributes of the input filenames other than "filetype" are not used to select the action that will be executed.

The "async" attribute can be used to start the application without waiting for it to finish. However, this only works for system commands and when multi-tasking is possible. Example:

        :do email {async} {remove} {to = piet} {subject = done building} logmessage

The "remove" attribute specifies that the files are to be deleted after the command is done, also when it fails.

When the filetype contains a "_" and no action can be found for it, the search for an action is done with the part before the "_". This is useful for specifying a variant on a filetype where some commands are different and the rest is the same.

"action" is usually one of these:

viewLook at the file. The "readonly" attribute is normally set, it can be reset to allow editing.
editEdit the file.
email
Send the file by e-mail. Relevant attributes:  
editedit before sending
subjectsubject of the message
todestination
buildBuild the file(s), resulting in a program file. The "target" attribute is needed to specify the output file.
compileBuild the file(s), resulting in object file(s). The "target" attribute may be used to specify the output file. When "target" is not specified the action may use a default, usually `src2obj(fname)`. See here
extractUnpack an archive. Requires the program for unpacking to be available. Unpacks in the current directory.
preprocRun the preprocessor on the file. The "target" attribute can be used to specify the output file. When it is missing the output file name is formed from the input file name with `src2obj(fname, sufname = None)` and then appending ".i".
referenceRun a cross referencer, creating or updating a symbol database.
stripStrip symbols from a program. Used to make it smaller after installing.

Examples:

        :do view COPYRIGHT {filetype = text}
        :do edit configargs.xml
        :do email {edit} bugreport
        :do build {target = myprog} main.c version.c
        :do compile foo.cpp

Default Actions

These are defined in the default.aap recipe.

:action edit default

Uses the environment variable $VISUAL or $EDITOR. Falls back to "vi" for Posix systems or "notepad" otherwise. You can set the Aap variable $EDITOR to the editor program to be used.

The {line = 999} attribute can be used to specify a line number to jump to. The {byte = 999} attribute can be used to jump to a character position in the file.

:action edit gif,jpeg,png

Uses the environment variable $PAINTER. You can set the Aap variable $PAINTER to the image editor program to be used.

The {line = 999} attribute can be used to specify a line number to jump to. The {byte = 999} attribute can be used to jump to a character position in the file.

:action depend

The default dependency checker. Works for C and C++ files. Uses gcc when available, because it is faster and more reliable. Uses an internal function otherwise. This attempts to handle the situation that a header file doesn't exist (when it needs to be fetched or build) but that is not fully implemented yet.

:action view html,xml,php

Uses the environment variable $BROWSER. Searches a list of known browsers if it's not set.

:action email default

Uses the environment variable $MAILER. Falls back to "mail".

:action build default default

Does: ":sys $CC $LDFLAGS $CFLAGS -o $target $source $?LIBS"

:action compile object c,cpp

Does ":sys $CC $CPPFLAGS $CFLAGS -c -o $target" $source

:action preproc default c,cpp

Does ":sys $CC $CPPFLAGS -E $source > $target"

Specifying Actions

Applications for an action-filetype pair can be specified with this command:

        :action action in-filetype
                commands

This associates the commands with action "action" and filetype "in-filetype". The commands are A-A-P commands, just like what is used in the build commands in a dependency.

The "in-filetype" is used for the source of the action, the type of the target is undefined. It is also possible to specify an action for turning a file of one filetype into another filetype. This is like a :rule command, but using filetypes instead of patterns.

        :action action out-filetype in-filetype
                commands

Actually, specifying an action with one filetype is like using "default" for the out-filetype.

Several variables are set and can be used by the commands:

$fnameThe first file name of the :do command.
$sourceAll the file names of the :do command.
$filetypeThe detected or specified filetype for the input.
$targettypeThe detected or specified filetype for the output.
$actionThe name of the action for which the commands are executed.

Furthermore, all attributes of the action are turned into variables. Thus when ":do action {arg = -x} file " is used, $arg is set to "-x". Example for using an optional "arg" attribute:

        :action view foo
                :sys fooviewer $?arg $source

In Python code check if the {edit} attribute is specified:

        :action email default
            # when $subject and/or $to is missing editing is required
            @if not globals().get("subject") or not globals().get("to"):
                edit = 1
            @if globals().get("edit"):
                :do edit $fname
            :sys mail -s $subject $to < $fname

"action" and "ftype" can also be a comma-separated list of filetypes. There can't be any white space though. Examples:

        :action view html,xml,php
            :sys netscape --remote $source
        :action edit,view text,c,cpp
            :sys vim $source

"filetype" can be "default" to specify an action used when there is no action specifically for the filetype.

When using a variable in the commands of an action without specifying a scope, or using the "_no" scope, its value is obtained from the first scope where it is defined from this list:

  1. The local scope, the scope of the action commands.

  2. Invoking command blocks. That is, the scope of the command block that invoked the action with a :do command. Then the scope of the command block that invoked that command block, and so on. This excludes the toplevel.

  3. The recipe in which the ":do" command was invoked, its parent and so on. This goes on until and including the toplevel.

  4. The recipe in which the action was defined, its parent and so on. This goes on until the toplevel.

The meaning of this all is that when using a variable such as $FOOCMD, you can give it a default value in the recipe where the action is defined. The location where the action is later invoked from can define a different value. Example:

        FOOCMD = foocc
        done =
        :action compile object foo
            :sys $FOOCMD $source > $target
            _recipe.done += $-source
        :action foolist default
            :print Compiled $_recipe.done

The action can be used like this:

        all:
            FOOCMD = foocc -O            # used instead of the default $FOOCMD
            :do compile {target = opt/piet.o} piet.f
            done = nothing               # This has no effect
            :do foolist any

In action the "done" variable is used with the "_recipe" scope prepended, thus a "done" variable in the scope of the commands that invoked this action will not be used.

Note that the "async" attribute of the :do command may cause the :sys command to work asynchronously. That is done because the "async" attribute is turned into the "async" variable for the :action commands. If you don't want a specific :sys command to work asynchronously, reset "async":

        :action view foo
            tt = $?async
            async = 
            :sys foo_prepare
            async = $tt
            :sys foo $source

However, since two consecutive :sys commands are executed together, this should do the same thing:

        :action view foo
            :sys foo_prepare
            :sys foo $source

Quite often the program to be used depends on whether it is available on the system. For example, viewing HTML files can be done with netscape, mozilla, konquerer or another browser. We don't want to search the system for all kinds of programs when starting, it would make the startup time quite long. And we don't want to search each time the action is invoked, the result of the first search can be used. This construct can be used to achieve this:

        BROWSER =
        :action view html
            @if not _recipe.BROWSER:
                :progsearch _recipe.BROWSER netscape mozilla konquerer
            :sys $_recipe.BROWSER $source

The generic form form :progsearch is:

        :progsearch varname progname ...

The "_recipe" scope needs to be used for variables set in the commands and need to be used after the action is finished.

The first argument is the variable name to which to assign the resulting program name. When none of the programs is found an error message is given. Note that shell aliases are not found, only executable programs.

When the action uses a temporary file, make sure it is deleted even when one of the commands fail. The Python try/finally construction is ideal for this. Example:

        :action filter .pdf .txt
            tmp = `tempfname()`
            @try:
                :sys extract_docs $source >$tmp
                :sys docs2pdf $tmp $target
            @finally:
                :delete $tmp
            :print Converted $source to $target

When the ":sys" commands execute without trouble the ":delete" and ":print" commands are executed. When the first ":sys" commands causes an exception, execution continues with the ":delete" command and then the exception is handled. Thus the second ":sys" command is skipped and the ":print" command is not reached.

More examples:

        :action compile object c,cpp
            :sys $CC $CPPFLAGS $CFLAGS -c $source
        :action build object
            :sys $CC $LDFLAGS $CFLAGS -o $target $source

Specifying Actions in Python

When an action needs to be defined with more properties, a Python function can be used. Example:

        :python
            define_action("buildme", 1, """
                @if DEFER_ACTION_NAME:
                    :do $DEFER_ACTION_NAME {target = $target} $source
                @else
                    :sys me < $source > $target
                """,
                outtypes = ["me"],
                intypes = ["moo"],
                defer_var_names = ["BUILDME_ACTION"])

This defines an action almost like this :action command

        :action buildme {primary} me moo
            @if _no.get("BUILDME_ACTION"):
                :do $BUILDME_ACTION {target = $target} $source
            @else:
                :sys me < $source > $target

An important difference is that when the action defined with $BUILDME_ACTION supports different input and output filetypes, the action defined with define_action() will use these. The action defined with ":action" will always support "me" and "moo", even though the actually used $BUILDME_ACTION action supports different types.

More info about the define_action() function in the reference manual.