Chapter 31. Adding A Language Module

This chapter explains how to add support for a language to Aap. Both for your own setup and for a module that is to be included in an Aap release.

Although the text explains modules for a language, this can also be used for adding a module for other purposes. It is a generic method to have Aap load a module that defines actions, sets variables, etc.

In a recipe a module can be used with the :import command. Thus when your recipe uses Java, you would add this line:

      :import java

For the standard modules see Chapter 39, Standard Modules.

Module Name

It is important to chose a good name for the module, because it must be unique on your system. For a programming language simply use the name of the language. For Java you would use "java", for the D language "d". In case there are variants of a language that are different enough to justify using a different module, add something to the name to make this clear. You can use letters, digits and the underscore. Examples: "foo", "foo2004", "foo_latest".

If you have a specific module with different functionality, you could prepend "my". For example, "myjava".

After the module has been imported, the variables in it can be accessed through the "m_name" scope, where "name" is the name of the module. For example the variable "$m_d.DMD" holds the command used for the "d" language to execute the compiler.

Note: It is not possible to have a variable and a scope with the same name. Thus if you have a module called "foo", which uses the scope "m_foo", you cannot have a variable "m_foo" anywhere in a recipe.

Since the modules are Aap recipes the file names always end in ".aap".

Module directories

Modules that are included with Aap are located in the Aap sub-directory "modules". If you are working on a module that is to be submitted for a next Aap release you may also want to put the module there for testing. But be aware that this directory may be overwritten or deleted when updating Aap!

Modules that you want to share with all users on your system should be placed in a system-wide directory. For Unix this usually is /usr/local/share/aap/modules. This can be changed with $AAPSYSDIR.

Modules that you only use for yourself are placed in $HOME/.aap/modules (for Unix and Mac OS X), $HOME/aap/modules/, $HOMEDRIVE/$HOMEPATH/aap/modules/ or c:/aap/modules/ (MS-Windows).

If you only want to change a few settings after the module is loaded, you can place these in the modules2 directory. See :import for details.

Parts Of A Module

In a module you will usually find these parts:

(1) filetype recognition

How to recognize files used in the module. For a module included with Aap this is normally missing, since it can be added to the generic Python filetype module. See Chapter 28, Customizing Filetype Detection and Actions.

(2) default values

Give default values to variables used in the commands further on. The user can change these values if he needs to. It is a good habit to put all specific command names, directories and arguments in a variable.

(3) object file suffixes

If the language uses different suffixes for object files, for static or dynamic libraries, this can be specified. See below.

(4) search for tools

Usually there are different ways to invoke the compiler on different platforms. And on one platforms various tools may be installed. The :toolsearch command is used to locate the tools. See below.

(5) actions, rules and routes

The generic methods for compiling and building. See below.

Example

This example is for the "D" language. This module is included with Aap.

      # (1) Filetype recognition
      :filetype
          suffix d d

      # (2) Default values
      DMD = dmd

      # (3) Object file suffixes
      # dll and lib objects are equal to normal objects.
      _top.D_LIBOBJSUF = $OBJSUF
      _top.D_DLLOBJSUF = $OBJSUF

      # (4) Search for tools
      # Actions are installed for every toolchain that exists.
      # The first one found sets $D_COMPILE_ACTION and friends.
      :toolsearch dmd

      # (5) Actions, Rules and Routes

      # :do compile

      :action compile object,libobject,dllobject,default d
          @if not _no.get("target"):
              target = `src2obj(fname)`
          # Use the d_build action for building and also for :dll and :lib.
          :attr {buildaction = d_build} $target
          @if _no.get("D_COMPILE_ACTION"):
              :do $D_COMPILE_ACTION {target = $target} $source
          @else:
              :sys $DMD $?DFLAGS -of$target -c $source

      :rule {global} {default} %$OBJSUF : {buildcheck = $DMD $?DFLAGS } %.d
          :do compile {target = $target} $source

      # :do build for object files resulting from "d" source files.

      :action d_build default
          @if _no.get("D_BUILD_ACTION"):
              :do $D_BUILD_ACTION {target = $target} $source
          @else:
              :sys $DMD $?DLINKFLAGS -of$target $source

      # default route

      :route {default} d object,libobject,dllobject
           compile

Object File Suffixes

Many languages use the same object file suffixes as used for C: ".o" on Unix and ".obj" on MS-Windows. But for some languages a different suffix is used and the suffixes for building a static and shared library are different.

The simplest example is when a language always uses the same suffix for all kinds of object files. For example, the "d" language does not make a difference between normal object files, static library objects, etc. This can be specified like this:

     _top.D_DLLOBJSUF = $OBJSUF
     _top.D_LIBOBJSUF = $OBJSUF

Note that the "_top" scope is used. If this would be left out the variables would only be set in the scope of the module and would not be available in the user recipe.

The name of the variable is made from the filetype in upper case, and underscore and the name of the generic variable. The src2obj() function will look for these variables. Note that the filetype is used, not the module name!

An extreme example is when all suffixes are different:

     _top.FOO_OBJSUF = .fo
     _top.FOO_DLLOBJSUF = .dfo
     _top.FOO_LIBOBJSUF = .lfo
     _top.FOO_LTOBJSUF = .tfo

An advantage of using a different suffix for every type of object file is that all types can be generated in the same directory.

If the compiler or linker does not support using different suffixes, but the various object files are different, the user must make sure that the object files are put in different build directories. This can be done by specifying an attribute on the source files to use a different build directory:

    :program foo : foo.c                    # uses $BDIR
    :dll foo : foo.c {var_BDIR = $BDIR-dll}
    :lib foo : foo.c {var_BDIR = $BDIR-lib}

Searching For Tools

When there is only one compiler that always uses the same (type of) arguments you can simply invoke it directly from the actions in the module. Otherwise, Aap can search for tools that are currently installed. The working of tools is explained in Chapter 30, Customizing Default Tools

As an example, let's look at how Aap searches for C compilers:

      @if osname() == 'mswin':
          :toolsearch msvc mingw gcc icc bcc
      @elif osname() == 'os2':
          :toolsearch icc gcc msvc

The :toolsearch command is used with a list of tools that need to be checked for. The first tool for which the exists() function returns True is then used. This works by setting variables to the name of the action to be used. For example, the "msvc" tool sets $C_COMPILE_ACTION to "compile_msvc".

The list of tools to be searched depends on the platform. Note that the "msvc" tool is the first choice for MS-Windows, but the last choice for OS/2. This ordering is based on what works best for most people.

Note that for Unix systems there is no search for tools. That is because the default command for the C compiler "always" works there.

There is one trick you should be aware of, which is using the "buildaction" attribute on object files. When a specific compiler is used to turn a source file into an object file, this often means a specific action needs to be used to turn this object file into a program or library. The compile action must then add a "buildaction" attribute to the object file, so that the generic build action can inspect the object files and invoke the proper build action. You can see this in the example above: The compile action for the "d" language contains:

          # Use the d_build action for building and also for :dll and :lib.
          :attr {buildaction = d_build} $target

The action is not executed when the target is already up-to-date. Therefore the "buildaction" attribute should also be set by the dependency or rule, using a ">always" section. See the section called “Command block sections”. Example:

          :rule %$OBJSUF : $.cpp
              >always
                  :attr {buildaction = cxx_build} $target
              >build
                  :do compile {target = $target} $source

Note that the generic build action cannot handle the situation that object files with different "buildaction" attributes are linked together. And object files that do not have a "buildaction" attribute are taken along, Aap assumes they can be linked together with the object files that specify the "buildaction" attribute. If this does not do what you intend, you can specify another build action to use with $BUILD_ACTION. Similarly, $BUILDDLL_ACTION and $BUILDLIB_ACTION are used for building libraries. Look in the distributed default.aap for the details.

Actions, Rules and Routes

Actions are used to specify how files are to be compiled and build into a program or library. Rules are used to specify which actions to use for files matching a pattern. Routes are used to specify which actions to use to turn one filetype into another. Thus these three items are closely related.

The most obvious thing for a language module is to define an action for compiling the language. Here is an excerpt from the "d" module:

      :action compile object,libobject,dllobject,default d
          @if not _no.get("target"):
              target = `src2obj(fname)`
          # Use the d_build action for building and also for :dll and :lib.
          :attr {buildaction = d_build} $target
          @if _no.get("D_COMPILE_ACTION"):
              :do $D_COMPILE_ACTION {target = $target} $source
          @else:
              :sys $DMD $?DFLAGS -of$target -c $source

This action is specified for all kind of object files, including dllobject and libobject. If compiling for a library has to be done differently you should specify a separate action for libobject and/or dllobject.

The action starts with checking if $target is set. In rare cases it isn't set and needs to be derived from $source. Since the object file suffixes have been defined (see above) calling src2obj() takes care of this. $fname is the first item in $source.

When a D file has been compiled into an object file it needs to be build in a special way. To accomplish this the "buildaction" attribute of the target is set to "d_build". This action is also defined in the D module.

The check for $D_COMPILE_ACTION takes care of invoking the action that was defined by a tool. The first tool found with :toolsearch will set this variable. It can later be changed with a :usetool command.

When $D_COMPILE_ACTION is not set, the default command to compile a D file is executed with :sys. This happens when no appropriate tool could be found. Often this is used for Unix, where the compile command is the same for various compilers. The user may set $m_d.DMD to the name of the compiler to be used and set $DFLAGS to additional arguments. What could be added here is using $DINCLUDE and $DDEFINE, like what $INCLUDE and $DEFINE do for C. A compiler may also use $DEBUG and $OPTIMIZE. Note that these can't be directly used as an argument but must be translated.

The other actions defined in the module are similar. You can see in the example for the D module how the build action is defined:

      :action d_build default
          @if _no.get("D_BUILD_ACTION"):
              :do $D_BUILD_ACTION {target = $target} $source
          @else:
              :sys $DMD $?DLINKFLAGS -of$target $source

Defining a rule is optional. The :rule command selects the source and target based on a match of the file name with a pattern. This is useful when a list of source files is defined without telling how they are to be turned into object files. This is the way the rule is defined in the D module:

      :rule {global} {default} %$OBJSUF : {buildcheck = $DMD $?DFLAGS } %.d
          :do compile {target = $target} $source

The {global} option is needed to make the rule available everywhere (otherwise it would only work in the module recipe). The {default} option allows overruling this rule without getting a warning.

The :route command is used to tell Aap how to turn a source file into an object file. This is used for the :program, :dll and :lib commands. For the D language it's simply a matter of invoking the compile action. for other languages it may consist of several steps with intermediate results.

      :route {default} d object,libobject,dllobject
           compile