Chapter 25. Automatic Package Install

Aap provides a very powerful feature: When a program is required and it does not exist on the system, it can be installed automatically.

Suppose a recipe specifies that a file is to be uploaded to a server with secure copy. This requires the "scp" command. When Aap cannot find this command it offers you the choice to install it. Aap will check what kind of system you have and find a recipe for it. This recipe is executed and it will install the package for you. Either by obtaining an executable program or by fetching the sources and building them. Then the original recipe continues and uses the "scp" command that has just been installed.

Note: This requires an internet connection!

How Does It Work?

The first step is to detect if a required program is present on the system. This is done internally when "scp" or "cvs" is to be used. You can also check explicitly in a recipe with the :assertpkg command. This uses the $PATH environment variable. Aap will also look in its own directory, because this is where previously installed programs are sometimes located. Also see the :progsearch command and the program_path() function.

When the program is not found, the user is asked what is to be done:

    Cannot find package "scp"!
    1. Let Aap attempt installing the package
    2. Retry (install it yourself first)
    q. Quit
    Choice: 

When the user types "1" the automatic install will be invoked. That will be explained below. An alternative is to install the package yourself. This is useful if you know how to do this or when you don't think the automatic mechanism will work. After the package has been installed you can enter "2" and Aap continues executing the recipe. The last choice is "a", which means you give up and abort executing the recipe.

In situations where the package name differs from the command name, or there are more complicated requirements, you can write a check yourself. When a package needs to be installed the :installpkg command can be used to have Aap install the package.

To install a package automatically Aap will download a recipe and execute it. To be able to execute the recipe in a proper environment a directory is created. On Unix this will be "~/.aap/packages/{package-name}/". The "boot.aap" recipe is downloaded from the A-A-P web site. This uses a simple PHP script that selects the recipe to use. Example:

    http://www.a-a-p.org/package.php?package=scp&osname=posix

The downloaded recipe contains further instructions for building and/or installing the package. This can be anything, thus it is very flexible. Usually the recipe finds out what kind of system you are using and selects another recipe to be used for it. Or it uses the standard package mechanism of your system.

If you are running a BSD system you probably have the BSD ports system installed (if not, you should install it!). This is a very well maintained system that takes care of installing almost any software you can think of. The only disadvantage is that you need to be super-user to use it. Aap will ask you if you want to do this:

    The devel/templ port appears to exist.
    Do you want to become root and use the port? (y/n) 

If you respond with "y" you will be asked to type the root password and Aap will invoke the commands to build and install the BSD port. If you respond with "n" the generic Unix method will be used (if there is one).

On MS-Windows it will often be possible to download an executable file. This works without asking questions. Since there is no standard directory for executables, they are often placed in the Aap directory. You might want to move them to a directory in $PATH if you want to use them at the command line.

On generic Unix systems (Posix, Linux) the recipe will attempt to download a source archive, unpack it and build and install it. This mostly works, but may fail on some systems. You may have to do some steps manually then, possibly by running "configure" with specific arguments. You can find the downloaded files in "~/.aap/packages/{package-name}/". The Aap message log "AAPDIR/log" may contain hints about what went wrong. If you know the solution, please report this to the maintainer of the recipe, so that it can be made to work automatically.

Adding Support For a Package

To make it possible to automatically install a package at least one recipe needs to be written. This recipe is placed on the A-A-P website, so that every Aap user can find it without typing a URL. You need to e-mail this recipe to the maintainer of the A-A-P website: webmaster AT a-a-p.org. Chose the package name carefully, because it must be unique. Mostly it is the name of the command that was to be executed, such as "scp" or "cvs".

If there is only one recipe it must take care of all systems. This is useful for a Python module, for example. Or when you want to redirect to another site where the recipes for this specific package are stored. Here is an example:

    all install:
        :execute generic.aap {fetch = http://www.foo.org/recipes/%file%} $buildtarget

This recipe redirects everything to another recipe, which is downloaded from the URL specified with the "fetch" attribute. Note that the "all" and "install" targets are supported. "all" is used to build the package (as the current user) and "install" to install it (possibly requiring root privileges).

Installing on MS-Windows

For MS-Windows you often have a different method to install a package. Especially when the command to be installed is available as one executable program. Here is an example for the "scp" command:

      # Package recipe for SCP on MS-Windows.
      # Maintainer:  Bram Moolenaar <Bram@a-a-p.org>
      # Last Update: 2003 May 1

      install:
              # We use the scp command from PuTTY, it appears to work well.
              # For info about PuTTY see:
              #	http://www.chiark.greenend.org.uk/~sgtatham/putty/
              :print This will install SCP on MS-Windows.
              dir = "`Global.aap_bindir`"
              :mkdir {force} $dir
              :fetch {fetch = http://the.earth.li/~sgtatham/putty/latest/x86/pscp.exe
                      ftp://ftp.chiark.greenend.org.uk/users/sgtatham/putty-latest/x86/pscp.exe}
                      $dir/scp.exe
              # That's all, it doesn't need to be installed.

The actual work is done with one :fetch command. It specifies two locations where the program can be downloaded. Specifying several locations is useful, because servers may be down or unreachable from behind a firewall (some companies disable access to ftp servers).

Note the use of "Global.aap_bindir". This is the directory where Aap itself is located with "bin" added. Putting the executable there avoids asking the user to make a choice. This directory is always searched for executable commands, it does not have to be in $PATH. Double quotes are used for the case the directory contains a space (e.g., "C:\Program Files").

Building and Installing on Unix

For Unix things are generally a bit more complicated. The best is to use the package mechanism of the system. Using a BSD port was mentioned above. Most systems have an equivalent mechanism. When this is not available you need to fall back to compiling from sources. Here is an example for "cvs":

      # Package recipe for CVS on Posix.
      # Maintainer:  Bram Moolenaar <Bram@a-a-p.org>
      # Last Update: 2003 May 1

      PREFIX =

      name = cvs-1.11.5
      tarname = $(name).tar.gz

      all install:
          # There is no check for a BSD port here, because BSD systems should have a
          # cvs command already.

          # Ask the prefix before compiling.
          @if not _recipe.PREFIX:
              @_recipe.use_asroot, _recipe.PREFIX = ask_prefix("cvs")
              @if not _recipe.PREFIX:
                  :error No prefix given

          # Should use a port recipe for this...

          @if buildtarget == "all":
              # Get the sources and build the executable.
              # This can be done by an ordinary user.
              :print This will build "cvs" on Posix machines.
              :update get-tar
              :sys tar xfz $tarname
              :cd $name
              :sys ./configure --prefix=$PREFIX

              # The GSSAPI stuff breaks the build for me on FreeBSD.  If we can find
              # a solution it can work on other systems..
              :cat config.h | :eval re.sub('#define HAVE_GSSAPI\\b', '', stdin) >! config.h

              :sys make
          @else:
              # Install the executable.
              # This may need to be done by root.
              :cd $name
              @if _recipe.use_asroot:
                  # Assume the directories already exist...
                  :asroot make $buildtarget
              @else:
                  :sys make $buildtarget

      get-tar {virtual}: $tarname {fetch = http://ftp.cvshome.org/$name/%file%}

Note that this recipe specifically mentions the version "1.11-5". This is a bit unusual. Better is to refer to the latest stable version. Unfortunately, for CVS the proper link is not available. This means the recipe has to be updated every time a new stable version is released.

The user has the choice of installing CVS for himself or for everybody on the system. Aap has a build-in function for this: ask_prefix(). When the user is root it will return "1" and "/usr/local/" without asking, assuming the super-user will want to install for everybody (why else would he be doing this as root?). Normal users will be asked to make a choice:

    Select where to install cvs:
    1. become root and install in "/usr/local/"
    2. become root and install in specified location
    3. install in your home directory "/home/mool/"
    4. install in a specified location
    a. abort
    choice: 

It is obvious what the choices will do. The "cvs" recipe then continues to obtain the sources, using ":update get-tar" to allow using a cached file. The archive is unpacked with "tar" and "configure" is run before invoking "make". This is the standard way how most Unix programs are build.

There is one extra step: While testing the recipe it was discovered that the configuration makes a mistake and defines "HAVE_GSSAPI", but that doesn't work. The recipe modifes "config.h" to fix this. This is not a nice solution, but it makes the building work. This kind of porting would actually better be done in a separate recipe. And by reporting the problem to the maintainers of the CVS configure script.

When the recipe is invoked with the "install" target the choice to install as root or not is used. This was stored in "_recipe.use_asroot" to avoid having to make the choice again when invoked a second time (the recipe is first invoked with "all" to build the program as the current user and then with "install" to do the actuall install, possibly as super-user). The :asroot command passes the command to a separate shell with root privileges. The user is asked confirmation for every executed command for safety.

Installing a Specific Package

This automatic package installation system is a nice way of installing a program without the need to know how it's done. You can also use it to install a package directly:

      % aap --install scp

This will attempt to install the "scp" package on your system. It works just like when Aap discovered that the "scp" command was needed for executing a recipe.

Obviously not just any package is available. Quite a few currently, but hopefully this will grow when people submit their packages. When a package cannot be found you get an error message:

      % aap --install foobar
      Aap: Creating directory "/home/mool/.aap/packages/foobar"
      Aap: Entering directory `/home/mool/.aap/packages/foobar'
      Aap: Attempting download of "http://www.a-a-p.org/package.php?package=foobar&osname=posix"
      Aap: Downloaded "http://www.a-a-p.org/package.php?package=foobar&osname=posix" to "boot.aap"
      Aap: Error in recipe "/home/mool/.aap/packages/foobar/boot.aap" line 3: Sorry, package 'foobar' is not available.
      %

Cleaning Up

Currently Aap does not delete the files downloaded and generated while installing a package. This is useful especially when something fails, so that you can read the log file "AAPDIR/log" and/or do part of the installation manually. But this does mean disk space is used.

In the comments produced while installing the package you can see which directory is used for the files. This depends on the system and environment variables. It should be one of these:

$HOME/.aap/packages
$HOME/aap/packages
$HOMEDRIVE$HOMEPATH/aap/packages
C:/aap/packages

It is fairly safe to delete the "packages" directory and everything it contains.

In case you are really low on disk space, you might want to check the Aap install directory for any programs that you no longer want to use. This is only relevant on MS-Windows.