A-A-P logo     




The architecture is made out of modules and interfaces between these modules. The descriptions below explain the interfaces that are worth describing. This is done in an informal way.

The functionality mentioned here is what will eventually be available. Not everything will be available in the first release. The plan shows how and when the modules will be made.

All modules interface with the Recipe Executive module. Therefore the interfaces are mostly named after the module the Recipe Executive interfaces with:
Issue Tracker Wrapper
Version Control Wrapper
Filetype Detection
Cross Referencer
Dependency Checker
Automatic Configurer
GUI and Console IDE

Issue Tracker Wrapper

Issue trackers tend to have very different ways to do their work. Some provide a web page for entering and modifying issues. Others require sending an e-mail to a maintainer or use of a specific program. Because of this diversity, the recipe writer must specify the commands for reporting and changing an issue himself.

Calling the following an interface between two modules is a stretch. But that's how it fits in the rest of the design, I didn't bother making a separate page for this.

Towards the user two targets are provided, so that he can do:

	:aap report
to report an issue, and:
	:aap tracker
to change the status of an issue. The recipe writer must define the commands for these targets, there is no default. Obviously there is no default way to report errors (would be nice to have a central organ that fixes the bugs in all applications!).

All the recipe commands can be used. A few of them are especially useful for the "report" and "tracker" targets:

  • ":do view": start a browser on the page where issues are to be reported. Possibly a few arguments can be added in the URL. In the future there way also be a way to fill in part of a form. For the time being, the suggestions for text to be entered in certain fields can be written in a file and presented to the user with: ":do view file".
  • ":do email {edit} file": send a message to the maintainer or automated tracking system. The file can be filled with relevant information, such as the version number and name of the user. The user will have to edit the message for the actual report.
  • ":sys" can be used for any other command, for example ":sys send-pr". Possibly preceded by ":require send-pr" to make sure the program is available.


Version Control Wrapper

There is one Version Control Wrapper (VCW) for each Version Control system. Each VCW is implemented as a Python module. The following describes the interface that each module must implement.

In the recipe the "commit" attribute is used on an item that is under version control. It has the form: "XXX://argument". "XXX" specifies the type of version control and is the name of the module. E.g., "cvs://" is used for CVS. The argument is used to specify the location of the version control repository. Its syntax depends on the version control system. The argument can be omitted if the module can figure out the location by itself (e.g., CVS remembers the name of the server after obtaining the initial version).

Each version control module must provide these functions:

XXX_command(url_tail, url_dict, nodelist, action)
Execute a version control command.
XXX_list(url_tail, url_dict, directory, recursive)
List entries currently in the version control system.
Common arguments are:
String: part of the "commit" attribute after XXX://. For: "{commit = cvs:// {path = bar}}" url_tail is: "". It can be empty, in which case the module must figure out the required parameters itself. Note: %file% must be expanded for each node. You can use repl_file_name() for this (see below).
Dictionary of the "commit" attribute and its attributes. For: {commit = cvs://asdf/root {path = foobar}} url_dict is: {"name" : "cvs://asdf/root", "path" : "foobar"}. Useful attributes:
path to directory with nodes
message for commit log
name for this version
Further arguments for XXX_command() are:
List of Node objects for which the action is to be carried out. The Node class is defined in the Node module.
String: Command to be executed. First argument of the ":verscont" command. Predefined are:
Update local version from repository. No-op for files that are already up-to-date.
Update the repository for local changes. No-op for a file that didn't change. The file is to be added to the repository when necessary. Do checkout/checkin when checkout is required. Don't change locking of the file.
Like commit and leave the file unlocked.
Add a tag to the current version.
Like fetch and additionally lock for editing when possible.
Like commit and leave the file unlocked.
Remove lock on file, don't change file in repository or locally. No-op for files that aren't locked.
Add file to repository. File does exist locally.
Remove file from repository. File may still exist locally.
The return value for XXX_command() must be the list of nodes for which the action failed. When everything worked an empty list is to be returned.

The function can throw a UserError exception when called with wrong arguments. UserError is defined in the Error module.

When there is something wrong that prevents version control to work (e.g., the repository doesn't exist) it should silently returning zero. A-A-P may try another method to obtain the file.

Further arguments for XXX_list() are:

Absolute path name of the directory for which the listing is requested.
When non-empty or non-zero: recursively list directories.
The return value must be a list of full file names that includes all the items in the repository in the specified directory. Directories are also to be included.

repl_file_name(name, node) can be used to replace "%file%" with the name of the node. The result is returned. For example, when "name" is "" and "" is "foo/bar.c" the result is "". However, in the future something more complicated may be done (esp. to handle changing directory).


Filetype Detection

The purpose of the Filetype Detection module is to detect the type of a file from its name or contents. The module is used in the Recipe Executive and the Cross Referencer. For the format of the files used to specify the filetype detection see the documentation: filetype.txt.

The Filetype module is implemented in Python. It provides these functions:

Returns the filetype for the file "fname". Both "fname" and the returned filetype are strings. This uses the default rules and any rules added by the following functions. When the filetype could not be detected None is returned.
ft_check_dir(dir [, errmsg)]
Scan directory "dir" for "*.afd" files, which are loaded with ft_read_file(). When "errmsg" is provided and is non-zero, an error message is given for a non-existing directory. Otherwise it is silently skipped.
Read file "fname" for detection rules.
ft_add_rules(str, lnum)
Add file type detection rules from "str". See "filetype.txt" for the syntax. "lnum" is the first line number in the file where "str" comes from. Use one when reading a whole file.
The DetectError exception is raised when there is something wrong. Not for an unknown filetype.

The filetype detection module can also be used as a separate program. This is useful for non-Python programs. The usage is: [-I ruledir] ...  [-f rulefile] ... filename
This will print the filetype of "filename" on stdout. When the type could not be detected the result is "None".

The "-I ruledir" argument can be used to specify a directory to load *.afd (Aap Filetype Detection) files from. These add rules for filetype detection. These are the default directories which are always scanned:

  • /usr/local/share/aap/afd/
  • ~/.aap/afd/
The "-f rulefile" argument can be used to specify a file to load rules from.

Cross Referencer

The Cross Referencer uses several interfaces. Most of them are provided to the outside world for whatever wants to use the Cross Referencer. These are not restricted to be used by specific applications. A few interfaces are for the Cross Referencer to use the Recipe Executive to perform certains tasks. These are decicated to interfacing these two modules.

The interfaces are:

Database updating as a program.
The Cross Referencer scans specified files and stores the information about isolated symbols and their context in a database file, replacing existing information for each file. The program is called "aref". This program can be invoked from a shell, directly by the user. Or started with a ":system" command in a recipe. The arguments for the program are:
  • The "-u" option, indicating the database is to be updated.
  • The list of filenames to be scanned.
  • A "-t" option, followed by the name of a filetype. This filetype is used for the following filenames, overriding the automatic detection. "-t filetype" can appear mixed in with the filenames. "-t auto" can be used to go back to automatic detection.
  • Optionally the argument "-o" followed by name of the database file to be written. When omitted a database file is created in each directory.
Example: "aref -u foo.c -t text README -t auto bar.h -o AAP/ref"
Database updating as a Python module.
This provides the same functionality as the program, but avoids starting a new interpreter. One function is provided:
   	ref_update(file_dictlist [, outfile])
"file_dictlist" is a list of dictionaries. Each file is one list item and provides a "name" and optionally a "filetype" entry. The "name" should be an absolute path name, not depending on the current directory. When "outfile" is specified it is to be used for the database file. Returns an error message if it fails, None if it works.
Symbol lookup as a program.
Produce grep-like output for a query or start an application for a location. This currently isn't used by other parts of A-A-P, thus it is not further specified.
Symbol lookup through a Python module.
Used by the IDE. Could also be used by an editor.
TODO: this is not used by the Recipe Executive, it will be specified later.
  • lookup a symbol literally or with a regexp
  • lookup with a query, for example to lookup the members of a class
  • specify where to search (file, directory, tree, whole system) and/or which database files to use
  • when looking up where a symbol is defined or used, return a list of files, optionally with a list of locations inside the file (much slower)
  • possibility to pass text directly instead of a file name, to lookup a symbol in a modified file without saving it
To Recipe Executive: fetch a file.
When a file referenced in the database does not exist, the Cross Referencer may use the Recipe Executive to download it. This interface may also be used by an editor or another program that has used the Cross Referencer to obtain the name of a file.
This is implemented as a Python function in the "Main" module:
   	Main.fetch(filename [, argv])
"filename" must be an absolute path name, not depending on the current directory. The optional "argv" argument is a list of command line arguments that the Recipe Executive uses. Most useful is ["-f", "file.aap"] to use recipe "file.aap" instead of the default "main.aap".
The function returns non-zero for success, zero for failure. By default it will search for a "main.aap" recipe in the directory of the file and directories above it. This recipe must contain information for fetching for the operation to work successfully.
To Recipe Executive: start a command to view a symbol.
The Recipe Executive provides the ":do" command to perform actions on different types of files. The program to be used is specified with recipes and A-A-P commands are used to start the program. The Cross referencer uses the equivalent of ":do view" and ":do edit" to view and/or edit a file.
This is implemented as a generic Python function in the "Main" module:
   	Main.execute(string [, argv])
"string" contains the commands to be executed, just like the A-A-P commands that appear in a recipe. Newlines can be used to separate lines. "argv" is an optional list of command line arguments that the Recipe Executive uses.
The function returns an error message if something went wrong (although this cannot always be detected), None otherwise. The Cross Referencer uses a command like this:
   	:do view {line = 123} {byte = 33}
The optional attributes to specify the position in the file are:
  • line: line offset in the file. First line is one.
  • byte: byte offset from the start of the file, or when "line" is specified, from the start of the line. First byte is one.
To Recipe Executive: update database.
When the Cross Referencer is started with an option to update the database when necessary, it invokes the Recipe Executive to do this. The recipe (either default or specified) contains the list of files to be scanned. The Recipe Executive will then in turn invoke the Cross Referencer for the list of files.
This is implemented with the same Main.execute() function as above. The command with which it is invoked is:
   	:update reference
To Filetype Detection module: detect the type of a file.
This is the interface the Filetype Detection module provides.


Dependency Checker

Since dependencies are only used in a recipe, the methods to use for automatic dependency checking are also specified with a recipe. The ":autodepend" command is used to specify a block of commands for checking the dependencies for one file with a specific filetype. See the documentation for how ":autodepend" is used.

For the time being, no specific items are provided. The checking has to be done with system or Python commands. The output of these commands must result in a file with the detected dependencies. The format of this file is like a dependency: "target : file1 file2". This means that commands used to produce Makefile dependencies should work.


Automatic Configurer


GUI and Console IDE

The GUI IDE and the Console IDE work the same way and have the same interfaces. However, some functionality may be missing in the Console IDE (e.g., opening a new window with an internet browser). The name IDE will be used here for the superset of the interfaces of both modules.

The IDE interfaces with many other modules, because it forms a framework for modules to work together. Additionally, there are interfaces to external applications, such as editors, debuggers and viewers. For completeness these will be briefly described as well, because we want the interfaces to be consistent.

IDE using the Recipe Executive

Since the project file for the IDE is a recipe, and the Recipe Executive knows how to deal with recipes, the IDE uses the Recipe Executive for many things:
store project settings
The settings are stored in a recipe. To be able to read them back a number of rules have to be defined how the settings are stored. These restrict the syntax used, otherwise the IDE doesn't understand the settings.
A number of variables is predefined to have a known meaning. These must be set with literal strings (thus no Python code or $var things). At the moment these are defined:
files to be compiled into $TARGET
files to be compiled into $TARGET_XYZ
result of building $SOURCE (can only be one file)
result of building $SOURCE_XYZ (can only be one file)
files used but not compiled
files used to generate documentation
documentation files, not generated
list of relevant reference database files
":attr" commands are used to specify version control, URL for publishing, etc. The argument includes the list of files involved (literal strings, as above). To avoid confusion with other ":attr" commands, a comment above it specifies what it is for:
# *IDE: version control
files under version control and the attributes specifying the type and location of the version control system.
# *IDE: published files
files to be published, and the attributes used for publishing.

The IDE can change anything in a recipe that it understands. The user can edit the recipe to have it do non-standard tasks. How these conflicting goals are achieved will be further defined over time.

The IDE will use skeletons for often used tasks. For example, building C code with autoconf. To be able to read back specific items in a recipe, also when the user has edited other parts, the IDE uses comment markers. For example, to mark the name of the file that stores the configure arguments: "# *IDE: configure arguments = config.arg".

Obtaining the actual list of nodes
The IDE invokes the recipe executive module to obtain the list of nodes used in the recipe and its included and child recipes. The recipe executive provides this function:
   	nodelist, dict = get_nodelist(filename)
"filename" is the name of the recipe file. The result is a tuple of two items. The first is a list of Node objects, which includes all the files mentioned in the recipes as being used as source and/or target and their attributes. Additionally a dictionary is returned that contains the variables defined in the recipe. The value of $SOURCE, $TARGET, etc. can be obtained from this (e.g., dict["SOURCE"] is the value of $SOURCE). The IDE can now present the files mentioned in the predefined variables in a nice way, and the remaining nodes in another way.

For a recipe that was generated by the IDE all the nodes should be in the predefined variables, so that the IDE can change these lists. For manually created recipes the IDE can offer the user the list of files, so that they can be viewed, but removing them from the recipe won't be possible, because the IDE doesn't know how the nodes were defined. Only the parts that look just like they were created with the IDE can be changed from the IDE.

Obtaining the tree of included and child recipes
The IDE should be able to handle a project that is split up in sub-projects. For each sub-project a separate recipe is used. The $SOURCE_XYZ and $TARGET_XYZ variables define the files in sub-project XYZ. Since the ":child filename" commands are easy to parse, the IDE can do this by itself without interfacing with the recipe executive. The IDE is also able to update a recipe to add a new sub-project or delete one.
Included recipes are not used by the IDE. It will not be able to change items in them.
Obtaining the actual list of variants
The IDE offers the user the choice to select one of the variants to build. This simply works by passing "VAR=value" arguments to the recipe executive.
The IDE parses the recipe itself to find the variants, since they are easy to spot. The IDE can also change the variants by offering a dialog to the user and updating the recipe. However, this only works for variants that are easy to understand by the IDE.
Fetch files
The IDE invokes the recipe executive to fetch files that the user wants to view and are missing or the user has ordered to be fetched. This uses the the same Main.execute() function as above. The command with which it is invoked is one of:
   	:fetch filename
Note that the IDE must write any updates to the recipe before invoking Main.execute().
Version control
The IDE assumes that all the files in the recipe are either in the same version control system or not version controlled. When the IDE adds or edits the location of the version control system, it uses an ":attr" command as mentioned above. A command with the same attributes but a different list of files may appear in child recipes. Other ways to specify the version control will not be understood by the IDE.
The IDE invokes the recipe executive to perform version control commands. This uses the the same Main.execute() function as above. The command with which it is invoked is, for example:
   	:commit filename
Building targets
The IDE invokes the recipe executive to build targets. This uses the the same Main.execute() function as above. The command with which it is invoked is, for example:
   	:update progname
A variant and specific recipe can specified with the "argv" argument. The IDE can read the "AAP/log" file to obtain error messages and make a list of locations out if this.

IDE using the Version Control Wrapper and Personal Version Control

Since the recipe executive already contains the code for the most often used version control commands, the IDE will use the recipe executive instead of directly interfacing with the version control wrapper. This is already mentioned above.

How the IDE uses the Personal Version Control is still TODO.

IDE using the Cross Referencer

The IDE offers a browser and search dialog for the Cross Referencer. This is the same interface that the Cross Referencer offers to other applications, see there.

IDE invoking applications

Applications can be invoked in three ways:
  1. run and wait for completion. Useful for having the user enter information or filtering a file.
  2. run asynchronously. Useful for displaying documentation.
  3. start and keep in contact. Useful for an editor that displays locations for error messages
The first two are done in the same way as a recipe, with the ":do" command. See the Main.execute() command.

The third method requires a bidirectional connection between the IDE and the application. This is TODO. Sketch of how it could work:

		editor = invoke_editor(args, myhandler)
			"myhandler" handles callbacks from editor (e.g.,
			build a target)"open", fname, byte)"moveto", fname, byte)
			check if editor can handle an errorlist
This can be used for an editor, a file viewer, a shell window, a debugger control window, etc.

The IDE offers a window with a list of locations. A viewer or editor can be started on a location, or instructed to jump to a location. The list of locations is obtained by parsing build output or cross referencer query. The list of locations can also be passed to an editor that can handle it (e.g., Vim).



  The A-A-P pages:
Zimbu award
     Recipe examples
     Aap manual
     Agide tutorial
     Aap version log
     Agide version log
     ported apps
maillists and chat
plan and tasks
     SourceForge pages
     Aap todo list
     Agide todo list
     use cases
     design decisions
tools overview
     script languages
     build tools
     Install tools
     issue tracking
     version control
     browse tools
     various tools

Zimbu! - browse the Keyword Map of
funded by:


Send comments on this page to Webmaster AT            Hosted by SourceForge Logo            Also known as