Chapter 28. Transformation Tool Composition with XTC

Table of Contents

28.1. Basic Mechanisms of XTC
28.1.1. Registration of Programs and Data
28.1.2. Importing other Repositories
28.1.3. Searching Repositories
28.2. Composing Tools in Stratego
28.2.1. Making an XT component
28.2.2. Invoking XT components
28.3. Summary

In Chapter 3, we explained how the Stratego/XT universe is built from small components, each a separately executable program. Many such components are provided for your reuse by the Stratego/XT distribution. We have already seen some of these in action throughout Part II. In this chapter, we will first explain how to compose existing components together, and then proceed to explain how you can make your own Stratego programs into reusable XT components.

28.1. Basic Mechanisms of XTC

Before we can compose XT components, we must place them in a component repository. This is referred to as registration. The registration associates each component with a name, a path and a version. The name is used later to look up components, and map their names to their actual location in the file system. This is very handy when you program with XT components. Inside our Stratego programs, we only have to specify the name of the program we want executed, and the XTC machinery will automatically figure out which program to execute, set up the piping, deal with temporary files, and even perform the execution and parameter passing for us.

In this section, we will cover the registration and lookup mechanisms in some detail.

28.1.1. Registration of Programs and Data

An XTC repository is a collection of programs and data files. The programs, which we usually refer to as XT components, must be registered explicitly using the xtc tool before they can be used. This is also the case for the data files. A normal program transformation system built with Stratego/XT will contain an XTC repository, where all its components have been registered. Though this registration is done automatically for you by the Stratego build system, it will prove instructive to know what takes place behind the scenes, by the understanding the xtc command (refer to Chapter 29 for an explaination of how to configure the Stratego/XT build system to automatically register your XT components).

Suppose our project, called Ogetarts, has been installed into .../ogetarts (where ... is a path of your choice). A typical directory layout for this project would include the following directories:


The part we are interested in now is the directory .../ogetarts/share/ogetarts/. This is where the registry for the XTC repository is normally placed, in the file XTC. Assuming we already have the XT component foo2bar placed in .../ogetarts/libexec/. The following xtc invocation will register it with the Ogetarts registry:

$ xtc -r .../ogetarts/share/ogetarts/XTC register -t foo2bar -l .../ogetarts/libexec -V 1.0

This command will also take care of creating a fresh XTC registry file if it does not already exist. The -l specifies the path to the component being registered, and -t specifies its name. This name is used for lookup purposes later. The -V is used to associate a version number with a given program. It is possible to include the version number in the lookup of a component, to find a particular version of a particular component, but this is not common practice, and should be avoided. However, every registration must have include a version number.

28.1.2. Importing other Repositories

The XTC system provides a form of inheritance, whereby one repository can refer to another. You may also think of it as a scoping mechanism on the namespaces of repositories. When searches for components fail at the local repository, the imported repositories will be searched in turn.

It is practically always necessary for your project to import the Stratego/XT repository, as most transformation systems built with Stratego/XT reuse many of the XT components. The following command adds the Stratego/XT repository to our project:

$ xtc -r .../ogetarts/share/ogetarts/XTC import /usr/share/StrategoXT/XTC

In case you wondered, the current version of XTC has no mechanism for declaring some components as private, and others public. Once a component c is registered in an XTC repository r, all other repositories importing r can ask for c.

28.1.3. Searching Repositories

When you have registered your components or imported other repositories, you may inspect your repository to see that everything looks good. This is done using the query option to xtc. Using query, you can either look for a particular component, or list all registrations in a given repository.

The following command will search the Ogetarts repository for the sglr component. This component was not registered by us, but is inherited from the Stratego/XT repository.

$ xtc -r .../ogetarts/share/ogetarts/XTC query -t sglr 
sglr (3.8) : /usr/bin/sglr

Alternatively, we can list all registrations, in one go:

$ xtc -r .../ogetarts/share/ogetarts/XTC query -a
foo2bar (1.0) : .../ogetarts/libexec/foo2bar
stratego-lib (0.16M1) : .../stratego-lib

The format of this list is name (version) : path, where name, we remember, is the handle used to look up the given component. The list follows a pre-determined order. First, all registrations in the Ogetarts will be displayed. Here, foo2bar is our only component. After the local components, the contents of each imported repository will be displayed. In our case, we only imported Stratego/XT, and the first component in Stratego/XT is stratego-lib. The other 490 registrations have been omitted, for the sake of clarity.

28.2. Composing Tools in Stratego

Much of the scalability of Stratego/XT comes from its component model, thus it is important to know how to design your own programs to take advantage of this infrastructure. Fortunately, the programming interface of XTC consists of a small set of clearly defined strategies. If you are already familiar with io-wrap, as introduced in Chapter 26, this will be even easier to comprehend.

There are three main usage scenarios for the XTC API. Either you use the API to create a new XT component of your own, or you use it to invoke an XT component from your Stratego program (which need not be an XT component), or both; you are writing an XT component which calls other XT components.

28.2.1. Making an XT component

The simplest way to make an XT component is to wrap your top level strategy with the xtc-io-wrap wrapper. This automatically bestows your program with basic command line option parsing capabilities, and also basic I/O.

The following is an example of a trivial XT component which just passes through the term passed to it.

module xtcid
imports libstratego-lib
  main = xtc-io-wrap(xtcfoo)
  xtcfoo = id

As with the io-wrap strategy explained in Chapter 26, a default set of command line options is provided by xtc-io-wrap. After compiling the xtcid.str, we can run it to inspect the default set of options.

$ ./xtcid --help

   -i f|--input f   Read input from f
   -o f|--output f  Write output to f
   -b               Write binary output
   -S|--silent      Silent execution (same as --verbose 0)
   --verbose i      Verbosity level i (default 1)
                    ( i as a number or as a verbosity descriptor:
                      emergency, alert, critical, error,
                      warning, notice, info, debug, vomit )
   -k i | --keep i  Keep intermediates (default 0)
   --statistics i  Print statistics (default 0 = none)
   -h|-?|--help     Display usage information
   --about          Display information about this program
   --version        Same as --about


This is normally all you have to do in order to have a working XT component. You can add additional options using the Option and ArgOption, as explained in Chapter 26, by wrapping your toplevel strategy with xtc-io-wrap(extra-opts, s) instead of xtc-io-wrap(s). In either case, your program will now automatically read the input term from the file specified with -i, pass this to s, then write the result of s to the file specified by -o. When -i or -o are not specified, stdin and stdout will be used instead, respectively.

$ echo "My(Term)" | ./xtcfoo -o myterm.trm
$ cat myterm.trm

In some situations, it does not make sense for your component to accept an input term, or to generate and output term. That is, your component may be a data generator, or a data sink. In these cases,your component should rather use xtc-input-wrap, in the case of a sink, or xtc-output-wrap, in the case of a generator.

The following programs shows a trivial generator, which produces the term "foo" when invoked.

module xtcfoo

  main = xtc-output-wrap(xtcfoo)
  xtcfoo = !"foo" ; write-to

xtcfoo strategy is our top level strategy. Note how it ends in a call to write-to. The argument s to xtc-output-wrap(s) must result in a file, not a term. Copying the current term to a file, is taken care of by the write-to strategy. write-to will create a fresh temporary file and write the current term to it. The inverse of write-to is read-from which reads a term from a file. This latter strategy is used together with xtc-input-wrap, analogously to the example above.

In both cases (xtc-input-wrap and xtc-output-wrap), you may add additional command line options, by using variants of the wrappers which accept extra options, xtc-input-wrap(extra-options, s) and xtc-output-wrap(extra-options, s), respectively.

Checking Component Dependencies.  If you write a component which depends on other components as part of its operation, you are encouraged to add dependency checking using the xtc-check-dependencies. This is a two step procedure: First, you add the --check option to your components command-line options, by adding check-options(deps) to the extra-opts argument of xtc-io-wrap. Afterwards, you should call xtc-check-dependencies as part of your option-processing.

Going back to our simple xtcid program, we would add the --check with functionality, as follows:

module xtcid
  main = xtc-io-wrap(check-options(!["sglr", "foo2bar"]), xtcid)
  xtcid = xtc-check-dependencies ; id

After recompiling xtcid, the user can now ask your component to do a self-check of its dependencies, by calling it with the --check. You may want to piggy-back on this option, adding other kinds of self-checks, such as data consistency checking.

Graceful Termination.  As you have witnessed, the implementation of the XTC model makes heavy use of temporary files. It is important that these files are cleaned up after exectuion of your program. In all programs we have seen so far, this was taken care of automatically by the wrappers. But what happens if you want to terminate your program in the middle of execution, by calling exit, for example? In that case, the temporary files will not be removed properly. A safe way of exiting XTC programs is provided by the xtc-io-exit strategy. This strategy is a plugin replacement for exit, but takes care of cleaning any and all temporary files created by your program before terminating.

28.2.2. Invoking XT components

Now that we know how to create XT components, and we also know that the Stratego/XT environment provides many reuseable XTC programs, we should take a bit of time to explain how we can invoke these inside our Stratego programs. It is important to realize that your Stratego program need not itself be an XT component before it can call out to other XT components. Both XTC programs and normal Stratego programs call XT components in the same way.

Before we proceed, we need to create a small XTC program that we can call. The following is an XTC version of the inc strategy. When invoked on an integer n, it will return n + 1.

module xtcinc
  main = xtc-io-wrap(read-from ; <add> (<id>, 1) ; write-to)

Let us quickly whip up a client for this component, that invokes xtcinc on the integer 1:

module xtcclient
  main = !1 ; xtc-transform(!"xtcinc")

Let us a compile and run this program.

$ ./xtcclient
[ identity crisis | error ] No XTC registration for xtcinc found
./xtcclient: rewriting failed

Harking back a few sections, we should now realize that we forgot to register xtcinc in a repository. We already know how to do this. For the sake of this demonstration, let us create the XTC registry in the same directory as the source code:

$ xtc -r ./XTC r -t xtcinc -V 1.0 -l `pwd`

If we were to run our program xtcclient again at this point, we would still get the same error. This is because we have not told xtcclient which repository it should use. This is done by the --xtc-repo option to the Stratego compiler.

$ strc --xtc-repo ./XTC -i xtcclient.str -la stratego-lib -la stratego-xtc

This should complete the necessary steps: We have created the XTC program xtcinc, we have registered it in the XTC registry in ./XTC, and we have told strc to compile our client program, xtcclient against this repository. We should be all set to run our composition. Let's try.

$ ./xtcclient

At last, it works. We have connected two components. By changing the internals of our components, we can change them to process any terms we want, and we can of course also add additional components into the mix, all using the same basic steps we have explained here. Having to remember the --xtc-repo option to strc all the time is a bit annoying, but as we shall in Chapter 29, this will be taken care of automatically by the Stratego/XT build system.

Finding XTC Data Files.  We have just seen how to use XTC programs from a repository, but so far, we have said nothing about any data files we have registered. This is where the xtc-find-file strategy comes in handy. Let us go through the process of creating a term, registering it, and then using it inside our program.

Suppose the file myterm.trm contains the following term, written in plain text:


Calling xtc as follows, will register it in the local repository we have already created.

$ xtc -r ./XTC r -t myterm.trm -V 1.0 -l `pwd`

This will register the file myterm.trm under the name myterm.trm, which we can use to look it up from inside our programs later. Let us make a new program that does just this.

module xtcload
  main = <xtc-find-file> "myterm.trm" ; read-from

xtc-find is applied to a string, which must be the name of an already registered data file in the repository. As before, we have to compile our program using strc, and remember to include the --xtc-repo option.

$ strc -i xtcload.str --xtc-repo ./XTC

We can now run our freshly compiled program.

$ ./xtcload

The result is as we anticipated. Congratulations! You have now mastered the basics of the XTC mechanics.

28.3. Summary

This chapter introduced you to the mechanics of the XTC model. We saw how to maintain a registry of XT components using the xtc tool, and also how to write XTC compositions in Stratego. Additional detail about the XTC API can be found in the API reference documentation. The complete documentation for the xtc tool is given on the manual page (xtc).