Part IV. The Stratego Library

Stratego does not have the normal mathematical syntax for arithmetic operators, such as +, -, / and *. These operators are used for other purposes. Instead, the library provides the operators as the strategies, namely add, subt, div and mul. Further, there is convenience strategy for integer increment, inc and decrement, dec.

While the Stratego language operates exclusively on terms, there are different kinds of primitive terms. The runtime maintains a distinction between real numbers and integer numbers. The library mirrors this distinction by providing a family of strategies for arithmetic operations. Arithmetic strategies which work on real numbers end in an r, e.g. addr, and strategies working on integers end in an i, e.g. subti. For each arithmetic operator, there is also a type-promoting variant, e.g. mul, which will type-promote from integer to real, when necessary. Finally, there are convenience strategies for working on strings containing numbers. For each arithmetic operation, there is a string variant, e.g divS.

The full set of arithmetic operations in Stratego:

add,  addr,  addi,  addS
div,  divr,  divi,  divS
mul,  mulr,  muli,  mulS
subt, subtr, subti, subtS

Using these strategies is straightforward.

stratego> <addr> (1.5, 1.5)
3.000000000000000e+00
stratego> <subti> (5, 2)
3
stratego> <mul> (1.5, 2)
3.000000000000000e+00
stratego> <inc> 2
3

As we can see, the mul operator can be applied to a pair which consists of different terms (real and integer). In this case, type promotion from integer to real happens automatically.

Working on Strings.  The string variants, e.g. addS and divS work on strings containing integers. The result in strings containing integers.

     
stratego> <addS> ("40", "2")
"42"
stratego> <divS> ("9", "3")
"3"

The strategies found in the library for comparing two numbers correspond to the usual mathematical operators for less-than (lt), less-than-equal (leq), equal (eq), greater-than (gt), greather-than-equal (geq). As with the arithmetic strategies, each of these operators comes in an integer variant, suffixed with i, a real variant (suffixed by r), a string variant (suffixed by S) and a type promoting variant without suffix. The full matrix of comparison functions thus looks like:

lt,  ltr,  lti,  ltS
gt,  gtr,  gti,  gtS
leq, leqr, leqi, leqS
geq, geqr, geqi, geqS

A few examples:

stratego> <lt> (1.0, 2)
(1.000000000000000e+00,2)
stratego> <ltS> ("1", "2")
("1", "2")
stratego> <geqS> ("2", "2")
("2","2")
stratego> <gtr> (0.9, 1.0)
command failed

The maximum and minimum of a two-element tuple of numbers can be found with the max and min strategies, respectively. These do not distinguish between real and integers. However, they do distinguish between numbers and strings; maxS and minS are applicable to strings.

stratego> <max> (0.9, 1.0)
1.0
stratego> <min> (99, 22)
22
stratego> <minS> ("99", "22")
"22"

Some other properties of numbers, such as whether a number is even, negative or positive, can be be tested with the strategies even, neg and pos, respectively.

The modulus (remainder) of dividing an integer by another is provided by the mod strategy. gcd gives the greatest common divisor of two numbers. Both mod and gcd work on a two-element tuple of integers. The log2 strategy can be used to find the binary logarithm of a number. It will only succeed if the provided number is an integer and that number has an integer binary logarithm.

stratego> <mod> (412,123)
43
stratego> <gcd> (412,123)
1
stratego> <log2> 16
4

The library provides a strategy for generating random numbers, called next-random. The algorithm powering this random generator requires an initial "seed" to be provided. This seed is just a first random number. You can pick any integer you want, but it's advisable to pick a different seed on each program execution. A popular choice (though not actually random) is the number of seconds since epoch, provided by time. The seed is initialized by the set-random-seed strategy. The following code shows the normal idiom for getting a random number in Stratego:

stratego> time ; set-random-seed
[]
stratego> next-random     
1543988747

The random number generator needs only be initialized with a seed once for every program invocation.

In this chapter, we saw that Stratego is different from many other languages in that it does not provide the normal arithmetic operators. We saw that instead, strategies such as add and mul are used to add and multiply numbers. We also saw which strategies to use for comparing numbers and generating random numbers.

The module term/integer contains strategies for working with numbers. Refer to the library reference documentation for more information.

The most fundamental operations on lists is the ability compose and decompose lists. In Stratego, list composition on "sugared" lists, that is, lists writen in the sugared form, has some sugar of its own. Assume xs is the list [1,2,3]. The code [0|xs] will prepend a 0 to it, yielding [0,1,2,3]. List decomposition is done using the match operator. The code ![0,1,2,3] ; ?[y|ys] will bind y to the head of the list, 0, and ys to the tail of the list, [1,2,3].

The module collection/list contains a lot of convenience functions for dealing with lists. (collection/list is contained in the liblib library.) For example, the strategy elem will check if a given value is in a list. If it is, the identity of the list will be returned.

stratego> import liblib
stratego> <elem> (1, [2,3,1,4])
[2,3,1,4]

Continuing on the above Stratego Shell session, we can exercise some of the other strategies:

stratego> <length> [1,2,3,4,5]
5
stratego> <last> [5,6,7,8,9]
9
stratego> <reverse> [1,2,3,4,5]
[5,4,3,2,1]

There are two strategies for concatenating lists. If the lists are given as a tuple, use conc. If you have a list of lists, use concat:

stratego> <conc> ([1,2,3],[4,5,6],[7,8,9])
[1,2,3,4,5,6,7,8,9]
stratego> <concat> [[1,2,3],[4,5,6],[7,8,9]]
[1,2,3,4,5,6,7,8,9]

The sublist of the first n elements can be picked out with the take(|n) strategy:

stratego> <take(|3)> [1,2,3,4,5]
[1,2,3]

Finally, the fetch(s) strategy can be used to find the first element for which s succeeds:

stratego> <fetch(?2)> [1,2,3]
2

The Stratego library contains many other convenient functions, which are documented in the API documentation.

The list sorting function is called qsort(s), and implements the Quicksort algorithm. The strategy parameter s is the comparator function.

  
stratego> <qsort(gt)> [2,3,5,1,9,7]
[9,7,5,3,2,1]

Stratego also has library support for associative lists, sometimes known as assoc lists. There are essentially lists of (key, value) pairs, and work like a poor man's hash table. The primary strategy for working with these lists is lookup. This strategy looks up the first value associated with a particular key, and returns it.

stratego> <lookup> (2, [(1, "a"), (2, "b"), (3, "c")]) => "b"

The library also contains some useful strategies for combining multiple lists. The cart(s) strategy makes a cartesian product of two lists. For each pair, the parameter strategy s will be called. In the second case below, each pair will be summed by add.

stratego> <cart(id)> ([1,2,3],[4,5,6])
[(1,4),(1,5),(1,6),(2,4),(2,5),(2,6),(3,4),(3,5),(3,6)]
stratego> <cart(add)> ([1,2,3],[4,5,6])
[5,6,7,6,7,8,7,8,9]

Two lists can be paired using zip(s). It takes a tuple of two lists, and will successively pick out the head of the lists and pair them into a tuple, and apply s to the tuple. zip is equivalent to zip(id).

stratego> <zip> ([1,2,3],[4,5,6])
[(1,4),(2,5),(3,6)]
stratego> <zip(add)> ([1,2,3],[4,5,6])
[5,6,7]

The inverse function of zip is unzip.

stratego> <unzip> [(1,4),(2,5),(3,6)]
([1,2,3],[4,5,6])

There is also unzip(s) which like unzip takes a list of two-element tuples , and applies s to each tuple before unzipping them into two lists.

In Stratego, lightweight sets are implemented as lists. A set differs from a list in that a given element (value) can only occur once. The strategy nub (also known as make-set) can be use to make a list into a set. It will remove duplicate elements. The normal functions on sets are provided, among them union, intersection, difference and equality:

stratego> <nub> [1,1,2,2,3,4,5,6,6]
[1,2,3,4,5,6]
stratego> <union> ([1,2,3],[3,4,5])
[1,2,3,4,5]
stratego> <diff> ([1,2,3],[3,4,5])
[1,2]
stratego> <isect> ([1,2,3],[3,4,5])
[3]
stratego> <set-eq> ([1,2,3],[1,2,3])
([1,2,3],[1,2,3])

Elementwise transformation of a list is normally done with the map(s) strategy. It must be applied to a list. When used, it will apply the strategy s to each element in the list, as shown here. It will return a list of equal length to the input. If the application of s fails on one of the elements map(s) fails.

stratego> <map(inc)> [1,2,3,4]
[2,3,4,5]

mapconcat(s) is another variant of the elementwise strategy application, equivalent to map(s) ; concat. It takes a strategy s which will be applied to each element. The strategy s must always result in a list, thus giving a list of lists, which will be concatenated. A slightly more convoluted version of the above mapping.

If we want to remove elements from the list, we can use filter(s). The filter strategy will apply s to each element of a list, and keep whichever elements it succeeds on:

stratego> <filter(?2 ; !6)> [1,2,3,2]
[6,6]

stratego> <mapconcat(\ x -> [ <inc> x ] \)> [1,2,3,4]

List folding is a somewhat flexible technique primarily intended for reducing a list of elements to a single value. Think of it as applying an operator between any two elements in the list, e.g. going from [1,2,3,4] to the result of 1 + 2 + 3 + 4. If the operator is not commutative, that is x <op> y is not the same as y <op> x, folding from the left will not be the same as folding from the right, hence the need for both foldl and foldr.

The foldr(init, oper) strategy takes a list of elements and starts folding them from the right. It starts after the rightmost element of the list. This means that if we use the + operator with foldr on the list [1,2,3,4], we get the expression 1 + 2 + 3 + 4 + , which obviously has a dangling +. The strategy argument init is used to supply the missing argument on the right hand side of the last plus. If the init supplied is id, [] will be supplied by default. We can see this from the this trial run:

stratego> <foldr>(id, debug)
(4,[])
(3,(4,[]))
(2,(3,(4,[])))
(1,(2,(3,(4,[]))))
(1,(2,(3,(4,[]))))

With this in mind, it should be obvious how we can sum a list of numbers using foldr:

stratego> <foldr(!0, add)> [1,2,3,4]
10

The related strategy foldl(s) works similarly to foldr. It takes a two-element tuple with a list and a single element, i.e. ([x | xs], elem). The folding will start in the left end of the list. The first application is s on (elem, x), as we can see from the following trial run:

stratego> <foldl(debug)> ([1,2,3,4], 0)
(1,0)
(2,(1,0))
(3,(2,(1,0)))
(4,(3,(2,(1,0))))
(4,(3,(2,(1,0))))

Again, summing the elements of the list is be pretty easy:

stratego> <foldl(add)> ([1,2,3,4], 0)
10

In this chapter we got a glimpse of the most important strategies for manipulating lists. We saw how to construct and deconstruct lists, using build and match. We also saw how we can sort, merge, split and otherwise transform lists. The strategies for associative lists and sets gave an impression of how we can construct new abstractions from basic lists.

More information about list strategies available can be found in the collections/list module, in the library reference documentation.

Strings, like all other primitive data types in Stratego, are terms. They are built with the build (!) operator and matched with the match (?) operator. Additional operations on and with strings are realized through strategies provided by the Stratego library. The most basic operations on strings provided by the library are concatenation, length computation and splitting. We will discuss operation each in turn.

The library provides two variants of the string concatenation operation. The first, concat-strings, takes a list of strings and returns the concatenated result. The second, conc-strings takes a two-element tuple of strings and returns the concatenated result:

stratego> <concat-strings> ["foo", "bar", "baz"]
"foobarbaz"
stratego> <conc-strings ("foo", "bar")
"foobar"

Once you have a string, you may want to know its length, i.e. the number of characters it contains. There are two equivalent strategies for determining the length of a string. If you come from a C background, you may favor the strlen strategy. If not, the string-length strategy may offer a clearer name.

The final basic operation on strings is splitting. There is a small family of closely related strategies for this, which all do simple string tokenization. The simplest of them is string-tokenize(|sepchars). It takes a list of characters as its term argument, and must of course be applied to a string.

stratego> <string-tokenize(|[' '])> "foo bar baz"
["foo","bar","baz"]

Another strategy in the tokenizer family is string-tokenize-keep-all(|sepchars). It works exactly like string-tokenize(|sepchars), except that it also keeps the separators that were matched:

     
stratego> <string-tokenize-keep-all(|[' '])> "foo bar baz"
["foo"," ","bar"," ","baz"]

Even if you don't maintain a phone directory, sorting lists of strings may come in handy in many other enterprises. The strategies string-sort and string-sort-desc sort a list of strings in ascending and descending order, respectively.

stratego> !["worf", "picard", "data", "riker"]
["worf", "picard", "data", "riker"]
stratego> string-sort
["data","picard","riker","worf"]
stratego> string-sort-desc
["worf","riker","picard","data"]

If you only have two strings to sort, it may be more intuitive to use the string comparison strategies instead. Both string-gt and string-lt take a two-element tuple of strings, and return 1 if the first string is lexicographically bigger (resp. smaller) than the second, otherwise they fail.

stratego> <string-gt> ("picard","data")
1
stratego> <string-lt> ("worf","data")
command failed

Not directly a sorting operation, string-starts-with(|pre) is a strategy used to compare prefixes of strings. It takes a string as the term argument pre and must be applied to a string. It will succeed if pre is a prefix of the string it was applied to:

stratego> <strings-starts-with(|"wes")> "wesley"
"wesley"

We already said that strings are terms. As with terms, we can also deconstruct strings, but we cannot use normal term deconstruction for this. Taking apart a string with explode-string will decompose a string into a list of characters. We can then manipulate this character list using normal list operations and term matching on the elements. Once finished, we can construct a new string by calling implode-string. Consider the following code, which reverses a string:

stratego> !"evil olive"
"evil olive"
stratego> explode-string
[101,118,105,108,32,111,108,105,118,101]
stratego> reverse
[101,118,105,108,111,32,108,105,118,101]
stratego> implode-string
"evilo live"

This explode-string, strategy, implode-string idiom is useful enough to warrant its own library strategy, namely string-as-chars(s). The code above may be written more succinctly:

stratego> <string-as-chars(reverse)> "evil olive"
"evilo live"

Sometimes, in the heat of battle, it is difficult to keep track of your primitive types. This is where is-string and is-char come in handy. As you might imagine, they will succeed when applied to a string and a character, respectively. A minor note about characters is in order. The Stratego runtime does not separate between characters and integers. The is-char must therefore be applied to an integer, and will verify that the value is within the printable range for ASCII characters, that is between 32 and 126, inclusive.

Finally, it may be useful to turn arbitrary terms into strings, and vice versa. This is done by write-to-string and read-from-string, which may be considered string I/O strategies.

stratego> <write-to-string> Foo(Bar())
"Foo(Bar)"
stratego> read-from-string
Foo(Bar)

Another interplay between primitive types in Stratego is between numbers and strings. Converting numbers to strings and strings to numbers is frequently useful when dealing with program transformation, perhaps particularly partial evaluation and interpretation. Going from numbers to strings is done by int-to-string and real-to-string. Both will accept reals and integers, but will treat is input as indicated by the name.

stratego> <int-to-string> 42.9
"42"
stratego> <real-to-string> 42.9
"42.899999999999999"

The resulting number will be pretty-printed as best as possible. In the second example above, the imprecision of floating point numbers results in a somewhat non-intuitive result.

Going the other way, from strings to numbers, is a bit more convoluted, due to the multiple bases available in the string notation. The simplest strategies, string-to-real and string-to-int, assume the input string is in decimal.

stratego> <string-to-real> "123.123"
1.231230000000000e+02
stratego> <string-to-int> "123"
123

For integers, the strategies hex-string-to-int, dec-string-to-int, oct-string-to-int and bin-string-to-int can be used to parse strings with numbers in the most popular bases.

The rewriting paradigm of Stratego is functional in nature, which is somewhat contradictory to the imperative nature of hashtables. Normally, this doesn't present any practical problems, but remember that changes to hashtables "stick", i.e. they are changed by side-effect.

The Stratego hashtable API is pretty straightforward. Hashtables are created by new-hastable and destroyed by hashtable-destroy.

stratego> import lib	
stratego> new-hashtable => h
Hashtable(136604296)

The result Hashtable(136604296) here is a handle to the actual hashtable. Consider it a pointer, if you will. The content of the hashtable must be retrieved with the hashtable-* strategies, which we introduce here. The strategy hashtable-copy can be used to copy a hashtable.

Adding a key with value to the table is done with hashtable-put(|k,v), where k is the key, v is the value. Retrieving the value again can be done by hashtable-get(|k).

stratego> <hashtable-put(|"one", 1)> h
Hashtable(136604296)
stratego> <hashtable-get(|"one")
1

The contents of the hashtable can be inspected with hashtable-values and hashtable-keys.

Nesting is also supported by the Stratego hashtables. Using hashtable-push(|k,v), a new "layer" can be added to an existing key (or an initial layer can be added to a non-existing key). Removing a layer for a key can be done with hashtable-pop(|k).

stratego> <hashtable-push("one",2)> h
Hashtable(136604296)
stratego> <hashtable-get("one")> h
[2,1]
stratego> <hashtable-pop(|"one")> h
Hashtable(136604296)
stratego> <hashtable-get(|"one")> h
[1]
stratego> <hashtable-remove(|"one")> h
Hashtable(136604296)
stratego> <hashtable-values> h
[]

The library provides a rather feature complete implementation of indexed sets, based on hashtables. A lightweight implementation of sets, based on lists, is explained in Chapter 22.

Similar to hashtables, indexed sets are created with the new-iset strategy, copied with iset-copy and destroyed with iset-destroy.

stratego> new-iset => i
IndexedSet(136662256)

The resulting term, IndexedSet(136662256), is a handle to the actual indexed set, which can only be manipulated through the iset-* strategies.

Adding a single element to a set is done with iset-add(|e), whereas an entire list can be added with the iset-addlist(|es). Its elements can be returned as a list using iset-elements.

stratego> <iset-addlist(|[1,2,3,4,4])> i
IndexedSet(136662256)
stratego> iset-elements
[1,2,3,4]

Notice that the result is indeed a set: every value is only represented once.

Using iset-get-index(|e), the index of a given element e can be found. Similarly, iset-get-elem(|i) is used to get the value for a particular index.

stratego> <iset-get-index(|3)> i
2
stratego> <iset-get-elem(|3)> i
4

Note that the indexes start at 0.

The set intersection between two sets can be computed with the iset-isect(|set2) strategy. The strategy iset-union(|set2) calculates the union of two sets, whereas iset-subset(|set2) checks if one set is a subset of another. Equality between two sets is checked by iset-eq(|set2). These strategies are all used in a similar way:

stratego> <iset-eq(|set2)> set1

A single element can be removed from the set with iset-remove(|e). iset-clear will remove all elements in a set, thus emptying it.

The basic strategies for console I/O print and printnl are used to write terms to stdout or stderr (or any other opened file). They both take a tuple. The first element of the tuple is the file to write to, the second is a list of terms. Each term in the list be converted to a string, and and these strings will be concatenated together to form the resulting output. The printnl will also append a newline to the end of the resulting string.

The following module should be compiled with strc, as usual.

module example
imports liblib
strategies
  main = 
    <print> (stdout, ["baz"])
    ; <printnl> (stdout, [ "foo", 0, "bar" ])

After compiling this file, running it will give the following result:

$ ./example
bazfoo0bar
$

Notice how the string baz will be written without a newline (or other space). Also, notice how the terms in the list argument were concatenated.

When using these strategies in the Stratego Shell, some care must be taken when using the std* files, as the following example shows.

stratego> <printnl> (stdout(), [ "foo", 0, "bar" ])
foo0bar

The shell requires that you put an extra parenthesis after the stdout.

The debug and error are convenience wrappers around printnl. They will always write their result to stderr. The error strategy is defined as:

error =
  where(<printnl> (stderr, <id>))

It is used similarly to the printnl strategy:

stratego> <error> ["foo", 0, "bar"]
foo0bar

The debug strategy accepts any term, i.e. not only lists of terms. The term will be written verbatim:

stratego> <debug> [ "foo", 0, "bar" ]
["foo",0,"bar"]

The library provides a small set of simple file and directory manipulation operations. Assume the directory /tmp only contains the files foo, bar, baz. Elementary directory operations can be done as illustrated below:

stratego> <readdir> "/tmp"
["foo","bar","baz"]
stratego> <rename-file> ("/tmp/foo", "/tmp/bax")
"/tmp/bax"
stratego> <remove-file> "/tmp/baz"
[]
stratego> <link-file> ("/tmp/bar", "/tmp/foo")
"/tmp/foo"
stratego> <link-file> ("/tmp/bar", "/tmp/foo")
"/tmp/foo"
stratego> <new-temp-dir> "/tmp"
"/tmp/StrategoXTnsGplS"

The library contains a family of strategies which must be applied to a File, and will return information about it. these include isdir, isatty, isfifo and islnk which are predicates checking if a file is a directory, TTY, FIFO or a symbolic link, respectively. To obtain a File object in the first place, we should call file-exists followed by filemode. Thus, checking if /etc is a directory is done as follows:

stratego> <file-exists ; filemode ; isdir> "/etc"

The library also has another family of strategies for getting information about files. These must be applied to a string containing the filename. The family includes is-executable, is-readable and is-writeable.

stratego> <is-executable> "/bin/bash"
"/bin/bash"

Finally, the directory strategies also include the usual suspects for dealing with paths.

stratego> <is-abspath> "../foo"
command failed
stratego> <dirname> "/foo/bar/baz"
"/foo/bar"
stratego> <base-filename> "/foo/bar/baz"
"baz"
stratego> <get-extension "/tmp/foo.trm"
"trm"
stratego> <abspath> "../foo"
/home/karltk/source/oss/stratego/strategoxt-manual/trunk/../foo

There are also a few strategies for finding files. We shall describe find-file(s). The other variants of find-file are described in the library documentation. The strategy find-file(s) finds one file with a specific file extension in a list of directories. It takes a two-element tuple. The first element is a file name as a string, then second element is a list of paths, i.e. (f, [d*]). The extension of f will be replaced by what is produced by s, and the directories given in [d*]. Consider the code below.

stratego> <find-file(!"rtree")> ("file.str", ["."])

This snippet will consider the filename file.str, replace its extension with rtree and look through the directories in the list ["."]. Effectively, it will search for file.rtree in the current directory.

Opening a file is done with the fopen strategy. It takes a two-element tuple, the first element is the filename as a string, the second is the open mode, which is also a string. The most important modes are read (r); write ("w") which opens and empty file for writing, truncating any existing file with the same name; and append (a) which appends to the file if it already exists. After all file operations stream have been finished, it should be closed with fclose, which will flush and close the file. Explicit flushing can also be done with fflush.

It should be pointed out that reading and writing text files with Stratego is rather rare. Normally, text files are read with a parser generated from an SDF description and written using a pretty-printer defined in the Box formalism. In rare cases, this may turn out be too heavy handed, especially if the file format is simplistic and line-based. In this instance, we can come up with an easier solution using read-text-file and read-text-line.

Assume the file /tmp/foo contains the following lines:

one
two
three

We can read this file in one big chunk into a string with the read-text-file strategy, which must be applied to a filename:

stratego> <read-text-file> "/tmp/foo"
"one\ntwo\nthree\n"

Alternatively, for example if the file is large, we can read it line by line. In this scenario, we must open the file and get a handle to a stream.

      
stratego> <fopen> ("foo.txt", "r") => inp
Stream(136788400)
stratego> <read-text-line> inp
"one"

The primary form of file I/O you will be using in Stratego is reading and writing terms. As explained earlier, the terms are stored on disk as either binary, compressed text or plain text ATerms. Reading a term, no matter which storage format, is done with the ReadFromFile strategy. It is applied to a filename.

stratego> <ReadFromFile> "/tmp/foo.trm"
Foo(Bar)

To write a term to file, you can use WriteToTextFile or WriteToBinaryFile. The binary format is approximately eight times more space-efficient on average. Both strategies take a two-element tuple where the first element is the filename and second is the term to write. Writing the current term requires a minor twist, which is shown here:

stratego> <WriteToBinaryFile> ("/tmp/bar.trm", <id>)
Foo(Bar)

It is also possible to read and write terms from and to strings, using read-from-string and write-to-string. Chapter 23 contains explanation of how these strategies work.

The strategies for logging are used pervasively throughout the Stratego toolchain. They are easy to use in your own applications, too. The logging system is built on top of the log(|severity, msg) and log(|severity, msg, term) strategies. It is possible to use these directory, as the following example demonstrates.

stratego> import util/log
stratego> log(|Error(), "my error")

However, it is preferrable to use the high-level wrapper strategies fatal-err-msg(|msg), err-msg(|msg), warn-msg(|msg) and notice-msg(|msg). Except for fatal-err-msg, these strategies will return with the current term untouched, and write the message as a side effect. The fatal-err-msg strategy will also terminate the program with error code 1, after writing the message.

When a compiled Stratego program is first started, the initial value of the current term is a list containing the command line arguments to the program. Fhe following program, foo, is a genuine "do nothing" program.

module foo
imports liblib
strategies
  main = id

In the course of this chapter, we will extend this program with new options and even a help screen for the user to enjoy. But first, let us compile and run foo with some arguments, to get an idea of where we stand.

$ strc -i foo.str
...
$ ./foo --help
["./foo","--help"]
$ ./foo -i foo.str --extra --option -s
["./foo","-i","foo.str","--extra","--option","-s"]

From this interaction, we see that a list of the arguments provided on the command line becomes the initial term of the program. Each command line argument becomes its own string element in the list, and the first element of the list is the command used to invoke the foo itself. Clearly, this list must be interpreted somehow, for the arguments to have any meaning. The library contains a collection of robust strategies that deal with just this. The option handling strategies will parse the argument list, let you set default values for options and transparently deal with long and short forms of the same option. Even more interesting, the library provides so-called wrap strategies that abstract away all of the dreary details of this option handling, and also provide a default set of options with some basic functionality.

Perhaps the most frequently used wrap strategy is io-wrap (or its XTC equivalent, xtc-io-wrap, which is not covered here). In fact, io-wrap is a family of strategy, which includes io-wrap(s), io-wrap(opts, s) and io-wrap(opts, usage, about, s). All of these variants provide the same basic functionality. The increasing number parameters allows you to override default bevhavior, when you need to. When using one of these strategies, a standard package of command line handling will be provided to your users through your program. Let us start with the simplest case, io-wrap(s) and work our way from there. Consider a revised edition of foo, from above:

module foo
imports liblib
strategies
  main = io-wrap(id)

Here, we wrap the core logic of our program (just id in our case) inside the io-wraper. If we run foo with the --help this time around, we will get a more instructive reply than previously:

$ ./foo --help
Options:
   -i f|--input f   Read input from f
   -o f|--output f  Write output to f
   -b               Write binary output
   -S|--silent      Silent execution (same as --verbose 0)
   --verbose i      Verbosity level i (default 1)
                    ( i as a number or as a verbosity descriptor:
                      emergency, alert, critical, error,
                      warning, notice, info, debug, vomit )
   -k i | --keep i  Keep intermediates (default 0)
   --statistics i  Print statistics (default 0 = none)
   -h|-?|--help     Display usage information
   --about          Display information about this program
   --version        Same as --about

Description:

All of a sudden, our program is a well-behaved citizen in the command line world. It answers to --help, and appears to have a few other tricks up its sleeve to. For example, the input term to foo can now be specified using the -i option, and output can be stored to a file using the -o option.

So does this actually work? All for free? Let's test with putting the following term into the file term.trm:

Yes(It(Works))

Bursting with excitement, we try:

$ ./foo -i term.trm
Yes(It(Works))

And if that's not enough, there is even a bit of extra convenience provided by io-wrap: If -i is not provided, stdin is read. Similarly, stdout is written to if -o is not specified.

So, using io-wrap is all you have to do for your program to gain a minimal, but functional set of command line options. As a bonus, these options also make your program compatible with XTC; it can be composed with other XTC components.

It is often necessary for programs to expose switches to turn functionality on and off, or to read extra configuration knowledge from external files. All these cases require additional command line options, so we need a mechanism for extending the basic io-wrap(s) strategy. The preferred way of adding new options is to use the io-wrap(opts,s) strategy, providing it with a strategy encoding the options.

When adding a new option, we must decide whether this option will require a argument of its own, or not. The term ArgOption is used to construct options that take arguments and Option is the term used for on/off switches. Suppose we want to expose an option --verify that enables the user to run our transformation in a self-verifying mode. This is clearly an on/off switch, and therefore a job for Option.

Adding this option to our program foo gives us the following code:

module foo
imports liblib
signature
  constructors
    Verify : Option

strategies

  main =  io-wrap(
    Option(
        "--verify"
      , <set-config> (Verify(), "on")
      , !"--verify         Turn on verification")
    , do-foo)

  do-foo = ...

Note that we made a new term type, Verify, to serve as our switch symbol. Inside the real logic of program, given by do-foo, we would write <get-config> Verify to get the state of the Verify switch. In the case where the user had specified --verify on the command line, get-config would result in the term "on", taken from the declaration of our verify Option. If the did not add --verify to his command line arguments, <get-config> Verify will fail.

Options with arguments is provided through ArgOption. The use is pretty much identical to that of Option. Assume our transformation needs a bit of help from configurable processing tables (whatever that might be), and that we want these tables configured at runtime, using the -p option. We want to add another alternative to the opts argument of io-wrap:

  main = io-wrap(
     Option(....)
     + 
     ArgOption(
        "-p"
       , where(<extend-config> ("-p", [<id>]))
       , !"-p file   Use processing table in file")
     , id)

Note how we compose options with the choice operator (+). With this addition, our users can now specify their elusive processing tables with the -p. The arity of this option is checked automatically. That is, if the user adds -p to his argument list without specifying a file afterwards, this will be signaled as an error, and the usage screen will be printed. Once the user has read this and corrected the error of his ways, the value (the filename) provided via -p can be obtained using the get-option strategy. If no -p was specified, get-option will fail. Sometimes, this failure may be inappropriate, and a default value is desired instead. If you browse through Stratego code, you may come across the following idiom for dealing with this situation:

  get-config-p =
    <get-config> "-p" <+ ![]

This is all you need to know about basic command-line option processing. When in doubt, you are advised to refer to the detailed API documentation is available in the API reference.

If your program is primarily intended for human use, you are encouraged to complete your program's option configuration with a short description of what your tool does.

We can easily add a short description and also an about section. The description is shown as part of the help screen (displayed with --help), whereas the about section is displayed when the arguments to foo contain --about. It is customary for the about screen to contain copyright notices and credits.

  
  main = io-wrap(
        ...
      , foo-tool-usage
      , foo-tool-about
      , id)

  foo-tool-usage =
    default-system-usage(
      !["Usage: foo -p proctbl [options]"]
    , ![ "This program verifies the input against a processing table.\n"]
    )

  foo-tool-about =
     <echo> "Written by Alan Turing <alan@turing.org>"

After compiling this version of foo, invoking it with --help give the following help screen:

$ ./foo --help
Usage: foo -p proctbl [options]

Options:
   --verify         Turn on verification
   -p file          Use processing table in file
   -i f|--input f   Read input from f
   -o f|--output f  Write output to f
   -b               Write binary output
   -S|--silent      Silent execution (same as --verbose 0)
   --verbose i      Verbosity level i (default 1)
                    ( i as a number or as a verbosity descriptor:
                      emergency, alert, critical, error,
                      warning, notice, info, debug, vomit )
   -k i | --keep i  Keep intermediates (default 0)
   --statistics i  Print statistics (default 0 = none)
   -h|-?|--help     Display usage information
   --about          Display information about this program
   --version        Same as --about

Description:
This program verifies the input against a processing table.

If we invoke our splendid program foo with the --about option, we can now observe:

$ ./foo --about
Written by Alan Turing <alan@turing.org>

Not all programs written in Stratego intended for processing ATerms. For these programs, the full I/O functionality provided by io-wrap may often turn out to be inappropriate. An alternative way of dealing with parameters is also provided by the library, centered around the option-wrap family of strategies. option-wrap works analogously to io-wrap, but does not provide the -i and -o options, nor does it read anything from stdin automatically.

The default set of options provided by option-wrap is shown below:

    
   --verify         Turn on verification
   -h|-?|--help     Display usage information
   --about          Display information about this program
   --version        Same as --about

Adding new options is identical to what we already explained for io-wrap.

Module list-zip-test.str in the Stratego Standard Library is an example test suite. SUnit test suites can be compiled to an executable program. When run the program gives the following output:

test suite: list-zip-test
nzip0-test
zip-test1
zip-test2
successes: 3
failures: 0
(3,0)

An extract from the module shows how a test suite can be set up:

module list-zip-test
imports list-zip sunit
strategies

  main =
    test-suite(!"list-zip-test",
        nzip0-test;
        zip-test
     )

  nzip0-test =
    apply-test(!"nzip0-test"
        ,nzip0(id)
        ,!["a","b","c"]
        ,![(0, "a"), (1, "b"), (2, "c")]
        )

The test-suite strategy takes a string and a strategy. The strategy is typically a sequential composition of calls to unit tests. Unit tests such as nzip0-test are defined using one of the unit-test strategies from module sunit, which is in the standard library.

The prototypical unit test is composed with the apply-test strategy, which takes a name, the strategy that is being tested, the input term, and the exptected output term. Note that the name, input, and output term must be specified in strategies, which means that if literal terms are used then you must build them using the ! operator.

apply-test(!"test3"
, ltS
, !("3","5")
, !("3","5")
)

It is also useful to test that a strategy fails if applied to a certain input. The ordinary apply-test cannot be used for this because it requires an output term to which the result will be compared. For testing failure the apply-and-fail strategy is available in sunit. It ensures that the strategy fails if applied to a certain input.

apply-and-fail(!"is-double-quote 3"
, is-double-quoted
, !"\"fnord"
)

Sometimes a test just want to check some condition on the output of a strategy application, without actually specifying the output itself. The apply-test strategy cannot be used for this because it performs an equivalence test of the required output and the actual output. In this case apply-and-check can be used. This strategy takes 4 arguments: a name, a strategy (s) to test, an input and a strategy that will be applied to the result of applying s to the input. This strategy must succeed if the output is correct, or fail if the output is incorrect.

new-test =
  apply-and-check(!"new test",
    (new, new)
  , !(1, 1)
  , not(eq); (is-string, is-string)
  )

XTC is the preferred system for writing components in Stratego/XT. Testing that an XTC component acts as intended is not much more difficult than for individual strategies. As the semantics of the interfaces to XTC programs is open-ended, there is currently no generic testing framework for handling all the details.

Nevertheless, adapting the unit testing framework explained above for testing your components is easy. Consider the strategy xtc-apply-test, given below.

xtc-apply-test(|nm, prog, args, inp, outp) =
  xtc-temp-files(
    apply-test(!nm,
        write-to
      ; xtc-transform(!prog, !args)
      ; read-from
    , !inp
    , !outp
    )
  )

This is a straightforward extension of apply-test. It will invoke the XTC program given in the prog term argument with the list of command line arguments in args. The input to the program is the term given by inp. The resulting term after prog has been run is checked against outp. If they match, the test succeeds.

The simple code above does not account for additional file arguments to prog. This may be useful if prog if prog is, say, a pretty-printer which takes both a document and a style sheet as arguments. The code below provides an example of how this may be handled.

xtc-apply-pp-test(|nm, prog, ssheet, args, inp, outp) =
  xtc-temp-files(
    apply-test(!nm,
      where(<print-to> [ssheet] => FILE(f))
      ; write-to
      ; xtc-transform(!prog, <concat> [ args, "--stylesheet", f])
      ; read-from
    , !inp
    , !outp
    )
  )      

Before we can compose XT components, we must place them in a component repository. This is referred to as registration. The registration associates each component with a name, a path and a version. The name is used later to look up components, and map their names to their actual location in the file system. This is very handy when you program with XT components. Inside our Stratego programs, we only have to specify the name of the program we want executed, and the XTC machinery will automatically figure out which program to execute, set up the piping, deal with temporary files, and even perform the execution and parameter passing for us.

In this section, we will cover the registration and lookup mechanisms in some detail.

.../ogetarts/
             share/
                   ogetarts/
             bin/
             libexec/
             lib/

$ xtc -r .../ogetarts/share/ogetarts/XTC register -t foo2bar -l .../ogetarts/libexec -V 1.0

$ xtc -r .../ogetarts/share/ogetarts/XTC import /usr/share/StrategoXT/XTC

$ xtc -r .../ogetarts/share/ogetarts/XTC query -t sglr 
sglr (3.8) : /usr/bin/sglr

$ xtc -r .../ogetarts/share/ogetarts/XTC query -a
foo2bar (1.0) : .../ogetarts/libexec/foo2bar
stratego-lib (0.16M1) : .../stratego-lib
...

Much of the scalability of Stratego/XT comes from its component model, thus it is important to know how to design your own programs to take advantage of this infrastructure. Fortunately, the programming interface of XTC consists of a small set of clearly defined strategies. If you are already familiar with io-wrap, as introduced in Chapter 26, this will be even easier to comprehend.

There are three main usage scenarios for the XTC API. Either you use the API to create a new XT component of your own, or you use it to invoke an XT component from your Stratego program (which need not be an XT component), or both; you are writing an XT component which calls other XT components.

module xtcid
imports liblib
strategies
  main = xtc-io-wrap(xtcfoo)
  xtcfoo = id

$ ./xtcid --help

Options:
   -i f|--input f   Read input from f
   -o f|--output f  Write output to f
   -b               Write binary output
   -S|--silent      Silent execution (same as --verbose 0)
   --verbose i      Verbosity level i (default 1)
                    ( i as a number or as a verbosity descriptor:
                      emergency, alert, critical, error,
                      warning, notice, info, debug, vomit )
   -k i | --keep i  Keep intermediates (default 0)
   --statistics i  Print statistics (default 0 = none)
   -h|-?|--help     Display usage information
   --about          Display information about this program
   --version        Same as --about

Description:

$ echo "My(Term)" | ./xtcfoo -o myterm.trm
$ cat myterm.trm
My(Term)
$

module xtcfoo
imports liblib
strategies
  main = xtc-output-wrap(xtcfoo)
  xtcfoo = !"foo" ; write-to

module xtcid
imports liblib
strategies
  main = xtc-io-wrap(check-options(!["sglr", "foo2bar"]), xtcid)
  xtcid = xtc-check-dependencies ; id

module xtcinc
imports liblib
strategies
  main = xtc-io-wrap(read-from ; <add> (<id>, 1) ; write-to)

module xtcclient
imports liblib
strategies
  main = !1 ; xtc-transform(!"xtcinc")

$ ./xtcclient
[ identity crisis | error ] No XTC registration for xtcinc found
./xtcclient: rewriting failed

$ xtc -r ./XTC r -t xtcinc -V 1.0 -l `pwd`

$ strc --xtc-repo ./XTC -i xtcclient.str
....

$ ./xtcclient
2

My(Term)

$ xtc -r ./XTC r -t myterm.trm -V 1.0 -l `pwd`

module xtcload
imports liblib
strategies
  main = <xtc-find-file> "myterm.trm" ; read-from

$ strc -i xtcload.str --xtc-repo ./XTC

$ ./xtcload
My(Term)

This chapter introduced you to the mechanics of the XTC model. We saw how to maintain a registry of XT components using the xtc tool, and also how to write XTC compositions in Stratego. Additional detail about the XTC API can be found in the API reference documentation. The complete documentation for the xtc tool is given on the manual page (xtc).

Here we describe how to use the Stratego compiler for small projects and for compiling one-off examples on the command-line. We recommend that you use Autotools for larger projects, i.e. when you need to build multiple artifacts (see the next sections).

Invoking the compiler on simple programs which only depend on the Stratego library is straightforward:

$ strc -i myprog.str 

This produces the executable file myprog. When your program depends on other Stratego packages (libraries), you need to provide the compiler with proper include paths (for finding the module definitions) and linking arguments (for linking the libraries with the final executable). For convenience, you should define an alias called strcflags as follows:

$ alias strcflags="pkg-config --variable=strcflags "

By calling strcflags with the the name of a specific package, e.g. java-front, all necessary include paths and library arguments will be provided for you. This gives rise to the following idiom:

$ strc -i myprog.str $(strcflags dryad java-front)

Note that providing several arguments (packages) to strcflags is allowed.

$ strc -i myprog.str -Xlinker -all-static

Setting up a build system for Stratego involves the Autotool programs automake, autoconf and libtool. In addition, Stratego provides a new tool called autoxt. If you are familiar with the Autotools, setting up a project for Stratego should be rather easy. If this is unfamiliar ground to you, don't fear. We will walk through it slowly in the next sections, but a full treatise is beyond the scope of this tutorial.

After creating your project directory, let's call it ogetarts, the first thing you should do is populate it with the basic build system files, namely configure.ac, bootstrap, Makefile.am. Additionally, you may want to add ChangeLog, AUTHORS, NEWS and README, but these are not essential. If you want to support the creation of RPMs, then you need to create a file ogetarts.spec.in.

For a normal Stratego project, with a syntax, some stand-alone tools, and a library, we suggest the project layout given below (directories end in /). We will discuss the all the components of this hierarchy in turn.

ogetarts/
  bootstrap
  configure.ac
  Makefile.am
  syn/
      Makefile.am
  lib/
      Makefile.am
  tools/
        Makefile.am

The build system is kickstarted by the bootstrap script. This script is responsible for generating a configure script. When run by the user of your package, the configure script will autodetect the Stratego compiler and other tools required to build your project. The concept of generation is very central to autotool-based build systems. The configure script is generated from a configure.ac declaration by the autoreconf tool. The Makefiles are generated from Makefile.in files by the configure script, and Makefile.in files are generated from Makefile.am files by automake. Simple, huh? Generally, the idea is that complicated scripts and makefiles can be generated from high-level declarations using tools. Let's start with the bootstrap script.

#! /bin/sh

autoxt || exit 1
autoreconf -ifv || exit 1

The bootstrap script should be an sh (or bash) shell script that takes care of running autoxt and autoreconf, as shown above. Note that we rely on reasonably recent versions of autoconf and automake.

Assume we are in a palindromic mood and want to name our project Ogetarts. The following file will then provide a reasonable starting point for the configure.ac file.

AC_PREREQ([2.58])
AC_INIT([ogetarts],[0.1],[ogetarts-bugs@ogetarts.org])
AM_INIT_AUTOMAKE([1.7.2 -Wall -Wno-portability foreign])

# set the prefix immediately to the default prefix
test "x$prefix" = xNONE && prefix=$ac_default_prefix

XT_SETUP
XT_USE_XT_PACKAGES

AC_PROG_CC
AC_PROG_LIBTOOL

### OUTPUT #############################
AC_CONFIG_FILES([
  Makefile
  syn/Makefile
  lib/Makefile
  tools/Makefile
])
AC_OUTPUT

Most of this is standard boilerplate. The foreign option specified in the arguments of AM_INIT_AUTOMAKE tells automake not the check that the package conforms to GNU package standards, which requires files such as ChangeLog, AUTHORS, COPYING, NEWS and README. For this small example, we do not want create all these files. You can leave out the foreign option if you want to make a complete GNU package.

The important line is XT_USE_XT_PACKAGES which is a macro invocation that will extend to shell script code that looks for the ATerm library, the SDF tools and Stratego/XT, respectively. These macros are provided by the autoxt tool, via a macro file called autoxt.m4. It provides the following macros.

At the end of the configure.ac above, the invocation of the AC_CONFIG_FILES macro lists other important files of the build system, particularly the Makefiles. We must provide these, but remember that these are generated from .in files which in turn come from .am files. Hence, we need to provide some Makefile.am files. The Makefile.am for the root of the project should look like:

include $(top_srcdir)/Makefile.xt

SUBDIRS           = syn lib tools
BOOTCLEAN_SUBDIRS = $(SUBDIRS)
DIST_SUBDIRS      = $(SUBDIRS)
EXTRA_DIST        =

ACLOCAL_AMFLAGS = -I .

Again, most of this is boilerplate. The important point here is that SUBDIRS = syn lib tools will eventually result in rules that tell make to delve into these directories. We will explain below how the Makefile.ams for each of the source directories should look like. For now, you can just create empty Makefile.am files in the sub-directories syn/, lib/, and tools/. This allows you to bootstrap and configure the package:

$ mkdir syn lib tools
$ touch syn/Makefile.am lib/Makefile.am tools/Makefile.am
$ chmod u+x bootstrap
$ ./bootstrap
$ ./configure
$ make

The content of the empty Makefile.am files depends on whether you are building a parser, stand-alone Stratego programs, or a Stratego library. We will discuss each variant separately, but you are of course free to mix several of these in your project, like we do in this project: In lib lives the library parts of Ogetarts, in syn a parser is generated, and in tools we place the command-line programs.

In Chapter 11, we showed how to compile stand-alone Stratego programs using the strc compiler. This process is automated by the build system, provided you supply a suitable Makefile.am. Take the one provided below as a starting point.

include $(top_srcdir)/Makefile.xt
include $(wildcard *.dep)

bin_PROGRAMS = ogetarts
ogetarts_LDADD = $(STRATEGO_LIB_LIBS) $(STRATEGO_RUNTIME_LIBS) $(ATERM_LIBS)

STRINCLUDES    = -I $(top_srcdir)/lib -I $(top_srcdir)/syn
STRCFLAGS      = --main io-$*

EXTRA_DIST = $(wildcard *.str) $(wildcard *.meta)
CLEANFILES = $(wildcard *.c) $(wildcard *.dep)

This file should be placed in tools/. The following list explains the various parts occurring in the file.

As we can see from the list above, the bin_PROGRAMS is a list of the stand-alone programs that should be compiled in this directory. For each program, a corresponding .str must exist, in this case, a ogetarts.str file. For each program the Stratego compiler will be passed the STRINCLUDES and STRCFLAGS variables. The program_LDADD variable is used to add additional native libraries that should be linked as part of the C compilation process. STRINCLUDES and STRCFLAGS were explained above.

autoxt installs Makefile.xt, a collection of Automake rules for compiling Stratego programs and applying other XT tools, such as signature generation. Using this makefile, a makefile reduces to a declaration of programs to be compiled. The makefile automatically takes care of distributing the generated C code. The specification will only be compiled when it is newer than the C code.

In Chapter 6 we introduced you to the Syntax Definition Formalism SDF for specifying grammars, and showed you how to use pack-sdf and sdf2table to construct parse tables out of these definitions. The grammar can also be used to derive Stratego signatures and regular tree grammars. Not surprisingly, the build system is equipped to do this for you. Consider the Makefile.am provided below.

include $(top_srcdir)/Makefile.xt
include $(wildcard *.dep)

DEF_TBL = Ogetarts.def Ogetarts.tbl
RTG_SIG = Ogetarts.rtg Ogetarts.str

sdfdata_DATA = $(DEF_TBL) $(wildcard *.sdf)
pkgdata_DATA = $(RTG_SIG)

EXTRA_DIST   = $(DEF_TBL) $(RTG_SIG) $(wildcard *.sdf)
CLEANFILES   = $(DEF_TBL) $(RTG_SIG)

SDFINCLUDES   =
SDF2RTG_FLAGS = -m $*
PGEN_FLAGS    = -m $*

This file should be placed in syn/. The following list explains the various parts occurring in the file.

Stratego allows you to freely organize the Stratego modules of your project, but we recommend to have a separate library in the directory lib/. Each .str file placed inside this directory becomes a module for your library. For sufficiently large projects, it is recommended that you further organize your modules into subdirectories inside lib/. Each such subdirectory becomes a package.

Importing modules and packages in Stratego is done using the imports statement. Using imports, you specify which module from which package to import. See Chapter 11 for an introduction to modules. There is a direct mapping between the directory name and the package name, and also between file and module names. Consider the following main module lib/ogetarts.str of your library, which imports all the Stratego modules that constitute the library:

module ogetarts
imports
  ogetarts/main
  ogetarts/front/parse

The import declaration ogetarts/main states that we want to import the module main from the package ogetarts. This tells the Stratego compiler to look for the file main.str inside the ogetarts/ directory. The line ogetarts/front/parse will import the file parse.str in the ogetarts/extensions package. On disk, we will have the following layout:

lib/
    ogetarts.str
    ogetarts/
             main.str
             front/
                   parse.str

In this example, we will assume that the module ogetarts/main provides a simple evaluator for arithmetic expressions:

module ogetarts/main
imports libstratego-lib Ogetarts
strategies

  ogetarts-eval = bottomup(try(Eval))

rules

  Eval : Plus(Int(i), Int(j)) -> Int(<addS> (i, j))
  Eval : Mul(Int(i), Int(j))  -> Int(<mulS> (i, j))

The module ogetarts/front/parse provides a strategy for parsing ogetarts expressions:

module ogetarts/front/parse
imports libstratego-sglr libstratego-lib
strategies

  parse-ogetarts-stream =
    where(tbl := <import-term(Ogetarts.tbl); open-parse-table>)
    ; finally(
        parse-stream(strsglr-report-parse-error | tbl)
      , <close-parse-table> tbl
      )

So how does the compiler know where to search for the packages (directories)? This is specified by the -I option to strc. In our case, we did already specify lib/ as the basepath for our library in the section on compiling Stratego programs. Thus, all module and package references are looked up from there. For programs using several libraries installed at different locations, multiple base directories should be specified, each with the -I option.

include $(top_srcdir)/Makefile.xt
include $(wildcard *.dep)

lib_LTLIBRARIES = libogetarts.la 
pkgdata_DATA    = libogetarts.rtree
EXTRA_DIST      = $(ogetartssrc) libogetarts.rtree
CLEANFILES      = libogetarts.c libogetarts.rtree

libogetarts_la_SOURCES  = libogetarts.c
libogetarts_la_LDFLAGS  = -avoid-version -no-undefined
libogetarts_la_CPPFLAGS = \
  $(STRATEGO_LIB_CFLAGS) $(STRATEGO_RUNTIME_CFLAGS) $(ATERM_CFLAGS)
libogetarts_la_LIBADD   = \
  $(STRATEGO_SGLR_LIBS) $(STRATEGO_LIB_LIBS) $(STRATEGO_RUNTIME_LIBS) $(ATERM_LIBS)

ogetartssrc = \
  ogetarts.str \
  $(wildcard $(srcdir)/ogetarts/*.str) \
  $(wildcard $(srcdir)/ogetarts/front/*.str)

STRINCLUDES = -I $(top_srcdir)/syn

libogetarts.c libogetarts.rtree : $(ogetartssrc)
    @$(STRC)/bin/strc -c --library -i ogetarts.str -o libogetarts.rtree $(STRINCLUDES)
    rm libogetarts.str

$ ./configure ... LDFLAGS=-static

module ogetarts-eval
imports
  libstratego-lib
  libogetarts

strategies

  io-ogetarts-eval =
    io-stream-wrap(
      ?(<id>, fout)
      ; parse-ogetarts-stream
      ; ogetarts-eval
      ; ?Int(<id>)
      ; <fputs> (<id>, fout); <fputs> ("\n", fout)
    )

bin_PROGRAMS = ogetarts-eval

ogetarts_eval_LDADD = \
  $(top_builddir)/lib/libogetarts.la \
  $(STRATEGO_LIB_LIBS) $(STRATEGO_RUNTIME_LIBS) $(ATERM_LIBS)

$ echo "1+2" | ./tools/ogetarts-eval

How to add a pkg-config file to the package.

Let's finish up the root build system first, closing with the ogetarts.spec.in file.

Summary: ogetarts
Name: @PACKAGE_TARNAME@
Version: @PACKAGE_VERSION@
Release: 1
License: LGPL
Group: Development/Tools/Ogetarts
URL: http://www.ogetarts.org/Ogetarts
Source: @PACKAGE_TARNAME@-@PACKAGE_VERSION@.tar.gz
BuildRoot: %{_tmppath}/%{name}-@PACKAGE_VERSION@-buildroot
Requires: aterm >= 2.5
Requires: sdf2-bundle >= 2.4
Requires: strategoxt >= 0.17
Provides: %{name} = %{version}

%description

%prep
%setup -q

%build
CFLAGS="-D__NO_CTYPE" ./configure --prefix=%{_prefix} --with-strategoxt=%{_prefix} \
       --with-aterm=%{_prefix} --with-sdf=%{_prefix}
make

%install
rm -rf $RPM_BUILD_ROOT
make DESTDIR=$RPM_BUILD_ROOT install

%clean
rm -rf $RPM_BUILD_ROOT

%files
%defattr(-,root,root,-)
%{_bindir}
%{_libexecdir}
%{_datadir}
%doc

%changelog

This file is not strictly necessary. It's a so-called .spec-file, which is a package descriptor for rpm-based distributions. When you provide it, the Stratego/XT build system can automatically make a .rpm package file for your project.

In this chapter we learned how to organize Stratego/XT projects, and how to set up the Stratego/XT build system. We saw how the build system, which is based on automake and autoconf, is used to generate many of the artifacts from the syntax definition, such as parse tables and signatures, in the way we discussed in Part II . We also learned how to make stand-alone programs, libraries and XT components from Stratego source code, and how modules and packages relate to files and directories.

More detail can be found by looking at the manual pages for the build tools, such as autoxt, strc, sdf2table, sdf2rtg and xtc. The detailed examples from the Stratego/XT Examples tutorial contain working build systems that can serve as a starting point.

Both the Stratego language paradigm and its syntax are rather different from most other languages. Knowing how to use the unique features of Stratego properly, in the way we have described in this manual, goes a long way towards avoiding future maintenance problems.

  EvalExpr:
  Expr(Plus(x), Plus(y) -> Expr(z)
  where
    <addS> (x,y) => z

  
  eval-all =
      EvalExpr
    ; s1
    ; s2

 /**
  * @type A -> B
  */
  foo = ...

  strat = 
    \ x -> y where is-okay(|x) => y \

  foo = 
      debug
    ; bar
    ; debug
    ; baz

  foo = 
     where(<debug> [ "Entered foo : ", <id> ])
     ; bar
     ; where(<debug> [ "After bar : ", <id> ])
     ; baz

  foo =
    debug(!"Entered foo : ")
    ; bar
    ; debug(!"After bar : ")
    ; baz

 Eval:
 If(t, e1, e2) -> If(t, e1', e2')
 where
     <simplify> e1 => e1'
   ; <simplify> e2 => e2'
 
 Eval:
 If(False, e1, e2) -> e2

  SimplifyIf
  If(t, e1, e2) -> If(t, e1', e2')
  where
     <simplify> e1 => e1'
   ; <simplify> e2 => e2'

  EvalIfCond:
  If(False, e1, e2) -> e2
  
  Eval = EvalIfCond <+ SimplifyIf

stratego> <map(\ x -> y where !x => y \)> [1]
[1]
stratego> <map(\ x -> y where !x => y \)> [1,1,1,1]
[1,1,1,1]
stratego> <map(\ x -> y where !x => y \)> [1,2,3,4]
command failed

stratego> <map({y: \ x -> y where !x => y \})> [1,2,3,4]
[1,2,3,4]

The XT component model is based on Unix pipes. Debugging XT compositions can therefore be done using many of the familiar Unix command line tools.

Checking XTC registrations.  Whenever you call XTC components using xtc-transform, the location of the component you are calling is looked up in a component registry. When invoking a component fails, it may be because the component you are calling has been removed. Checking the registrations inside a component registry is done using the xtc command:

# xtc -r /usr/local/apps/dryad/share/dryad/XTC q -a
dryad (0.1pre11840) : /usr/local/apps/dryad/dryad
dryad.m4 (0.1pre11840) : /usr/local/apps/dryad/share/dryad/dryad.m4
...

The -r option is used to specify which registry you want to inspect. The path given to -r must be the XTC registry file of your installed program transformation system that you built with Stratego/XT. By default, xtc will work on the Stratego/XT XTC repository, and only list the components provided by Stratego/XT. This is seldom what you want.

XTC registries are hierarchical. The XTC repository of your project imports (refers back to) the other projects you used in your build process, such as Stratego/XT itself. The component list you get from xtc when giving it your repository is therefore a full closure of all components visible to transformations in your project.

Now that you know how to obtain the paths for all XT components, it is easy to determine that they actually exist at the locations recorded, and that the access rights are correct.

Programs such as strace may also be useful at the lowest level of debugging, to see which parameters are passed between components, whether a given component is located correctly, and whether execution of a given component succeeds.

Format Checking.  Each component in a system built with Stratego/XT accepts a term, definable by some grammar, and outputs another term, also definable by a (possibly the same) grammar. During debugging of XT compositions, it is useful to check that the data flowing between the various components actually conform to the defined grammars. It is not always the case that the grammar in question has been defined, but you are highly encouraged to do so, see Chapter 8 for how to define regular tree grammars.

Once you have a formal declaration of your data, in the form of a regular tree grammar, you can insert calls to the format-check between your XT components to verify data correctness, i.e. the correctness of the terms.

  ast2il = 
      xtc-transform(!"format-check", !["--rtg", "language-ast.rtg"])
    ; xtc-transform(!"ast2il")
    ; xtc-transform(!"format-check", !["--rtg", "language-il.rtg"])

The ast2il component transforms from the abstract syntax tree representation of a given language to an intermediate language (IL). format-check is used to verify that the AST passed to ast2il is well-formed, and that the result obtained from ast2il is also well-formed.

Tool Debugging Options.  Most of the XT tools accept a common set of options useful when debugging. These include --keep, for adjusting the amount of intermediate results you want to keep as separate files on disk after transformation, --verbose for adjusting the level of debugging information printed by the tool, and --statistics for displaying runtime statistics.

The SDF toolkit comes with some very useful debugging aids. The first is the sdfchecker command, which will analyze your SDF definition and offer a list of issues it finds. You do not need to invoke sdfchecker directly. It is invoked by the sdf2table by default, whenever you generate a parse table from a syntax definition. Be advised that the issues pointed to by sdfchecker are not always errors. Nontheless, it is usually prudent to fix them.

The other SDF debugging tool is the visamb command. visamb is used to display ambiguities in parse trees. Its usage is detailed in the command reference (visamb).

Pitfalls with Concrete Syntax.  Doing transformations with concrete syntax in Stratego, as explained in Chapter 19 depends in the correct placement of .meta files. When creating, splitting, moving or removing Stratego source files (.str files), it is important that you bring along the accompanying .meta files.

Another thing to be aware of with concrete syntax, is the presence of reserved meta variables. Typically, x, xs, e, t and f have a reserved meaning inside the concrete syntax fragments as being meta variables, i.e. variables in the Stratego language, not in the object language.

A final stumbling block is the general problem of ambiguities in the syntax definition. While SDF allows you to write ambiguous grammars, and sglr accepts these gracefully, you are not allowed to have ambiguous syntax fragments in your Stratego code. In cases where the Stratego compiler (strc) fails due to ambiguous fragments, you can run parse-stratego on your source code to see exactly which parts are ambiguous. The visamb tool should then be applied to the output from parse-stratego to visualize the ambiguities.