Chapter 27. Unit Testing with SUnit

Table of Contents

27.1. Setting up a test suite
27.2. Compare expected and actual output
27.3. Check for failure
27.4. Check arbitrary conditions on output
27.5. Unit testing with XTC

StrategoUnit or SUnit is a Unit Testing framework for Stratego inspired by JUnit. The idea is to specify tests that apply a strategy to a specific term and compare the result to the expected output. The tests are combined into a test suite, which runs all tests and reports the number of successes and failures. When all tests pass, the program exits with status 0, otherwise it exists with status 1 to flag the error to the calling program, which is typically a makefile.

27.1. Setting up a test suite

Module list-zip-test.str in the Stratego Standard Library is an example test suite. SUnit test suites can be compiled to an executable program. When run the program gives the following output:

test suite: list-zip-test
nzip0-test
zip-test1
zip-test2
successes: 3
failures: 0
(3,0)

An extract from the module shows how a test suite can be set up:

module list-zip-test
imports list-zip sunit
strategies

  main =
    test-suite(!"list-zip-test",
        nzip0-test;
        zip-test
     )

  nzip0-test =
    apply-test(!"nzip0-test"
        ,nzip0(id)
        ,!["a","b","c"]
        ,![(0, "a"), (1, "b"), (2, "c")]
        )

The test-suite strategy takes a string and a strategy. The strategy is typically a sequential composition of calls to unit tests. Unit tests such as nzip0-test are defined using one of the unit-test strategies from module sunit, which is in the standard library.

27.2. Compare expected and actual output

The prototypical unit test is composed with the apply-test strategy, which takes a name, the strategy that is being tested, the input term, and the exptected output term. Note that the name, input, and output term must be specified in strategies, which means that if literal terms are used then you must build them using the ! operator.

apply-test(!"test3"
, ltS
, !("3","5")
, !("3","5")
)

27.3. Check for failure

It is also useful to test that a strategy fails if applied to a certain input. The ordinary apply-test cannot be used for this because it requires an output term to which the result will be compared. For testing failure the apply-and-fail strategy is available in sunit. It ensures that the strategy fails if applied to a certain input.

apply-and-fail(!"is-double-quote 3"
, is-double-quoted
, !"\"fnord"
)

27.4. Check arbitrary conditions on output

Sometimes a test just want to check some condition on the output of a strategy application, without actually specifying the output itself. The apply-test strategy cannot be used for this because it performs an equivalence test of the required output and the actual output. In this case apply-and-check can be used. This strategy takes 4 arguments: a name, a strategy (s) to test, an input and a strategy that will be applied to the result of applying s to the input. This strategy must succeed if the output is correct, or fail if the output is incorrect.

new-test =
  apply-and-check(!"new test",
    (new, new)
  , !(1, 1)
  , not(eq); (is-string, is-string)
  )

27.5. Unit testing with XTC

XTC is the preferred system for writing components in Stratego/XT. Testing that an XTC component acts as intended is not much more difficult than for individual strategies. As the semantics of the interfaces to XTC programs is open-ended, there is currently no generic testing framework for handling all the details.

Nevertheless, adapting the unit testing framework explained above for testing your components is easy. Consider the strategy xtc-apply-test, given below.

xtc-apply-test(|nm, prog, args, inp, outp) =
  xtc-temp-files(
    apply-test(!nm,
        write-to
      ; xtc-transform(!prog, !args)
      ; read-from
    , !inp
    , !outp
    )
  )

This is a straightforward extension of apply-test. It will invoke the XTC program given in the prog term argument with the list of command line arguments in args. The input to the program is the term given by inp. The resulting term after prog has been run is checked against outp. If they match, the test succeeds.

The simple code above does not account for additional file arguments to prog. This may be useful if prog if prog is, say, a pretty-printer which takes both a document and a style sheet as arguments. The code below provides an example of how this may be handled.

xtc-apply-pp-test(|nm, prog, ssheet, args, inp, outp) =
  xtc-temp-files(
    apply-test(!nm,
      where(<print-to> [ssheet] => FILE(f))
      ; write-to
      ; xtc-transform(!prog, <concat> [ args, "--stylesheet", f])
      ; read-from
    , !inp
    , !outp
    )
  )