Lightest Documentation

This documentation provides some useful reference material related to using Lightest. For a more hands-on reference, try the Lightest tutorial!

What Is Lightest?

Lightest is a task-oriented functional and integration test automation framework built on Groovy and TestNG. It's a "framework framework" that is intended to greatly simplify the process of creating your own custom framework by implementing the testcase management, execution, and reporting for you - all you do is define the test environment and write the tasks!

You might want to try Lightest if:

• You want to whip up a test automation solution in a flash
• You find you are using a unit testing framework for functional or integration level testing
• You are testing a GUI or web application
• You want to transition from an unscripted test automation solution to a scripted one
• Your test automation framework needs to use Java libraries
• You love Groovy
• You love TestNG
• You love staying DRY

The Lightest project is hosted on Google Code. You may download releases, browse the source, or file issues there.

Running Lightest

The Lightest framework comes with a runner packaged in an executable JAR. The standalone distribution includes all required dependencies; no external libraries are needed. Tests are run from the command line by specifying a configuration file (see below for syntax) and either a list of Groovy test files (which subclass LightestTestCase, also see below) or a TestNG suite XML file which specifies the test classes and groups to run. The most important configuration to get right is the classPaths element; most other configurations have sensible defaults.

$ java -jar lightest-core-0.4-standalone.jar config.txt MyTest.groovy MyTest2.groovy

$ java -jar lightest-core-0.4-standalone.jar config.txt testng.xml

You can start the runner in interactive mode by providing the right switch:

$ java -jar lightest-core-0.4-standalone.jar --interactive config.txt MyTest.groovy

Lightest requires at least Java 1.5 . Surprisingly, you shouldn't need to install Groovy on your machine to run Lightest. However, you'll probably want to have a reasonably recent version (1.5.4 or later) to use when developing your tasks. If you don't have it, get it here!

The runner produces an HTML report, which in turn is generated from an XML report. HTML is probably the most convenient reporting format; however, it is possible to customize Lightest to create reports in other formats. See the tutorial for screenshots of the reports that are produced.

Configuration File Reference

The configuration file specified as the first argument to the runner follows a Groovy builder syntax, with the config element as its root node.

config {
...
    classPaths {
        path ('/path/to/tasks')
        path ('../path/to/tests')
    }
...
}

config {
...
    outputDir ('/path/to/report')
...
}

config {
...
    prefs (class: 'my.package.Preferences') {
        timeout (5000)
        corpus ('main')
    }
...
}

config {
...
    envs (class: 'my.package.Environment') {
        env (id: 'windows') {
            homeDir ('C:\\Documents and Settings\\me')
            appName ('my.app')
        }
        env (id: 'linux') {
            homeDir ('/home/me')
            appName ('my.app')
        }
        env (id: 'macos') {
            homeDir ('/Users/me')
            appName ('my.app')
        }
    }
...
}

config {
...
    reporters {
        reporter (class: 'my.package.Reporter') {
            updateEnabled (false)
            scheduled     (false)
        }
        reporter (class: 'my.package.Reporter2')
        reporter (class: 'my.package.Reporter3', role: 'XMLReporter') {
            prop3 ('value3')
            prop4 ('value4')
        }
    }
...
}

config {
...
    listeners {
        listener (class: 'my.package.Listener') {
            prop1 ('value1')
            prop2 ('value2')
        }
        listener (class: 'my.package.Listener2')
    }
...
}

config {
...
    dispatcherAssignmentStrategy (class: 'my.package.Strategy')
...
}

config {
...
    taskDispatchStrategy (class: 'my.package.Strategy') {
        prop1 ('value1')
        prop2 ('value2')
    }
...
}

Core Classes Overview

SimpleApi

The SimpleApi class (com.googlecode.lightest.core.SimpleApi) provides a convenient way to define the set of tasks that are available to a given testcase. It can search one or more Java package paths for tasks by name, and returns the first task found. By default, all tasks in the current working directory of the test runner are included by SimpleApi instances.

Use addPackage(String packageName) to add a new package path to the API. For example, if your tasks are defined in the package com.sample.tasks, you could do:

def api = new SimpleApi()
api.addPackage('com.sample.tasks')

Or simply:

def api = new SimpleApi('com.sample.tasks')

SimpleApi implements IDomainSpecificApi, specifically its getTask(String name) method. You may use a custom implementation of this interface that doesn't obtain tasks by package at all, and uses some other mechanism.

LightestTestCase

All Lightest tests extend LightestTestCase (com.googlecode.lightest.core.LightestTestCase), which is responsible for properly interpreting tasks specified with the builder syntax in its test methods. Subclasses should call setApi(IDomainSpecificApi api) before invoking any tasks. This can be done in the constructor of the test class, or in a @Before method. In its simplest form:

class MyTest extends LightestTestCase {

    MyTest() {
        setApi(new SimpleApi())
    }
}

LightestTestCase's are run by the TestNG runner; you must always use the @Test annotation to mark test methods and @Before / @After to mark fixture setup and teardown methods, respectively. To make use of JUnit assertions, add the appropriate static import to the top of the file:

import static org.testng.AssertJUnit.*

You can access the test environment (implementing ITestEnvironment) assigned to the testcase in any given test method directly via the env property. You can access the preferences via prefs. You may also obtain either from the context, e.g. context.env:

    @Test
    void myTest() {
        def port = env.port  // equivalent to "context.env.port"
        ...
    }

You can add properties to the context in the scope of any test method as if it were a Map, and these can be accessed by any tasks that are invoked, also via the context. Each test method always starts with a fresh context. Note that some special property names are reserved, i.e. env and prefs .

LightestTask

Each task used in a testcase should have as its implementation a corresponding subclass of LightestTask (com.googlecode.lightest.core.LightestTask). The task is responsible for performing some action, and populating a result object (see the next section) depending on whether the action succeeded or failed. Subclasses of LightestTask only have to implement a single method, doPerform(ITaskResult result):

class MyTask extends LightestTask {

    void doPerform(ITaskResult result) {
        println 'Been there, done that.'
        result.fail()
    }
}

Three key properties are available for inspection in task classes - config, prefs, and env .

The config object is essentially a groovy.util.Node from which the parameters to the task may be accessed. Attributes are accessed using the .'@attributeName' notation, and values can be accessed with .nodeValue() . For example:

    @Test
    void myTest() {
        MyTask (attr1: 'foo', attr2: 'bar, 'baz')
    }

    void doPerform(ITaskResult result) {
        println config.'@attr1'     // prints "foo"
        println config.'@attr2'     // prints "bar"
        println config.nodeValue()  // prints "baz"
    }

Two attributes on config have special meaning in the context of Lightest - description and breakpoint. If set, the String representation of description will be automatically visible with the task in the report. If for a given task breakpoint is set to any value that evaluates to true, the test will pause at that task if the runner is in interactive mode. You don't have to do anything in the task definition for these features to work.

Any preference values specified in the configuration file may be accessed from prefs. And any environment values may be accessed from env. Also available is the context object.

ITaskResult

Executing a task always produces a ITaskResult (com.googlecode.lightest.core.ITaskResult). The values populated on this object will appear in the report for the test run. The important values to consider setting are status, message, detailedMessage, and links . The ITaskResult object is passed into the doPerform() method, and should be populated therein.

The status of the result is what determines whether the task succeeded or failed. By default, the status is set to indicate success (STATUS_OK). Setting it to anything else indicates failure at some level. In order of increasing severity, you can invoke flag(), fail(), or doom() on the result object to do this.

Use setMessage(String message) to set an informational message to be displayed in the report. In particular, display any information related to why the task may have failed.

Use setDetailedMessage(String detailedMessage) to set larger chunks of text that may have been produced in performing the task.

Finally, use addLink(String link) to add a link to related resources, such as documents or screenshots. The link text will appear in the report, exactly as specified using this method. Multiple links may be specified. Use addLink(TaskResultLink link) if you need greater control of the HTML rendering of the link.

Task Lifecycle

The following table describes the lifecycle of a Lightest task, which is an implementation of ITask (and typically, but not necessarily, a subclass of LightestTask). This may be useful to know if you want to inject custom logic at specific points of the lifecycle.

class MyTask extends LightestTask {
    boolean waitForSignal
    ...
}

MyTask (waitForSignal: 'false')

waitForSignal == config.'@waitForSignal'

config.'@name'
config.'@value'

Other Resources

• Lightest API
• Lightest Tutorial
• Lightest Project Page
• Lightest @ stressfreetesting.com
• TestNG Documentation
• Groovy API

- Haw-Bin Chai (hbchai @t gmail d0t com)