Fork me on GitHub

Reporting results

Reporters are in charge of handling the state and outcome of test-runs. They might output machine- or human-readable reports on the console, or display the results in a graphical user interface.

attest.reporters.get_reporter_by_name(name, default='auto')[source]

Get an AbstractReporter by name, falling back on a default.

Reporters are registered via setuptools entry points, in the 'attest.reporters' group. A third-party reporter can thus register itself using this in its setup.py:

setup(
    entry_points = {
        'attest.reporters': [
            'name = import.path.to:callable'
        ]
    }
)

Names for the built in reporters:

Parameters:
  • name – One of the above strings.
  • default – The fallback reporter if no reporter has the supplied name, defaulting to 'auto'.
Raises KeyError:
 

If neither the name or the default is a valid name of a reporter.

Return type:

Callable returning an instance of an AbstractReporter.

Changed in version 0.4: Reporters are registered via setuptools entry points.

attest.reporters.get_all_reporters()[source]

Iterable yielding the names of all registered reporters.

>>> from attest import get_all_reporters
>>> list(get_all_reporters())
['xml', 'plain', 'quickfix', 'fancy', 'auto']

New in version 0.4.

Testing on the command-line

attest.reporters.auto_reporter(style=None)[source]

Select a reporter based on the target output and installed dependencies.

This is the default reporter.

Parameters:style – Passed to FancyReporter if it is used.
Return type:FancyReporter if output is a terminal and the progressbar and pygments packages are installed, otherwise a PlainReporter.

Changed in version 0.5: A test_loader function attribute similar to AbstractReporter.test_loader().

class attest.reporters.FancyReporter(style=None)[source]

Heavily uses ANSI escape codes for fancy output to 256-color terminals. Progress of running the tests is indicated by a progressbar and failures are shown with syntax highlighted tracebacks.

Parameters:stylePygments style for tracebacks. If None, uses ATTEST_PYGMENTS_STYLE falling back on 'bw' because it looks good on most terminals.
class attest.reporters.PlainReporter[source]

Plain text ASCII output for humans.

Testing from Vim (and possibly other editors)

class attest.reporters.QuickFixReporter[source]

Report failures in a format that’s understood by Vim’s quickfix feature.

Write a Makefile that runs your tests with this reporter and then from Vim you can do :mak. If there’s failures, Vim will jump to the first one by opening the offending file and positioning the cursor at the relevant line; you can jump between failures with :cn and :cp. For more information try :help quickfix.

Example Makefile (remember to indent with tabs not spaces):

test:
    @python runtests.py -rquickfix

New in version 0.5.

Integrating testing with non-Python tools

class attest.reporters.XmlReporter[source]

Report the result of a testrun in an XML format. Not compatible with JUnit or XUnit.

Writing new reporters

class attest.reporters.AbstractReporter[source]

Optional base for reporters, serves as documentation and improves errors for incomplete reporters.

classmethod test_loader()[source]

Creates a basic unittest test loader using this reporter. This can be used to run tests via distribute, for example:

setup(
    test_loader='attest:FancyReporter.test_loader',
    test_suite='tests.collection',
)

Now, python setup.py -q test is equivalent to:

from attest import FancyReporter
from tests import collection
collection.run(FancyReporter)

If you want to run the tests as a normal unittest suite, try test_suite() instead:

setup(
    test_suite='tests.collection.test_suite'
)

New in version 0.5.

begin(tests)[source]

Called when a test run has begun.

Parameters:tests – The list of test functions we will be running.
success(result)[source]

Called when a test succeeds.

Parameters:result (TestResult) – Result data for the succeeding test.

Changed in version 0.4: Parameters changed to result.

failure(result)[source]

Called when a test fails.

Parameters:result (TestResult) – Result data for the failing test.

Changed in version 0.4: Parameters changed to result.

finished()[source]

Called when all tests have run.

class attest.reporters.TestResult[source]

Container for result data from running a test.

New in version 0.4.

error

The exception instance, if the test failed.

exc_info

The exc_info() of the exception, if the test failed.

raw_traceback[source]

Like traceback.extract_tb() with uninteresting entries removed.

New in version 0.5.

stderr

A list of lines the test printed on the standard error.

stdout

A list of lines the test printed on the standard output.

test

The test callable.

test_name[source]

A representative name for the test, similar to its import path.

traceback[source]

The traceback for the exception, if the test failed, cleaned up.