Html-Reporter Setup

Overview

Use the html-reporter plugin to get an html-report on the tests run.

Requirements for testplane

For the html-reporter plugin to work correctly, hermione version 4 and higher is required.

The plugin saves the tests run results to the SQLite database. Therefore, you will not be able to open the local report as a file using the file:// protocol.

To view the report, run Testplane in GUI mode:

npx testplane gui

Or run http-server in the report folder:

npx http-server -p 8000

If you are starting a local server not from the folder with the report, then specify the path to the report:

npx http-server ./testplane-report -p 8000

Then open the page http://localhost:8000 in the browser.

Install

npm install -D html-reporter

Setup

Add the plugin to the plugins section of the Testplane config:

Minimum config
Maximum config

    module.exports = {
        plugins: {
            'html-reporter/testplane': {
                enabled: true
            },

            // other Testplane plugins...
        },

        // other Testplane settings...
    };

    module.exports = {
        plugins: {
            'html-reporter/testplane': {
                enabled: true,
                path: 'my/testplane-reports',
                defaultView: 'all',
                baseHost: 'test.com',
                errorPatterns: [
                    'Parameter .* must be a string',
                    {
                        name: 'Cannot read property of undefined',
                        pattern: 'Cannot read property .* of undefined',
                        hint: '<div>google it, i dont know how to fix it =(</div>'
                    }
                ],
                customScripts: [
                    function() {
                        // custom script
                    },

                    // other scripts...
                ],
                customGui: {
                    // DEPRECATED
                    // additional controls for GUI mode
                    // use report plugins instead of customGui
                },
                pluginsEnabled: true,
                plugins: [
                    // report plugins...
                ],
                yandexMetrika: {
                    counter: 1234567
                }
            },

            // other Testplane plugins...
        },

        // other Testplane settings...
    };

Description of configuration parameters

Parameter	Type	Default value	Description
enabled	Boolean	true	Enable / disable the plugin.
path	String	"testplane-report"	The path to the folder for saving html-report files.
saveErrorDetails	Boolean	false	Save / do not save error details in json files.
defaultView	String	"all"	The test filtering mode when displayed, which will be set by default.
diffMode	String	"3-up"	The mode of viewing diffs, which will be set by default.
baseHost	String	N/A	Replaces the original host address for viewing in the browser.
errorPatterns	Object[] or String[]	[ ]	Error patterns with hints to improve the UX of the report.
metaInfoBaseUrls	Object	`{ }`	Base URLs for generating links in the Meta section based on meta information about the tests run.
saveFormat	String	"sqlite"	DEPRECATED. Allows you to set the format in which the results of the tests run will be saved.
customGui	Object	`{ }`	DEPRECATED. Use plugins instead. Description of custom controls for GUI mode.
pluginsEnabled	Boolean	false	Enable plugins for the report.
plugins	Object[]	[ ]	A list of plugins with their settings.
customScripts	Function[]	[ ]	A list of functions that implement custom scripts. For example, Yandex.Metrika scripts or a Bug.
yandexMetrika	Object	`{ }`	Yandex.Metrika.

enabled

Enables or disables the plugin.

path

The path to the folder for saving html-report files. By default, the files will be saved to the testplane-report folder in the current working folder.

saveErrorDetails

Save or not save error details in json files (to the error-details folder).

By default, "do not save": false.

Any Testplane plugin can add any details to the error object when it occurs. These details can help the user in debugging problems that have occurred in the test. Html-reporter saves these details in the error-details folder in a file named: <hash from the full name of the test>-<browser>_<retry number>_<timestamp>.json.

Under the stack trace, the html-reporter adds an Error details section with a <title> link pointing to the json file. The user can open this file either in the browser or in any IDE.

Example of how to add details to an error object from a plugin:

const err = new Error("some error");

err.details = {
    title: "description, will be used as url title",
    data: {}, // or [], or String
};

throw err;

defaultView

The test filtering mode when displayed, which will be set by default. The following values are available:

View mode	Description
all	all tests
passed	only passed tests
failed	only failed tests
retried	only those tests in which there were retries (re-runs)
skipped	only disabled (skipped) tests

By default: all, that is, if the parameter is not set, all tests will be displayed.

diffMode

The mode of viewing diffs, which will be set by default. The following values are available:

Diff mode	Description
3-up	all images (expected, actual, diff) in one column, under each other
3‑up‑scaled	all images (expected, actual, diff) in one row so that they fit on the screen
only-diff	only diff image
switch	reference image with the ability to switch to the actual image by mouse click
swipe	the actual image on top of the reference image, with a separator opening the reference image
onion-skin	the actual image on top of the reference with the ability to change the transparency of the actual image

By default: 3-up.

baseHost

Replaces the original host address for viewing in the browser. By default, the original host address is not changed.

errorPatterns

Error patterns are used:

to show more understandable error information if they correspond to patterns for which there is a detailed description;
in the Group by display mode with the error key selected.

Error patterns can be set either as objects or as strings.

To set the error pattern as an object, use the following format:

{
    name: '<error name>',
    pattern: '<error pattern>',
    hint: '<hint to the user>'
}

where:

Parameter	Type	Description
name	String	Error name.
pattern	String or RegExp	A regular expression or a regular string that the error should match.
hint	String	Optional parameter. A hint of what can be done with this error: why it occurred, how to fix it, etc.

If the error pattern is specified as a string, for example: <error>, then this string is automatically treated as an object of the form:

{
    name: '<error>',
    pattern: '<error>'
}

This way of setting a pattern is convenient for those errors where name and pattern are completely the same.

When one of the error patterns matches the error message, then:

the name of the error template will be displayed as the title of the error message, and the original error message will be hidden under the cut;
the hint for the error will be displayed after the error field stack. The hint can be specified as an html string. For example, <div>some-useful-hint</div>.

In the Group by mode with the selected error key, the test will be associated with the group if the test error matches the group error pattern. If the test cannot be linked to existing groups, a new group will be created.

metaInfoBaseUrls

Base URLs for generating links in the Meta section based on meta information about the tests run.

The parameter is set as an object:

{
    '<option-1>': 'value of option 1',
    '<option-2>': 'value of option 2',
    // etc.
}

For example:

{
    'file': 'base/path'
}

saveFormat

DEPRECATED

Allows you to set the format in which the results of the tests run will be saved.

Only one value is available now, which is used by default:

sqlite — save the results of the tests run in the database of SQLite format.

customGui

DEPRECATED

warning

Instead of customGui, it is recommended to use report plugins.

Description of custom controls for GUI mode.

See more in the section "Customizing GUI".

pluginsEnabled

Enable plugins for html-reporter.

plugins

A list of plugins with their settings.

See more details in the section "Plugins".

customScripts

A list of custom scripts in the form of an array of functions. Using this parameter, you can add any script to the html-report page. For example, to collect any metrics or implement additional functionality. The scripts will be executed immediately after rendering the page.

For example:

customScripts: [
    function () {
        console.info("some logs");
    },
    () => {
        const div = document.createElement("div");
        div.innerHTML = "hello";
        document.body.prepend(div);
    },
];

yandexMetrika

This parameter allows you to add Yandex.Metrika to the report. The parameter is set as an object with the key counterNumber. As the key value, you must specify the Yandex.Metrica counter number (see "How to create a counter"). The number should be set as a Number, not a String.

Also, in the Yandex.Metrika interface, go to the "Counter" tab in the settings section, click "Copy" and paste the counter code into the customScripts field.

With the help of metrics, you can find out how developers interact with your report and what kind of problems they face.

The report supports the following goals of metrics:

ACCEPT_SCREENSHOT—there was a click on the Accept button to accept a screenshot;
ACCEPT_OPENED_SCREENSHOTS—there was a click on the Accept opened button to accept screenshots from open tests.

Example of setting up Yandex.Metrika in one of the projects:

module.exports = {
    plugins: {
        'html-reporter/testplane': {
            customScripts: [
                function(){(function(m,e,t,r,i,k,a){m[i]=m[i]<tr><td>function(){(m[i].a=m[i].a</td></tr>[]).push(arguments)}; m[i].l=1*new Date();k=e.createElement(t),a=e.getElementsByTagName(t)[0],k.async=1,k.src=r,a.parentNode.insertBefore(k,a)}) (window, document, "script", "https://mc.yandex.ru/metrika/tag.js", "ym"); ym(56782912, "init", { clickmap:true, trackLinks:true, accurateTrackBounce:true, webvisor:true })},

                // other scripts...
            ],
            yandexMetrika: {
                counterNumber: 1234567
            },

            // other plugin settings...
        },

        // other Testplane plugins...
    },

    // other Testplane settings...
};

Passing parameters via the CLI

All plugin parameters that can be defined in the config can also be passed as command line options or through environment variables during Testplane startup. Use the prefix --html-reporter- for command line options and html_reporter_ for environment variables.

For example, the settings parameter path can be passed in the following ways:

testplane path/to/mytest.js --html-reporter-path custom/dir

html_reporter_path=custom/dir testplane path/to/mytest.js

Data storage

As mentioned above, html-reporter saves the tests run results to the SQLite database.

Why do we use SQLite? Because it is:

serverless, easy to connect and requires no configuration
cross-platform, runs on any operating system
single-file, easy to reuse reports and share them
faster than if you store the report on the file system
compact and has full SQL.

Files that are created during the execution of tests:

sqlite.db—Sqlite database with tests run results
data.js—report configuration
databaseUrls.json—absolute or relative URLs to SQLite databases (sqlite.db) and/or URLs of other databaseUrls.json files (see command merge-reports).

Html-Reporter Setup

Overview​

Install​

Setup​

Description of configuration parameters​

enabled​

path​

saveErrorDetails​

defaultView​

diffMode​

baseHost​

errorPatterns​

metaInfoBaseUrls​

saveFormat​

customGui​

pluginsEnabled​

plugins​

customScripts​

yandexMetrika​

Passing parameters via the CLI​

Data storage​