CLI

testplane command

Main command to run tests.

> testplane --help

  Usage: testplane [options] [command] [paths...]

  Run tests

  Options:

    -V, --version                    output the version number
    -c, --config <path>              path to configuration file
    -b, --browser <browser>          run tests only in specified browser
    -s, --set <set>                  run tests only in the specified set
    -r, --require <module>           require module
    --grep <grep>                    run only tests matching the pattern
    --reporter <name>                test reporters
    --update-refs                    update screenshot references or gather them if they do not exist ("assertView" command)
    --inspect [inspect]              nodejs inspector on [=[host:]port]
    --inspect-brk [inspect-brk]      nodejs inspector with break at the start
    --repl [type]                    run one test, call `browser.switchToRepl` in test code to open repl interface (default: false)
    --repl-before-test [type]        open repl interface before test run (default: false)
    --repl-on-fail [type]            open repl interface on test fail only (default: false)
    --devtools                       switches the browser to the devtools mode with using CDP protocol
    --local                          use automatically downloaded browsers and drivers, provided by Testplane
    -h, --help                       output usage information

For example,

npx testplane --config ./config.js --reporter flat --browser firefox --grep name

Options

Version

Print current testplane version.

testplane --version

Config

Use custom configuration file.

testplane --config ./local.testplane.config.ts

Browser

Run tests only in specified browser.

testplane --browser chrome

Set

Run tests only in the specified set.

testplane --set desktop

Require

Load external modules, locally existing in your machine, before running testplane. This is useful for loaders, such as ECMAScript modules via esm.

testplane --require ./tsconfig-register-paths.js

Reporter

Can be used to set one of the following reporters:

• flat - all information about failed and retried tests would be grouped by browsers at the end of the report;
• plain - information about fails and retries would be placed after each test;
• jsonl - displays detailed information about each test result in jsonl form.

Default is flat.

Information about test results is displayed to the command line by default. But there is an ability to redirect the output to a file:

testplane --reporter '{"type": "jsonl", "path": "./some-path/result.jsonl"}'

You can also specify multiple reporters:

testplane --reporter '{"type": "jsonl", "path": "./some-path/result.jsonl"}' --reporter flat

Aside these terminal reporters, you can use html-reporter plugin in order to generate html reports.

Grep

Run only tests, which full name matches the pattern.

Example

If you have some tests:

describe("test", () => {
    describe("with", () => {
        describe("nested path", () => {
            ...
        });
        describe("other path", () => {
            ...
        })
    });
});

You can run tests inside "nested path" suite without running tests inside "other path" suite with any of these variants:

testplane --grep "nested path"
testplane --grep "with nested path"
testplane --grep "test with nested path"

Update refs

Run tests, updating all screenshot references, created by assertView command.

testplane --update-refs

info

Recommended way to update screenshots is using html-reporter plugin.

Inspect

Runs Testplane tests using nodejs inspector.

testplane --inspect

info

In the debugging mode, only one worker is started and all tests are performed only in it. Use this mode with option sessionsPerBrowser=1 in order to debug tests one at a time.

Inspect break

The same as Inspect, but with breakpoint at the start.

testplane --inspect-brk

REPL

Enables a REPL. It also disables test duration timeout. Can be used by specifying following CLI options:

• --repl - in this mode, only one test in one browser should be run, otherwise an error is thrown. REPL interface does not start automatically, so you need to call switchToRepl command in the test code;
• --repl-before-test - the same as --repl option except that REPL interface opens automatically before run test;
• --repl-on-fail - the same as --repl option except that REPL interface opens automatically on test fail.

testplane --repl --grep 'my test name' --browser chrome

info

More information about Testplane REPL mode can be found in switchToRepl command documentation.

Devtools

Runs Testplane tests using devtools automation protocol.

testplane --devtools

Help

Prints out information about options and commands. Testplane Plugins can add their own commands and options.

For example, html-reporter adds gui command.

testplane --help

list-tests command

Command to get list of tests in one of available formats (list or tree).

> testplane list-tests --help

  Usage: list-tests [options] [paths...]

  Lists all tests info in one of available formats

  Options:

    -c, --config <path>        path to configuration file
    -b, --browser <browser>    list tests only in specified browser
    -s, --set <set>            list tests only in the specified set
    -r, --require <module>     require module
    --grep <grep>              list only tests matching the pattern
    --ignore <file-path>       exclude paths from tests read
    --silent [type]            flag to disable events emitting while reading tests (default: false)
    --output-file <file-path>  save results to specified file
    --formatter [name]         return tests in specified format (default: list)
    -h, --help                 output usage information

For example,

npx testplane list-tests --config ./config.js --browser firefox --grep name --formatter tree

Options

Config

Use custom configuration file.

testplane list-tests --config ./local.testplane.config.ts

Browser

List tests only in specified browser.

testplane list-tests --browser chrome

Set

List tests only in the specified set.

testplane list-tests --set desktop

Require

Load external modules, locally existing in your machine, before listing testplane tests. This is useful for loaders, such as ECMAScript modules via esm.

testplane list-tests --require ./tsconfig-register-paths.js

Grep

List only tests, which full name matches the pattern.

Example

If you have some tests:

describe("test", () => {
    describe("with", () => {
        describe("nested path", () => {
            ...
        });
        describe("other path", () => {
            ...
        })
    });
});

You can list tests inside "nested path" suite without listing tests inside "other path" suite with any of these variants:

testplane list-tests --grep "nested path"
testplane list-tests --grep "with nested path"
testplane list-tests --grep "test with nested path"

Silent

List tests silently (without emitting events).

testplane list-tests --silent

Output file

Save listed tests to specified file.

testplane list-tests --output-file "./tests.json"

Formatter

List tests in specified format. Available formatters: list (default) and tree.

List formatter

List tests in list format.

testplane list-tests --formatter "list"

For following tests:

example.testplane.ts

describe("suite1", () => {
    it("test1", () => {});
});

it("test2", () => {});

Returns the following output:

[
    {
        "id": "d2b3179",
        "titlePath": [
            "suite",
            "test1"
        ],
        "browserIds": [
            "yandex",
            "chrome"
        ],
        "file": "tests/example.testplane.ts",
        "pending": false,
        "skipReason": ""
    }
    {
        "id": "5a105e8",
        "titlePath": [
            "test1"
        ],
        "browserIds": [
            "yandex",
            "chrome"
        ],
        "file": "tests/example.testplane.ts",
        "pending": false,
        "skipReason": ""
    }
]

Here, we got plain list of unique tests, where:

• id (String) - unique identifier of the test;
• titlePath (String[]) - full name of the test. Each element of the array is the title of a suite or test. To get the full title, you need just join titlePath with single whitespace;
• browserIds (String[]) - list of browsers in which the test will be launched;
• file (String) - path to the file relative to the working directory;
• pending (Boolean) - flag that means if the test is disabled or not;
• skipReason (String) - the reason why the test was disabled.

Tree formatter

List tests in tree format.

testplane list-tests --formatter "tree"

For following tests:

example.testplane.ts

describe("suite1", () => {
    it("test1", () => {});
});

it("test2", () => {});

Returns the following output:

[
    {
        "id": "36749990",
        "title": "suite1",
        "line": 1,
        "column": 1,
        "file": "example.testplane.ts",
        "suites": [],
        "tests": [
            {
                "id": "d2b3179",
                "title": "test1",
                "line": 2,
                "column": 5,
                "browserIds": ["yandex", "chrome"]
            }
        ],
        "pending": false,
        "skipReason": ""
    },
    {
        "id": "5a105e8",
        "title": "test2",
        "line": 5,
        "column": 1,
        "browserIds": ["yandex", "chrome"],
        "file": "example.testplane.ts",
        "pending": false,
        "skipReason": ""
    }
]

Here, we got list of unique tests in the form of a tree structure (with parent suites), where Suite has following options:

• id (String) - unique identifier of the suite;
• title (String) - unique identifier of the suite;
• line (Number) - line on which the suite was called;
• column (Number) - column on which the suite was called;
• file (String, only in topmost suite) - path to the file relative to the working directory;
• suites (Suite[], exists only in suite) - list of child suites;
• tests (Test[]) - list of tests;
• pending (Boolean) - flag that means if the test is disabled or not;
• skipReason (String) - the reason why the test was disabled.

And Test has following options:

• id (String) - unique identifier of the test;
• title (String) - unique identifier of the test;
• line (Number) - line on which the test was called;
• column (Number) - column on which the test was called;
• browserIds (String[]) - list of browsers in which the test will be launched;
• file (String, only in tests without parent suites) - path to the file relative to the working directory;
• pending (Boolean) - flag that means if the test is disabled or not;
• skipReason (String) - the reason why the test was disabled.

Help

Prints out information about options and commands. Testplane Plugins can add their own commands and options.

For example, html-reporter adds gui command.

testplane list-tests --help

list-browsers command

Command to get all browsers from the config.

> npx testplane list-browsers --help

  Usage: list-browsers [options]

  Lists all browsers from the config

  Options:

    --type [type]      return browsers in specified type ('tags': browserName and browserVersion, 'ids': browserId from config) (default: tags)
    --format [format]  return browsers in specified format ('json' / 'plain') (default: json)
    -h, --help         output usage information

For example,

npx testplane list-browsers --type ids --format plain

Options

Type

Specify type of output data: browserId from config or browserName with browserVersion.

testplane list-browsers --type tags

Format

Specify format of output data: plain text or json.

testplane list-browsers --format json

Example

If you have the following "browsers" section in your Testplane config:

{
    "my-chrome": { "desiredCapabilities": { "browserName": "chrome", "browserVersion": "130.0" } },
    "my-safari": { "desiredCapabilities": { "browserName": "safari" } }
}

The following commands will provide corresponding result:

Command	Result
testplane list-browsers --type ids --format plain	my-chrome my-safari
testplane list-browsers --type ids --format json	["my-chrome","my-safari"]
testplane list-browsers --type tags --format plain	chrome@130.0 safari
testplane list-browsers --type tags --format json	[{"browserName":"chrome","browserVersion":"130.0"},{"browserName":"safari"}]

Help

Prints out information about the command and its options.

testplane list-browsers --help

The config Command

This command is used to retrieve all configuration values of Testplane, including default values and overridden by environment variables ones.

> npx testplane config --help

  Usage: config [options]

  Outputs Testplane config (including default and overriden by environment variables values)

  Options:

    -c, --config <path>  path to configuration file
    --space <count>      white spaces count to insert into the JSON output (default: 0)
    -h, --help           output usage information

For example:

npx testplane config --space 4

Options

Space

Specify the number of spaces used for formatting the human-readable output of the command.

testplane config --space 4

Help

Outputs information about the command and its options.

testplane config --help

install-deps command

This command is a part of the guide How to launch Testplane in the local browser.

Use the install-deps command to download all browsers, specified in Testplane config.

If this command is launched on supported ubuntu version, all missing necessary ubuntu packages would be downloaded too.

Usage

npx testplane install-deps

Command arguments

You can also specify, which browsers are need to be downloaded.

For example, if you have browsers "chrome-dark", "firefox-dark" described in Testplane config, you can use the following command to only download these two browsers:

npx testplane install-deps chrome-dark firefox-dark

Using browser names with versions in a format like <browserName>@<browserVersion> is also supported:

npx testplane install-deps chrome@130 firefox@104

Installation directory

By default, browsers and drivers are downloaded to ".testplane" directory at home directory.

You can specify other path by setting TESTPLANE_BROWSERS_PATH env variable:

TESTPLANE_BROWSERS_PATH=./node_modules/.testplane npx testplane install-deps
TESTPLANE_BROWSERS_PATH=~/.testplane npx testplane install-deps

Overriding settings

All options can be overridden via command-line flags or environment variables with the following priorities, by descending:

• A command-line option;

• An environment variable;

• A config file value;

• The default value.

Overriding with CLI option

To override a config setting with a CLI option, convert the full option path to --kebab-case.

For example, to override config's baseUrl property:

testplane path/to/mytest.js --base-url http://example.com

It also works for nested properties. Example of overriding browsers.firefox.sessionsPerBrowser value (assuming you have defined browser with id "firefox"):

testplane path/to/mytest.js --browsers-firefox-sessions-per-browser 7

Overriding with environment variable

To override a setting with an environment variable, convert its full path to snake_case and add the testplane_ prefix.

Examples of overriding the same baseUrl and browsers.firefox.sessionsPerBrowser values using environment variables instead of CLI options:

testplane_base_url=http://example.com testplane path/to/mytest.js
testplane_browsers_firefox_sessions_per_browser=7 testplane path/to/mytest.js

Besides overriding config values, there also two extra environment variables: "TESTPLANE_SKIP_BROWSERS" and "TESTPLANE_SETS":

TESTPLANE_SKIP_BROWSERS

Skip the browsers specified in the config by passing the browser IDs. Multiple browser IDs should be separated by commas (spaces after commas are allowed).

TESTPLANE_SKIP_BROWSERS=ie10,ie11 testplane

TESTPLANE_SETS

Runs only specified sets (CLI option --set alternative).

TESTPLANE_SETS=desktop,touch testplane