Skip to main content

CLI

testplane command​

Main command to run tests.

> testplane --help

  Usage: testplane [options] [command] [paths...]

  Run tests

  Options:

    -V, --version                    output the version number
    -c, --config <path>              path to configuration file
    -b, --browser <browser>          run tests only in specified browser
    -s, --set <set>                  run tests only in the specified set
    -r, --require <module>           require module
    --grep <grep>                    run only tests matching the pattern
    --reporter <name>                test reporters
    --update-refs                    update screenshot references or gather them if they do not exist ("assertView" command)
    --inspect [inspect]              nodejs inspector on [=[host:]port]
    --inspect-brk [inspect-brk]      nodejs inspector with break at the start
    --repl [type]                    run one test, call `browser.switchToRepl` in test code to open repl interface (default: false)
    --repl-before-test [type]        open repl interface before test run (default: false)
    --repl-on-fail [type]            open repl interface on test fail only (default: false)
    --devtools                       switches the browser to the devtools mode with using CDP protocol
    -h, --help                       output usage information

For example,

npx testplane --config ./config.js --reporter flat --browser firefox --grep name

Options​

Version​

Print current testplane version.

testplane --version

Config​

Use custom configuration file.

testplane --config ./local.testplane.config.ts

Browser​

Run tests only in specified browser.

testplane --browser chrome

Set​

Run tests only in the specified set.

testplane --set desktop

Require​

Load external modules, locally existing in your machine, before running testplane. This is useful for loaders, such as ECMAScript modules via esm.

testplane --require ./tsconfig-register-paths.js

Reporter​

Can be used to set one of the following reporters:

  • flat - all information about failed and retried tests would be grouped by browsers at the end of the report;
  • plain - information about fails and retries would be placed after each test;
  • jsonl - displays detailed information about each test result in jsonl form.

Default is flat.

Information about test results is displayed to the command line by default. But there is an ability to redirect the output to a file:

testplane --reporter '{"type": "jsonl", "path": "./some-path/result.jsonl"}'

You can also specify multiple reporters:

testplane --reporter '{"type": "jsonl", "path": "./some-path/result.jsonl"}' --reporter flat

Aside these terminal reporters, you can use html-reporter plugin in order to generate html reports.

Grep​

Run only tests, which full name matches the pattern.

Example​

If you have some tests:

describe("test", () => {
    describe("with", () => {
        describe("nested path", () => {
            ...
        });
        describe("other path", () => {
            ...
        })
    });
});

You can run tests inside "nested path" suite without running tests inside "other path" suite with any of these variants:

testplane --grep "nested path"
testplane --grep "with nested path"
testplane --grep "test with nested path"

Update refs​

Run tests, updating all screenshot references, created by assertView command.

testplane --update-refs
info

Recommended way to update screenshots is using html-reporter plugin.

Inspect​

Runs Testplane tests using nodejs inspector.

testplane --inspect
info

In the debugging mode, only one worker is started and all tests are performed only in it. Use this mode with option sessionsPerBrowser=1 in order to debug tests one at a time.

Inspect break​

The same as Inspect, but with breakpoint at the start.

testplane --inspect-brk

REPL​

Enables a REPL. It also disables test duration timeout. Can be used by specifying following CLI options:

  • --repl - in this mode, only one test in one browser should be run, otherwise an error is thrown. REPL interface does not start automatically, so you need to call switchToRepl command in the test code;
  • --repl-before-test - the same as --repl option except that REPL interface opens automatically before run test;
  • --repl-on-fail - the same as --repl option except that REPL interface opens automatically on test fail.
testplane --repl --grep 'my test name' --browser chrome
info

More information about Testplane REPL mode can be found in switchToRepl command documentation.

Devtools​

Runs Testplane tests using devtools automation protocol.

testplane --devtools

Help​

Prints out information about options and commands. Testplane Plugins can add their own commands and options.

For example, html-reporter adds gui command.

testplane --help

list-tests command​

Command to get list of tests in one of available formats (list or tree).

> testplane list-tests --help

  Usage: list-tests [options] [paths...]

  Lists all tests info in one of available formats

  Options:

    -c, --config <path>        path to configuration file
    -b, --browser <browser>    list tests only in specified browser
    -s, --set <set>            list tests only in the specified set
    -r, --require <module>     require module
    --grep <grep>              list only tests matching the pattern
    --ignore <file-path>       exclude paths from tests read
    --silent [type]            flag to disable events emitting while reading tests (default: false)
    --output-file <file-path>  save results to specified file
    --formatter [name]         return tests in specified format (default: list)
    -h, --help                 output usage information

For example,

npx testplane list-tests --config ./config.js --browser firefox --grep name --formatter tree

Options​

Config​

Use custom configuration file.

testplane list-tests --config ./local.testplane.config.ts

Browser​

List tests only in specified browser.

testplane list-tests --browser chrome

Set​

List tests only in the specified set.

testplane list-tests --set desktop

Require​

Load external modules, locally existing in your machine, before listing testplane tests. This is useful for loaders, such as ECMAScript modules via esm.

testplane list-tests --require ./tsconfig-register-paths.js

Grep​

List only tests, which full name matches the pattern.

Example​

If you have some tests:

describe("test", () => {
    describe("with", () => {
        describe("nested path", () => {
            ...
        });
        describe("other path", () => {
            ...
        })
    });
});

You can list tests inside "nested path" suite without listing tests inside "other path" suite with any of these variants:

testplane list-tests --grep "nested path"
testplane list-tests --grep "with nested path"
testplane list-tests --grep "test with nested path"

Silent​

List tests silently (without emitting events).

testplane list-tests --silent

Output file​

Save listed tests to specified file.

testplane list-tests --output-file "./tests.json"

Formatter​

List tests in specified format. Available formatters: list (default) and tree.

List formatter​

List tests in list format.

testplane list-tests --formatter "list"

For following tests:

example.testplane.ts
describe("suite1", () => {
    it("test1", () => {});
});

it("test2", () => {});

Returns the following output:

[
    {
        "id": "d2b3179",
        "titlePath": [
            "suite",
            "test1"
        ],
        "browserIds": [
            "yandex",
            "chrome"
        ],
        "file": "tests/example.testplane.ts",
        "pending": false,
        "skipReason": ""
    }
    {
        "id": "5a105e8",
        "titlePath": [
            "test1"
        ],
        "browserIds": [
            "yandex",
            "chrome"
        ],
        "file": "tests/example.testplane.ts",
        "pending": false,
        "skipReason": ""
    }
]

Here, we got plain list of unique tests, where:

  • id (String) - unique identifier of the test;
  • titlePath (String[]) - full name of the test. Each element of the array is the title of a suite or test. To get the full title, you need just join titlePath with single whitespace;
  • browserIds (String[]) - list of browsers in which the test will be launched;
  • file (String) - path to the file relative to the working directory;
  • pending (Boolean) - flag that means if the test is disabled or not;
  • skipReason (String) - the reason why the test was disabled.
Tree formatter​

List tests in tree format.

testplane list-tests --formatter "tree"

For following tests:

example.testplane.ts
describe("suite1", () => {
    it("test1", () => {});
});

it("test2", () => {});

Returns the following output:

[
    {
        "id": "36749990",
        "title": "suite1",
        "line": 1,
        "column": 1,
        "file": "example.testplane.ts",
        "suites": [],
        "tests": [
            {
                "id": "d2b3179",
                "title": "test1",
                "line": 2,
                "column": 5,
                "browserIds": ["yandex", "chrome"]
            }
        ],
        "pending": false,
        "skipReason": ""
    },
    {
        "id": "5a105e8",
        "title": "test2",
        "line": 5,
        "column": 1,
        "browserIds": ["yandex", "chrome"],
        "file": "example.testplane.ts",
        "pending": false,
        "skipReason": ""
    }
]

Here, we got list of unique tests in the form of a tree structure (with parent suites), where Suite has following options:

  • id (String) - unique identifier of the suite;
  • title (String) - unique identifier of the suite;
  • line (Number) - line on which the suite was called;
  • column (Number) - column on which the suite was called;
  • file (String, only in topmost suite) - path to the file relative to the working directory;
  • suites (Suite[], exists only in suite) - list of child suites;
  • tests (Test[]) - list of tests;
  • pending (Boolean) - flag that means if the test is disabled or not;
  • skipReason (String) - the reason why the test was disabled.

And Test has following options:

  • id (String) - unique identifier of the test;
  • title (String) - unique identifier of the test;
  • line (Number) - line on which the test was called;
  • column (Number) - column on which the test was called;
  • browserIds (String[]) - list of browsers in which the test will be launched;
  • file (String, only in tests without parent suites) - path to the file relative to the working directory;
  • pending (Boolean) - flag that means if the test is disabled or not;
  • skipReason (String) - the reason why the test was disabled.

Help​

Prints out information about options and commands. Testplane Plugins can add their own commands and options.

For example, html-reporter adds gui command.

testplane list-tests --help

Overriding settings​

All options can be overridden via command-line flags or environment variables with the following priorities, by descending:

  • A command-line option;

  • An environment variable;

  • A config file value;

  • The default value.

Overriding with CLI option​

To override a config setting with a CLI option, convert the full option path to --kebab-case.

For example, to override config's baseUrl property:

testplane path/to/mytest.js --base-url http://example.com

It also works for nested properties. Example of overriding browsers.firefox.sessionsPerBrowser value (assuming you have defined browser with id "firefox"):

testplane path/to/mytest.js --browsers-firefox-sessions-per-browser 7

Overriding with environment variable​

To override a setting with an environment variable, convert its full path to snake_case and add the testplane_ prefix.

Examples of overriding the same baseUrl and browsers.firefox.sessionsPerBrowser values using environment variables instead of CLI options:

testplane_base_url=http://example.com testplane path/to/mytest.js
testplane_browsers_firefox_sessions_per_browser=7 testplane path/to/mytest.js

Besides overriding config values, there also two extra environment variables: "TESTPLANE_SKIP_BROWSERS" and "TESTPLANE_SETS":

TESTPLANE_SKIP_BROWSERS​

Skip the browsers specified in the config by passing the browser IDs. Multiple browser IDs should be separated by commas (spaces after commas are allowed).

TESTPLANE_SKIP_BROWSERS=ie10,ie11 testplane

TESTPLANE_SETS​

Runs only specified sets (CLI option --set alternative).

TESTPLANE_SETS=desktop,touch testplane