How to Get a Test Run Report
Testplane Report
After completing all test runs, Testplane writes the result to the console in the form of a string like this:
Total: 1812 Passed: 1792 Failed: 0 Skipped: 20 Retries: 47
Where:
| Status | Description |
| Total | Total number of tests that Testplane read from the file system during launch. |
| Passed | Number of tests that passed successfully. |
| Failed | Number of tests that failed. |
| Skipped | Number of tests that were skipped during the run. |
| Retries | Total number of test retries that occurred during the run. |
However, this information may not be sufficient, so you can add the stat-reporter plugin to your project.
stat-reporter Plugin Report
If you add the stat-reporter plugin to your project, you will get a more detailed report of the run results in the console after completing all the tests. For example:
┌──────────────────────┬────────┬───────┬────────┬────────┬─────────┬─────────┬──────────┐
│ Browser │ Status │ Tests │ Passed │ Failed │ Skipped │ Retries │ Duration │
├──── ──────────────────┼────────┼───────┼────────┼────────┼─────────┼─────────┼──────────┤
│ firefox │ passed │ 25 │ 24 │ 0 │ 1 │ 0 │ 01:02 │
├──────────────────────┼────────┼───────┼────────┼────────┼─────────┼─────────┼──────────┤
│ chrome-desktop │ passed │ 466 │ 464 │ 0 │ 2 │ 4 │ 07:40 │
├──────────────────────┼────────┼───────┼────────┼────────┼─────────┼─────────┼──────────┤
│ ipad │ passed │ 24 │ 23 │ 0 │ 1 │ 0 │ 01:27 │
├──────────────────────┼────────┼───────┼────────┼────────┼─────────┼─────────┼──────────┤
│ iphone │ passed │ 376 │ 372 │ 0 │ 4 │ 7 │ 07:12 │
├──────────────────────┼────────┼───────┼────────┼────────┼─────────┼─────────┼──────────┤
│ chrome-phone │ passed │ 427 │ 421 │ 0 │ 6 │ 14 │ 07:32 │
├──────────────────────┼────────┼───────┼────────┼────────┼─────────┼─────────┼──────────┤
│ iphone-dark │ passed │ 74 │ 72 │ 0 │ 2 │ 4 │ 02:18 │
├──────────────────────┼────────┼───────┼────────┼────────┼─────────┼─────────┼──────────┤
│ searchapp-phone │ passed │ 319 │ 317 │ 0 │ 2 │ 9 │ 10:00 │
├──────────────────────┼────────┼───────┼────────┼────────┼─────────┼─────────┼──────────┤
│ safari13 │ passed │ 15 │ 13 │ 0 │ 2 │ 4 │ 02:42 │
├──────────────────────┼────────┼───────┼────────┼────────┼─────────┼─────────┼──────────┤
│ chrome-desktop-1920 │ passed │ 3 │ 3 │ 0 │ 0 │ 0 │ 00:57 │
├──────────────────────┼────────┼───────┼────────┼────────┼─────────┼─────────┼──────────┤
│ iphoneX │ passed │ 3 │ 3 │ 0 │ 0 │ 0 │ 00:36 │
├──────────────────────┼────────┼───────┼────────┼────────┼─────────┼─────────┼──────────┤
│ chrome-desktop-dark │ passed │ 77 │ 77 │ 0 │ 0 │ 5 │ 01:33 │
├──────────────────────┼────────┼───────┼────────┼────────┼─────────┼─────────┼──────────┤
│ yandex-browser-phone │ passed │ 1 │ 1 │ 0 │ 0 │ 0 │ 00:28 │
├──────────────────────┼────────┼───────┼────────┼────────┼─────────┼─────────┼──────────┤
│ chrome-grid-720 │ passed │ 2 │ 2 │ 0 │ 0 │ 0 │ 00:49 │
└──────────────────────┴────────┴───────┴────────┴────────┴─────────┴─────────┴──────────┘
Unlike the simple Testplane report, the stat-reporter plugin report breaks down the results by browser. It also displays the maximum execution time (Duration) in minutes and seconds and the test run result (Status) in each browser.
Such a report allows for a better understanding of which browsers encountered problems, specifically: where the most tests failed or the execution time sharply increased.
The stat-reporter plugin also allows generating reports in HTML or JSON formats. See details in the plugin description.
However, in terms of capabilities, the stat-reporter plugin is significantly inferior to the html-reporter plugin, which provides much more advanced ways to work with tests and their run results.
html-reporter Plugin Report
Add the html-reporter plugin to your project to get a graphical HTML report with the results of all test runs. Additionally, in the generated report, you will be able to:
- filter tests by completion status;
- group tests by errors or any parameter from the test metadata;
- view screenshot differences in 6 different ways;
- view all retries or errors in tests separately.
Moreover, the html-reporter allows running Testplane in a special GUI mode. In this mode, you can run and rerun tests, reshoot screenshots, use special debugging modes, and much more.
Keywords
- stat-reporter
- html-reporter
- gui