1. Docs
  2. Build tests
  3. Automate
  4. How the numbers work

How the numbers work

Every number, icon and colour in Automate, how it is calculated, and an example you can check against your own project.

5 min read Updated CustomersAdmins

In short

On this page you can

  • Read what the last-run icon in the tree really reflects.
  • Understand the ×n badge on templates and the action usage chip.
  • Check how a folder’s passed, failing and never-run counts are made.
  • See why dry-run progress may stop below 100%.

How to use it

  1. Find the number on screen in the headings below.
  2. Read its formula.
  3. Compare the worked example with your own project.
  4. Follow the link to the page where you can change it.

Status icon in the tree#

Shown in: test case tree, before each test case; tooltip “Last run: …” with the date
source       = the latest PIPELINE run of the test case (dry runs never change it)
Failed       = any step of that run ended FAILED or ERROR
Success      = every step SUCCESS, or SUCCESS plus WARNING
Running      = the run has not finished
Disconnected = the run stayed Running for more than 2 hours (connection to the runner lost)
Never run    = no pipeline run yet

Example: a pipeline runs Checkout with voucher. Steps 1 to 9 pass; step 10, a network step with the rule “warn”, ends WARNING. The icon shows Success. The next night step 4 cannot find the voucher field: the icon shows Failed. You fix the locator and dry run it green: the icon still shows Failed until the pipeline runs again.

The tooltips also exist for Skipped (“Last run: Skipped”) and Passed with warnings (“Last run: Passed with warnings”). A run with warnings is stored as Success at test case level, so in practice you see Success.

The Filter By Status (Last Run) filter uses the same value. See Find test cases.

Template badge#

Shown in: test case tree, after a template
×n     = number of distinct test cases, not archived, with a step that calls this template
unused = n is 0
hover  = “Used by n test cases” and up to 8 names, then “+k more”

Example: Login as standard user is a step in 14 test cases; one of them calls it twice and one is archived. The badge shows ×13: the case that calls it twice counts once and the archived one not at all. Hovering lists 8 names and “+5 more”. Before you archive or change a template, check this number: archiving is blocked while it is above 0. See Archive and restore.

Action usage chip#

Shown in: right panel when a step is selected: “This action is used in n test cases”
n      = number of distinct test cases with a step that uses this action directly
amber  = n > 1   (a change to the action affects other test cases)
hidden = for steps that have their own action (code, AI, API, database)

Example: step 2 of your test case is Click Sign In. The chip says “This action is used in 31 test cases”, in amber. Renaming the step or editing its locators in Action Detail changes all 31. For a change only this test needs, use Save as Copy. Click the chip to list the 31 test cases. Tests that reach the action only through a template are not counted, the same as the Cases badge in Discover.

Folder summary#

Shown in: the editor when a folder (or nothing) is selected
test cases   = test cases in the folder and its sub folders, templates excluded
failing      = test cases whose last run is failed, error or disconnected
never run    = test cases with no run status and no trace in run logs or pipelines
passed       = test cases − failing − never run
+n templates = templates in the folder (not counted above)

condition coverage = Σ min(covered, target) ÷ Σ target
                     over the pages this folder's test cases use, that have a target
colour             = ≥ 80% green · 40–79% amber · < 40% red

Example: folder Orders holds 12 test cases and 2 templates. 2 failed in their last run, 1 is disconnected, 3 never ran. It shows “12 test cases”, “6 passed”, “3 failing”, “3 never run” and “+2 templates (not counted)”.

Its test cases use three pages: cart with 4 of 4 conditions covered, checkout with 5 of 7, order_detail with 2 of 9. Condition coverage = (4 + 5 + 2) ÷ (4 + 7 + 9) = 11 ÷ 20 = 55%, amber.

A page’s conditions are its test points from the App Map (tasks, validation actions and transitions); see App Map · Tested. Without an App Map update the summary says “Condition coverage not measured”.

Likely duplicates: two or more test cases whose names are equal once letters and digits are kept and case is ignored (names of at least 4 such characters) form a group; up to 20 groups are listed.

Dry-run progress#

Shown in: the Test Run panel
rows                  = active steps + 1 row “Open <start page>” (web)
Progress: p%          = rows that ended SUCCESS, WARNING or ERROR ÷ rows × 100
c / t steps completed = rows that ended SUCCESS ÷ rows

Example: a web test case has 10 active steps, so there are 11 rows. The start page and steps 1 to 5 pass, step 6 fails and steps 7 to 10 are marked SKIPPED. Progress = (6 + 1) ÷ 11 = 64%, and “6 / 11 steps completed”. A failed run never reaches 100%, because skipped rows do not count.

Deactivated steps are not sent to the run and are not rows. A template counts as its steps.

Recent runs#

Shown in: Test Case Detail › Recent runs
list     = the last 10 pipeline runs of this test case, newest first (3 visible, the rest scroll)
attempts = shown when one run tried the test case more than once (retries or data rows)

Example: the nightly pipeline has a retry count of 2. Last night the test case failed once and passed on the second try: the row shows the run’s date, its duration and “2 attempts”. Dry runs are not listed.

Other indicators#

IndicatorHow it is decided
“n steps” on a template stepNumber of steps in the template.
Step number in the listThe step’s position, 1 to n, renumbered after every drag, add or remove. Inactive steps keep their number.
“n issue(s) found” on SaveNumber of invalid step values; up to five are listed.
Archive preview“n folder(s)”, “n test case(s)”, “n manual case(s)” inside the item, and the pipelines it will be removed from.
“n scheduler(s) updated. Added: a, Deleted: d”Pipelines synced after a parameter change, and the parameters added to and removed from them in total.
Page list in Generate Test Cases with AIpage (n): number of recorded actions on that page.
Dot on the filter iconAt least one tree filter is active.

Didn’t find your answer?Write to support with your project name and what you see on screen.

info@smartestqa.ai
© 2026 Selenity Technology · SmartestQAsmartestqa.aiThis page as MarkdownChecked against smarttest-fe predev 2a8c79cc, smarttest-be 633c2a03, research/automate-editor.md, research/automate-steps-and-runs.md