How the numbers work
Every number, icon and colour in Automate, how it is calculated, and an example you can check against your own project.
In short
On this page you can
- Read what the last-run icon in the tree really reflects.
- Understand the ×n badge on templates and the action usage chip.
- Check how a folder’s passed, failing and never-run counts are made.
- See why dry-run progress may stop below 100%.
How to use it
- Find the number on screen in the headings below.
- Read its formula.
- Compare the worked example with your own project.
- Follow the link to the page where you can change it.
Status icon in the tree#
source = the latest PIPELINE run of the test case (dry runs never change it) Failed = any step of that run ended FAILED or ERROR Success = every step SUCCESS, or SUCCESS plus WARNING Running = the run has not finished Disconnected = the run stayed Running for more than 2 hours (connection to the runner lost) Never run = no pipeline run yet
Example: a pipeline runs Checkout with voucher. Steps 1 to 9 pass; step 10, a network step with the rule “warn”, ends WARNING. The icon shows Success. The next night step 4 cannot find the voucher field: the icon shows Failed. You fix the locator and dry run it green: the icon still shows Failed until the pipeline runs again.
The tooltips also exist for Skipped (“Last run: Skipped”) and Passed with warnings (“Last run: Passed with warnings”). A run with warnings is stored as Success at test case level, so in practice you see Success.
The Filter By Status (Last Run) filter uses the same value. See Find test cases.
Template badge#
×n = number of distinct test cases, not archived, with a step that calls this template unused = n is 0 hover = “Used by n test cases” and up to 8 names, then “+k more”
Example: Login as standard user is a step in 14 test cases; one of them calls it twice and one is archived. The badge shows ×13: the case that calls it twice counts once and the archived one not at all. Hovering lists 8 names and “+5 more”. Before you archive or change a template, check this number: archiving is blocked while it is above 0. See Archive and restore.
Action usage chip#
n = number of distinct test cases with a step that uses this action directly amber = n > 1 (a change to the action affects other test cases) hidden = for steps that have their own action (code, AI, API, database)
Example: step 2 of your test case is Click Sign In. The chip says “This action is used in 31 test cases”, in amber. Renaming the step or editing its locators in Action Detail changes all 31. For a change only this test needs, use Save as Copy. Click the chip to list the 31 test cases. Tests that reach the action only through a template are not counted, the same as the Cases badge in Discover.
Folder summary#
test cases = test cases in the folder and its sub folders, templates excluded
failing = test cases whose last run is failed, error or disconnected
never run = test cases with no run status and no trace in run logs or pipelines
passed = test cases − failing − never run
+n templates = templates in the folder (not counted above)
condition coverage = Σ min(covered, target) ÷ Σ target
over the pages this folder's test cases use, that have a target
colour = ≥ 80% green · 40–79% amber · < 40% red
Example: folder Orders holds 12 test cases and 2 templates. 2 failed in their last run, 1 is disconnected, 3 never ran. It shows “12 test cases”, “6 passed”, “3 failing”, “3 never run” and “+2 templates (not counted)”.
Its test cases use three pages: cart with 4 of 4 conditions covered, checkout with 5 of 7, order_detail with 2 of 9. Condition coverage = (4 + 5 + 2) ÷ (4 + 7 + 9) = 11 ÷ 20 = 55%, amber.
A page’s conditions are its test points from the App Map (tasks, validation actions and transitions); see App Map · Tested. Without an App Map update the summary says “Condition coverage not measured”.
Likely duplicates: two or more test cases whose names are equal once letters and digits are kept and case is ignored (names of at least 4 such characters) form a group; up to 20 groups are listed.
Dry-run progress#
rows = active steps + 1 row “Open <start page>” (web) Progress: p% = rows that ended SUCCESS, WARNING or ERROR ÷ rows × 100 c / t steps completed = rows that ended SUCCESS ÷ rows
Example: a web test case has 10 active steps, so there are 11 rows. The start page and steps 1 to 5 pass, step 6 fails and steps 7 to 10 are marked SKIPPED. Progress = (6 + 1) ÷ 11 = 64%, and “6 / 11 steps completed”. A failed run never reaches 100%, because skipped rows do not count.
Deactivated steps are not sent to the run and are not rows. A template counts as its steps.
Recent runs#
list = the last 10 pipeline runs of this test case, newest first (3 visible, the rest scroll) attempts = shown when one run tried the test case more than once (retries or data rows)
Example: the nightly pipeline has a retry count of 2. Last night the test case failed once and passed on the second try: the row shows the run’s date, its duration and “2 attempts”. Dry runs are not listed.
Other indicators#
| Indicator | How it is decided |
|---|---|
| “n steps” on a template step | Number of steps in the template. |
| Step number in the list | The step’s position, 1 to n, renumbered after every drag, add or remove. Inactive steps keep their number. |
| “n issue(s) found” on Save | Number of invalid step values; up to five are listed. |
| Archive preview | “n folder(s)”, “n test case(s)”, “n manual case(s)” inside the item, and the pipelines it will be removed from. |
| “n scheduler(s) updated. Added: a, Deleted: d” | Pipelines synced after a parameter change, and the parameters added to and removed from them in total. |
| Page list in Generate Test Cases with AI | page (n): number of recorded actions on that page. |
| Dot on the filter icon | At least one tree filter is active. |
Didn’t find your answer?Write to support with your project name and what you see on screen.
info@smartestqa.ai