How the numbers work
Every number and status in Schedule, how it is calculated, and an example you can check against your own pipelines.
In short
On this page you can
- Check how many case runs a pipeline run should have before you start it.
- Read History’s Result as passed case runs over planned case runs.
- Understand why a run is Failed or Disconnected.
- Read the queue message: your company’s runs ahead and the pool’s slots.
- Read a queue’s Success 24h, Failed 24h and over-SLA counts.
How to use it
- Count your Test, Setup and Teardown rows, then add Before and After times the Test rows.
- Multiply by repeat counts, data rows and requirement cases.
- Compare with the denominator in History: they should match.
- If they do not, look for stopped or disconnected case runs.
Items badge (Pipelines list)#
n = rows in the run list
= test cases + requirements + sub-pipelines (one row each, whatever it contains)
Example: a pipeline has a Setup case, three test cases and one sub-pipeline with eight cases inside. The badge shows 5 items. Hover it for the Run order. The number of case runs is calculated differently, below.
Planned case runs#
units(row) = repeat count of the row (1 unless Data & loop sets more)
× test cases linked to it, for a requirement row
× planned case runs of the sub-pipeline, for a sub-pipeline row
tests = Σ units of Test rows that are test cases or requirements
total = Σ units of Test, Setup and Teardown rows (sub-pipelines included)
+ (Σ units of Before and After rows) × tests
data loop = a repeat of "first N rows" counts min(N, rows in the dataset)
| Pipeline | Calculation | Planned |
|---|---|---|
| 3 test cases | 3 | 3 |
| Setup, Before, 3 Test, After, Teardown (the phase example) | (1 + 3 + 1) + (1 + 1) × 3 | 11 |
| 1 requirement with 4 linked cases, looped over the first 5 rows | 4 × 5 | 20 |
| The same, looped over “first 10 rows” of a 7-row dataset | 4 × min(10, 7) | 28 |
| 2 test cases + a sub-pipeline that plans 6 | 2 + 6 | 8 |
| Before + After, no Test row | cannot run | — |
Sub-pipelines are not wrapped by Before and After and do not count as tests. Setup, Before, After and Teardown are never repeated by the repeat count.
History Result#
Result = passed / planned
passed = case runs that ended as Completed (passed)
planned = planned case runs (above)
colour = grey if planned = 0
green if passed = planned
light green if passed ÷ planned ≥ 80%
orange if ≥ 50%
red below 50%
| Result | Share | Colour |
|---|---|---|
| 11 / 11 | 100% | Green |
| 9 / 11 | 82% | Light green |
| 6 / 11 | 55% | Orange |
| 3 / 11 | 27% | Red |
Behind the scenes a run also counts failed = failed + disconnected case runs + cases that never ran because the run was stopped, and pending = planned − finished − running. A stopped run therefore keeps its planned total: “5 planned, 3 ran” stays visible as 3 / 5, not 3 / 3.
Last run status#
Running while any case run of the latest run has not finished
then, in this order:
Disconnected if any case run of the run is Disconnected
Failed if any case has a failed or error step,
or the run was stopped and some cases never ran
Success otherwise (steps with only warnings count as passed; shown as "Completed")
Example: 11 case runs, 10 pass, one has an assertion error in step 7: Failed. Same run, but the agent loses its connection during case 9: Disconnected, even if the other ten passed. You stop a run after 4 of 11 cases: Failed.
When a case run becomes Disconnected#
quiet case = a case run still Running with no update for 3 minutes
→ set to Disconnected when another case of the same run reports in
long case = a case run still Running more than 2 hours after it started
→ set to Disconnected by a background check that runs every 4 hours
server lost = the SmartestQA server running it sends no heartbeat for 120 s
→ the run is not restarted; it ends as Disconnected
Example: a nightly run starts at 01:00 on a laptop agent. At 01:05 the laptop sleeps during case 3; no other case reports. The background check at 04:00 sees case 3 running since 01:05, more than two hours, and sets it to Disconnected; the pipeline shows Disconnected from then on. Use an always-on machine for night runs.
Thread Count and Retry Count limits#
Thread Count ≤ your licence's parallel threads (a higher value is cut back when saved) new project Thread Count = min(licence's parallel threads, 3) Retry Count 0 … 5 (a higher value is cut back to 5)
Example: if your company’s licence allows 2 parallel threads, a new project starts with Thread Count 2, and a value of 4 is cut back to 2 when you save. If it allows more than 3, a new project starts with 3 and you can raise it up to the licence’s value.
An Admin sees the licence in ConfigurationCompany SettingsBilling (Billing).
Server queue message#
pool used/slots = slots running now / slots of the pool your run goes to
(your dedicated servers if you have them, otherwise the shared pool)
n run(s) ahead = runs of YOUR company that will get a slot before this one
order 1. source: sub-pipeline > started by hand > CI/CD API > schedule > RPA queue
2. among equals: the company with the fewest runs going now
3. then the oldest first
"Now: … q of your runs waiting" = your company's runs waiting at this moment
Example: the shared pool has 4 servers with 2 slots each: 8 slots, all busy. Your company already has two runs waiting, both started by hand. You start a third by hand: “Queued · 2 run(s) of your company ahead · pool 8/8”. A colleague then starts one from CI/CD: it waits behind your three, because hand-started runs go first. A scheduled run of your company that was already waiting falls behind both. Other companies’ runs are never counted in “ahead”; fair share decides between companies.
Next run under the cron field#
from = Start Date Time if it is in the future, otherwise now
next run = the first whole minute after "from" that matches all five fields,
searched up to 400 days ahead, and not after End Date Time
clock = your browser's (the real start uses the server's clock: see Time zone)
Example: 30 2 * * 1-5, Start Date Time in the past, it is Friday 14:10. Next matching minute: Monday 02:30. The summary reads “Weekdays, at 02:30 · Next run: 16 Nov, 02:30”. Compare it with Next run on Scheduled Runs, which comes from the server (Time zone).
Data & loop count#
n = Σ over datasets ( rows taken per run × Test rows that read the dataset ) + Σ over other Test rows ( their repeat count ) rows taken per run = all rows, or N when you choose "first N" or "N random" (at most the rows there are)
Example: dataset customers has 12 rows. You take the first 5, in order. Three test cases read it; two others run once. 5 × 3 + 2 = 17 case runs. Setup, Before, After and Teardown come on top as in Planned case runs.
Queue counts#
Pending, In progress = items in that status now Success 24h = items that finished as Success in the last 24 hours Failed 24h = items that finished as Failed in the last 24 hours n over SLA = items still Pending, In progress or Deferred that were created more than SLA minutes ago "on arrival · n agents" / "every m min · n agents" = Run mode, Interval and Max parallel
Example: SLA 240 minutes. An item created at 08:00 is still Pending at 12:30: it counts in “1 over SLA”. Once it succeeds at 12:40 it leaves that count and adds one to Success 24h until 12:40 the next day.
Other indicators#
| Indicator | How it is decided |
|---|---|
| Run list (n) on the pipeline page | Rows in the run list, as the items badge |
| Status icon on a run-list row | The last run status of that test case, wherever it last ran |
| Stopping… / “1/2 stopping” | Stop requested for that many of the pipeline’s running runs |
| Ended on Scheduled Runs | End Date Time is in the past |
| “n records” in History | Runs in the selected date range |
| Agent dot in Test Agent List | Green online, red offline (90-second rule) |
Didn’t find your answer?Write to support with your project name and what you see on screen.
info@smartestqa.ai