- Docs
- Build tests
- Maintenance
- How the numbers work
How the numbers work
Every number in Maintain, how it is calculated, and an example you can check against your own project.
In short
On this page you can
- Read the counts on the status filters and see why All is larger than their sum.
- Work out when a repair moves from 1/3 to Confirmed by runs.
- Check whether an AI score of 88 against 80 is accepted (it is not).
- Understand the n and m in the InSight auto-heal line.
- Read “n runs” and “c calls” in AI Steps and Network Calls.
How to use it
- Find the number on screen in the headings below.
- Read its formula in the grey box.
- Compare with the worked example and your own project.
Status filter counts#
count(status) = repairs of this project with that status
(repairs of deleted test cases are left out)
All = every repair, including "Replaced by a newer repair"
Example: the filter shows 4 Waiting for review, 10 Confirmed by runs, 3 Approved, 1 Rejected and 20 All. The four add up to 18; the other 2 are replaced repairs, which have no filter of their own.
A count is missing when it could not be loaded. Counts refresh after each decision.
Successful runs since the repair (1/3)#
n = runs since the repair in which the new locator found the step's element
(at most 1 per run, however many test cases use the action;
the run that made the repair does not count)
m = 3
n = m → status "Confirmed by runs", decided by "auto-heal"
| Run | What happens | Card |
|---|---|---|
| Monday nightly | The id of Save changed; the runner repairs the action | 0/3 |
| Tuesday nightly | 5 test cases use the action; the new locator finds the button | 1/3 |
| Tuesday, rerun of a pipeline | Found again | 2/3 |
| Wednesday nightly | Found again | Confirmed by runs |
If on Wednesday the new locator had broken and been repaired again, the card would become Replaced by a newer repair and a new card would start at 0/3. Counting happens in web runs. Assertion and desktop repairs show no bar and are never confirmed by runs.
AI score and the acceptance rule#
each candidate gets a score 0–100 from the AI (up to 30 candidates)
accepted ⇔ best ≥ 85 AND best − second best ≥ 15
(no second candidate → second = 0)
every candidate must be scored, otherwise the repair is refused
card shows "AI · best"; an exact match shows no score
| Best | Second | Check | Result |
|---|---|---|---|
| 96 | 40 | 96 ≥ 85, lead 56 | Accepted |
| 90 | 75 | 90 ≥ 85, lead 15 | Accepted |
| 88 | 80 | lead 8 < 15 | Refused: “two candidates are too close” |
| 80 | 20 | 80 < 85 | Refused: “AI is not sure enough” |
| 91 | — | 91 ≥ 85, lead 91 | Accepted |
In Details, the score bar of the chosen candidate is green; others are light blue from 50 and grey below. Up to 5 candidates are shown, best first. An accepted repair still has to pass the effect check before it is saved.
Desktop: the AI proposes one locator with a confidence; it is accepted at 85 or more when it matches exactly one element.
Locator confidence after a repair#
new locator confidence = AI score (exact match: 95; no score sent: 80)
last run = success
old locators confidence = 0, last run = failed → the new one is tried first
after Reject every locator gets back its earlier value, status and confidence
Example: Click Save had id=btn-save at 90%. An AI repair with score 92 adds css=[data-testid="save-order"] at 92% (green) and sets id=btn-save to 0% (red). Rejecting the repair removes the css locator and puts id=btn-save back at 90%. Colours are explained in Discover › Locator chip.
InSight: repairs waiting for review#
n = repairs with status "Waiting for review" (= the filter count) m = distinct test cases among those repairs (repairs without a test case are not counted in m) the line is hidden when n = 0
Example: 3 repairs on Checkout, 1 on Login and 1 desktop repair with no test case: “5 auto-heal repair(s) on 2 test case(s) wait for review.”
InSight: Auto-Heal Actions card#
count = actions of the project marked as auto-healed
marked: when a locator repair is saved
unmarked: only when a rejection leaves the action with no waiting repair
with a date range: only actions CREATED in that range
Why it differs from the waiting count: approved and confirmed repairs keep the action marked. 12 actions repaired over three months, 10 of them confirmed and 2 waiting, show 12 here and 2 in Waiting for review. With a date range, an action created a year ago and repaired yesterday is not counted.
AI Steps counts#
filter count = listed steps in that state (Pending, Can't convert, Converted) "n AI steps" = steps of this test case in the current filter "n runs" = saved results of this step in pipeline runs (passed or failed) date = the latest successful run
Example: step #4 Click the Pay button ran in 6 nightly runs and failed in 2 of them. The row shows the date of the latest pass and “6 runs”. Rows are ordered Pending first, then newest run first.
Network Calls counts#
s = Network steps in the test case now
c = Σ over these steps' latest saved result:
List mode → number of captured requests; Check / Extract → 1
n = distinct pipeline runs in which these steps saved a result
dialog: "n requests selected" = ticked rows in all filters = steps created
Fixed values#
| Value | Number |
|---|---|
| Runs until a repair confirms itself | 3 |
| AI acceptance: minimum score · minimum lead | 85 · 15 |
| Elements examined · candidates sent to the AI · candidates shown in Details | 800 · 30 · 5 |
| Look-back for the app’s own server errors (page check) | 20 s |
| Wait for the page to react in the effect check | up to 4 s |
| Repair cards per page | 10 · 20 (default) · 50 |
| Name of an action created from an AI step | 60 characters |
Didn’t find your answer?Write to support with your project name and what you see on screen.
info@smartestqa.ai