1. Docs
  2. Build tests
  3. Maintenance
  4. How auto-heal decides

How auto-heal decides

When every locator of a step fails, the runner tries to find the element again. It repairs the step only when it is sure, checks that the step still does what it did before, and then saves the repair for your review.

7 min read Updated CustomersAdmins

In short

On this page you can

  • Understand when the runner refuses to heal and lets the step fail as a defect.
  • See how an exact identity match differs from an AI choice.
  • Know what the effect check verifies before a repair is saved.
  • Learn which assertions can pass on a format-only difference.
  • Set Auto-Heal per project or per pipeline, and know what it controls.

How to use it

  1. Turn on Enable AI and Auto-Heal in Project Panel › AI Options.
  2. Run a test once successfully so the runner can record each element’s identity.
  3. When a locator breaks, read the run log for the stage that repaired it, or why not.
  4. Review the saved repair in Maintain › Auto-heal.
  5. Add a data-testid to elements that repair often, so the next match is exact.

When the runner heals#

Healing starts only after the step’s own search has failed completely: every locator of the action was tried with the full wait time, frames were searched, and the find was retried. A broken locator therefore always costs the full wait before healing starts.

  • The element was only slow. If one of the action’s own locators now finds exactly one visible element, the runner uses it and changes nothing. The log says “the element appeared late … nothing was changed.”
  • Same run, same action. Once a pipeline run has repaired an action, later test cases in that run use the repaired locator at once, without waiting again.

The stages#

  1. Page check: is the app itself broken?

    Healing to “some other element” would hide a real bug. The runner does not heal, and the step fails, when:

    • the page itself returned HTTP 500 or higher, or 401 / 403;
    • the application’s own API returned a server error (5xx) in the last 20 seconds;
    • a short page shows an error message such as “Internal Server Error” or “Something went wrong”;
    • the browser is on a login page with a password field, and the step is not about logging in (the session ended).
  2. Exact match: the same identity as last time

    On every successful run the runner records the identity of the step’s element: data-testid (or data-test, data-test-id, data-cy, data-qa), a stable id, name, and its role with its accessible name. If exactly one visible element on the page carries that identity, it is the winner. This is a yes-or-no match with no points, and it needs no AI.

    Ids that look generated (long numbers, UUIDs, prefixes such as mui-, radix-, ember) are ignored. For read steps (Get, Visibility Check) only the attributes count, because the visible name is the value being read.

  3. AI choice: only when it is sure

    Without an exact match, the AI decides, if AI auto-heal is on. The runner examines up to 800 elements, keeps up to 30 that suit the step type (a Type step needs an input), draws a numbered box on each in a screenshot, and sends it with the step, the old locators and the screenshot recorded with the action. The AI scores every candidate from 0 to 100.

    The best candidate wins only if it scores at least 85 and leads the second best by at least 15 points. Otherwise the step fails with the reason, for example “AI is not sure enough (best 72 < 85)” or “two candidates are too close (best 90, second 82; needs a lead of 15)”. Worked examples are in How the numbers work.

  4. New locator: unique and stable

    The runner builds a locator for the winner in this order of preference: data-testid, id, name, aria-label, placeholder, label, text, title, then absolute XPath. The locator must match exactly the chosen element and nothing else.

    If only an absolute, position-based XPath identifies the element, the repair is used for this run only and is never saved, because it breaks on the next layout change. The log says “step repaired for this run, but the locator was not saved”.

  5. Effect check: did the step still do its job?

    A wrong element can react too, so the result is checked before anything is saved:

    Step kindSaved when
    Type, Clear & TypeThe field now holds the value
    SelectThe selection changed or is set
    Click and other actionsWithin 4 seconds the page reacted like on the last successful run: the same page change, the same API call, a new tab or a dialog. With no record of the last run, any visible reaction counts, except that an AI-chosen element also needs the AI to see the expected result on before and after screenshots.
    Read steps (Get, Visibility Check, Hover)The step itself passes with the element

    When the rules are not met and AI is on, the AI compares the before and after screenshots as a last check. A failed check fails the step and saves nothing: “the repaired element did not behave like the original (…); nothing was saved”.

  6. Save for review

    The new locator is written to the action as a new locator marked as auto-healed. The action’s other locators are marked as failed with confidence 0, so the next run tries the new one first. A card appears in Auto-heal as Waiting for review and the test case gets the healed tag. The run log says “auto-heal saved the new locator”.

What you see in the run log#

Log message (short)MeaningCard?
“locator failed — auto-heal … is checking the page”Healing started—
“the element appeared late … nothing was changed”Slow page, no repair neededNo
“element found again via exact identity / AI …”A winner was chosen; the check comes nextNot yet
“auto-heal saved the new locator”Repair savedYes
“step repaired for this run, but the locator was not saved”Only a position-based XPath, or saving failedNo
“the step failed after auto-heal — the new locator was not saved”The step failed with the repaired elementNo
“auto-heal did not repair the step — …”Refused, with the reason (page check, no match, AI off, not sure enough, check failed)No

Each message is explained in Troubleshooting.

Assertions that differ only in format#

In web projects a failed text assertion can pass when the two texts mean the same value. This applies to EQUALS, CONTAINS, STARTS_WITH and ENDS_WITH compared as text. Numbers and booleans are already compared as numbers and booleans, so 50 and 50.0 never need it.

  1. Whitespace only (spaces at the ends, double spaces, line breaks, non-breaking spaces): the texts are equivalent. No AI is asked.
  2. Anything else: the AI decides EQUIVALENT or DIFFERENT, if AI auto-heal is on. Any doubt means DIFFERENT.
ExpectedActualResult
Order savedOrder saved Passes (whitespace)
50 TL50,00 TLPasses if the AI says equivalent
05.03.20262026-03-05Passes if the AI says equivalent
50 TL55 TLFails
anythingemptyFails: an empty side is never the same value

The step passes and its message adds “[AUTO-HEAL] Passed although the texts are not identical (…)”. The expected value is not changed. An assertion card appears in Auto-heal and the test case gets the healed tag. NOT_EQUALS, NOT_CONTAINS, regex and number comparisons are never healed.

Dry Run and pipeline runs#

Dry RunPipeline or scheduled run
Heals locators and assertionsYesYes
Saves a repaired locator to the actionYes, the change is permanentYes
Creates an Auto-heal cardYesYes
AI auto-heal setting usedThe project’sThe pipeline’s, if it sets one; otherwise the project’s
Feeds AI Steps and Network CallsNo, Dry Run results are not savedYes

Runs on the server and on your agent behave the same. After a Dry Run that repaired a step, the test case editor shows the action’s new locator.

Project types and engines#

ProjectLocator repairAssertion repairConfirmed by runs
WEB, SeleniumAll stagesYesYes
WEB, PlaywrightAll stages, elements in the main page onlyYesYes
DESKTOPAI only: the AI reads the UI tree and screenshot; accepted at confidence 85 or more with exactly one matchNoNo, decide yourself
MOBILE (Android, iOS)NoNo—

Desktop repairs appear as cards with the method Previous auto-heal engine. They are not linked to a test case, so the card shows “—” as the test case name and no healed tag is set. In MOBILE projects keep several locators per action in Discover, because a broken locator fails the step.

Settings#

SettingWhereControls
Enable AIConfigurationProject PanelAI OptionsThe master switch. Off means no AI anywhere in runs, including healing.
Auto-Heal (“Auto-heal locators”)Same page, under AI CapabilitiesThe AI stage of locator repair, the AI screenshot check, the AI assertion comparison, and desktop repair.
Auto-Heal per pipelineThe pipeline’s settings in ScheduleOverrides the project value for that pipeline’s runs.

What the switch does not control. The page check, the exact identity match, the effect check and the whitespace rule for assertions run in every web run. With AI off, a web step can still be repaired by an exact match. There is no setting in the app that turns these off. Project Panel is open to admins; see roles.

What is sent to the AI

For an AI choice: a screenshot of the page with numbered boxes, the candidates’ tags, roles, names and nearby text, the step name and type, the old locators, and the screenshot recorded with the action. For an assertion: the two texts and the step name. Values typed into fields are never stored in the element identity.

Make repairs rare#

  • Ask developers to add data-testid to key controls. It is the first thing an exact match looks for and the first choice for a new locator.
  • Keep a passing run on record: exact matches and the effect check compare with the last successful run of each step.
  • Give actions a second locator by hand where the app has one (Action Detail), so a single change does not break the step.
  • Watch the locator chip in the Action List after approving repairs.

Didn’t find your answer?Write to support with your project name and what you see on screen.

info@smartestqa.ai
© 2026 Selenity Technology · SmartestQAsmartestqa.aiThis page as MarkdownChecked against smarttest-testsuite 017fcec (business/auto_heal/heal_v2), smarttest-be 633c2a03, smarttest-fe predev 2a8c79cc