1. Docs
  2. Use cases
  3. Use cases
  4. What happens to my tests when the UI changes?

What happens to my tests when the UI changes?

When a control is renamed or moved and its locator no longer finds it, the runner looks for the same element again, checks that the step still did its job and saves the new locator, so the run goes on. Every repair then waits in Maintain › Auto-heal, where you approve or reject it.

8 min read Updated CustomersAdmins

In short

On this page you can

  • Keep a run going when a locator breaks after a UI change.
  • See every repair as a card: old locator, new locator, how it was found and checked.
  • Approve a repair to keep it, or reject it to put the action back.
  • Let a repair confirm itself after 3 more successful runs.
  • Find the test cases that ran on a repaired locator with the healed tag.

How to use it

  1. Run your pipeline as usual; healing starts by itself when a step cannot find its element.
  2. Open Maintain › Auto-heal; the filter starts on Waiting for review.
  3. Read the change on the card and open Details for the evidence.
  4. Click Approve or Reject and confirm.

The short answer#

A test step finds its control through the locators of an action. When the application changes and none of them finds the control any more, the runner does not fail the step at once. It first checks that the application itself is not broken, then looks for the same element by its identity, and only then asks the AI. A repair is saved only when the step still does what it did on the last successful run.

Nothing is hidden: each repair becomes a card in Maintain › Auto-heal and the test case gets a healed tag until someone decides.

The problem this solves#

UI tests find controls by locators, and locators break when the application changes. A developer renames an id or moves a button, the application still works, and the next morning a block of tests is red. Someone then opens each failing test, finds the control again and edits the locator.

The usual defences have a cost. Scripts with several fallback locators keep running, but nobody sees which fallback was used or whether it clicked the right element.

An example#

This is an illustration, not a customer story. A team tests an HR portal in the browser, and a developer changes the id of the button that sends a leave request. Its role and its name stay the same.

  1. In the night’s pipeline the step that clicks the button tries every locator of its action with the full wait time. None finds the button.
  2. The runner checks the page first. It shows no error, so healing goes on. Exactly one visible element carries the identity recorded on the last successful run, so it wins as an Exact match. No AI is asked.
  3. The effect is checked: the click made the page react like on the last successful run. The run log says “auto-heal saved the new locator”, the run goes on, and later test cases of the same run use the repaired locator at once.
  4. In the morning a tester opens Maintain › Auto-heal. Under Waiting for review there is one card with the old locator struck through and the new one, and the test case carries the healed tag.
  5. The tester opens Details, reads the change, the method and how it was checked, clicks Approve and confirms. The message is “Repair approved”.

The run went on after a change that broke nothing, and the repair still waited for a person to decide.

By hand, with fallback locators in your own scripts, and with SmartestQA#

QuestionFixing locators by handYour own scripts with fallback locatorsSmartestQA
What happens in the run when a locator breaks?The step failsThe next locator in the list is triedThe runner looks for the same element by its recorded identity, then asks the AI if AI auto-heal is on. The AI’s choice needs a score of at least 85 and a lead of at least 15 points
Can a repair hide a real bug?No, a person looks at every failureYes, if a fallback matches another elementThe runner does not heal when the page or the application’s API returned an error, and the step’s effect is checked before a repair is saved
Who sees what was changed?The person who edited it, and the repository historyUsually nobody, unless the script logs itEach repair is a card in Maintain › Auto-heal and waits for review
Can it be undone?Yes, from the repository historyThere is nothing to undo; the script is unchangedReject puts the action’s locators back exactly as they were
What if nobody reviews?The test stays redNothing; the fallback stays in useA locator repair stays in use and is confirmed after 3 more successful runs
Where does it apply?EverywhereWherever you write itWeb projects on Selenium and Playwright get every stage; desktop projects get AI repairs only; mobile projects have no locator repair

What the runner does, and what you do#

  1. The step’s own search fails

    Healing starts only after every locator of the action was tried with the full wait time. If the element was only slow and one of the action’s own locators now finds it, the runner uses it and changes nothing.

  2. The page is checked first

    The runner does not heal when the application itself looks broken: the page returned HTTP 500 or higher, or 401 / 403; the application’s API returned a server error in the last 20 seconds; the page shows an error message; or the session ended on a login page. The step fails, because healing to another element would hide a real bug.

  3. Exact match

    On every successful run the runner records the identity of the step’s element, such as its data-testid, a stable id, name, and its role with its accessible name. If exactly one visible element carries that identity, it wins. This needs no AI.

  4. AI choice, only when it is sure

    Without an exact match, and with AI auto-heal on, the AI scores the candidates on the page from 0 to 100. The best one wins only if it scores at least 85 and leads the second best by at least 15 points. Otherwise the step fails with the reason.

  5. The result is checked

    A wrong element can react too, so the step’s effect is checked before anything is saved: a typed field holds the value, a selection is set, a click made the page react like on the last successful run. A failed check fails the step and saves nothing.

    The run log says “auto-heal saved the new locator”. The run goes on, and later test cases of the same run use the repaired locator at once.

  6. Review the repair

    Open Maintain › Auto-heal. The filter starts on Waiting for review. Each card names the test case and step and shows the old locator struck through and the new one. Details shows the whole change, the method and how it was checked.

  7. Decide

    Click Approve to keep the new locator on the action, or Reject to put the action’s locators back exactly as they were. Not sure? Leave it: the repair stays in use and is confirmed by itself after 3 more successful runs.

Watch it#

The video shows the review side: the Auto-heal list, the status filter, a repair card, Details, and what Approve and Reject do. It does not show the run in which the repair was made.

Video transcript

Auto-heal lists the repairs the runner made during test runs. The status row filters them: waiting for review, confirmed by runs, approved or rejected. Each card names the test case and step, and shows what was expected and what the run found. Details shows the whole change and how it was checked. Approve keeps the repair. Reject puts the action back as it was. Left alone, a repair is approved automatically after three more successful runs.

What the runner fixed during a run, and how to decide.

What you get#

  • Runs that survive a renamed or moved control, without a person editing the locator first.
  • No silent changes. Each repair is a card with one of five statuses: Waiting for review, Confirmed by runs, Approved, Rejected or Replaced by a newer repair.
  • Evidence per repair. The method (Exact match or the AI with its score), how the result was checked, and for AI repairs up to 5 candidates the AI compared.
  • A clear threshold. The AI’s choice is used only at a score of 85 or more with a lead of 15 points.
  • An undo. Reject restores the action’s locators with their values, last-run status and confidence.
  • A trace. Each decision stores who decided and when, and is written to the company audit log.
  • Text assertions that differ only in format can pass too, for example when only spaces or line breaks differ; they get a card of their own and the expected value is not changed.

Questions and answers#

Can self-healing hide a real bug?Customers

It is built not to. The runner refuses to heal when the page or the application’s API returned an error, the AI must be sure, and the step’s effect is checked before a repair is saved. Every repair still waits for review. See The stages.

Does auto-heal need AI?Customers

Not always. An exact match on the element’s recorded identity is a yes-or-no match with no AI. The AI is asked only when there is no exact match and AI auto-heal is on. See Settings.

What happens if nobody reviews a repair?Customers

The repair stays in use. A locator repair moves to Confirmed by runs after the new locator has found its element in 3 more runs. Assertion repairs and desktop repairs never confirm themselves. See Confirmed by runs.

What happens when I reject a repair?Customers

The action’s locators go back to what they were before the repair, so the next run fails on this step until the test is fixed. Add a locator that finds the right control before the next run. See What Reject changes.

Which project types does it work for?Customers

Web projects on Selenium and Playwright get every stage. Desktop projects get AI repairs only, and you decide each one yourself. Mobile projects have no locator repair. See Project types and engines.

How do I see which tests ran on a repaired locator?Customers

While a test case has a repair waiting, it carries the healed tag; filter by it in Automate. InSight shows the same queue as one line with Review. See The healed tag.

Does a Dry Run heal too?Customers

Yes. A Dry Run heals locators and assertions, saves the repaired locator to the action and creates a card, like a pipeline run. See Dry Run and pipeline runs.

Read more#

Didn’t find your answer?Write to support with your project name and what you see on screen.

info@smartestqa.ai