1. Docs
  2. Build tests
  3. Maintenance
  4. FAQ, MCP & limits

FAQ, MCP & limits

Answers to common questions about Maintain, how AI assistants decide repairs through MCP, and the module’s limits.

5 min read Updated CustomersAdmins

In short

On this page you can

  • Find out whether a repair changes your test before anyone approves it.
  • Learn what works without AI and what the Auto-Heal setting turns off.
  • Decide repairs from an AI assistant with smartest_heal.
  • Show self-healing to a prospect in five minutes.
  • Check the fixed limits: 3 runs, score 85, lead 15, 30 candidates.

How to use it

  1. Filter the questions by Customers or Admins.
  2. Open a question for its answer and links.
  3. For MCP, connect an assistant, then ask it to list repairs waiting for review.
  4. Approve or reject through the assistant only after it showed you the preview.

Questions and answers#

Does a repair change my test before anyone approves it?Customers

Yes, for locators. The new locator is written to the action as soon as the repair passes its checks, so the next runs use it and do not wait on the broken one again. The review comes afterwards: Reject puts the locators back exactly. Assertion repairs change nothing in the test.

What happens if nobody reviews the repairs?Customers

A locator repair whose new locator keeps working is confirmed by runs after 3 more runs. Assertion repairs and desktop repairs stay Waiting for review, the test cases keep the healed tag, and InSight keeps showing the line.

Can auto-heal pick the wrong element?Customers

It is designed not to. It refuses when the app looks broken, accepts an AI choice only at score 85 with a 15-point lead, requires a locator that matches only that element, and checks that the step did what it did on the last passing run. Still, review AI repairs in Details; that is why every repair waits for you. See the stages.

Will auto-heal hide real bugs?Customers

No. Server errors, error pages and ended sessions fail the step without a repair. Assertions pass only when the texts mean the same value: 50 TL and 55 TL still fail. Every repair is listed for review.

Does auto-heal work without AI?Customers

Partly. In web runs the exact identity match and the whitespace rule for assertions work without AI. Choosing among candidates, comparing screenshots, comparing formatted assertion texts and desktop repair need Enable AI and Auto-Heal. See Settings.

Can I turn auto-heal off completely?Admins

You can turn off the AI parts with Auto-Heal in Project Panel or per pipeline. The exact identity match and the whitespace rule always run in web runs; there is no setting for them in the app. Reject any repair you do not want.

Does a Dry Run change my locators?Customers

Yes. A repair made in a Dry Run is saved to the action and appears in Auto-heal like one from a pipeline. See Dry Run and pipeline runs.

Why are there no repairs in my mobile project?Customers

The runner does not repair Android or iOS steps today. Give important actions a second locator in Discover, preferably accessibility_id or resource-id.

Can I undo an approval or a rejection?Customers

No, decisions are final. To change the locators afterwards, edit the action in Discover › Action List.

Who can approve and reject, and is it logged?Admins

Admins and Users, in the projects they can open. The card shows who decided and when, and each decision is written to the company audit log under Auto-heal. See Who can decide.

What does auto-heal send to the AI?Admins

Only at the AI stage: a screenshot of the page with numbered boxes, the candidates’ tags, roles, names and nearby text, the step name and type, the old locators and the action’s recorded screenshot. For assertions, the two texts. Values typed into fields are never stored in the element identity. Run tests on a test environment with test data (see What is sent to the AI).

Does auto-heal call the AI on every run?Admins

No. For a locator, the AI is called only when all locators of a step failed and no exact match was found: once to choose among the candidates and at most once more to compare screenshots. For an assertion, only when the texts differ by more than whitespace. Within one pipeline run a repaired action is not repaired again. AI steps, in contrast, call the AI on every run, which is why converting them saves time.

Can I undo converting an AI step?Customers

There is no undo. The AI step was removed from the test case. Add a new AI step in Automate with the same instruction if you need it back; the action stays in the Action List.

Can I load test the calls I captured?Customers

Yes, in two steps: make an API test here, then generate a JMeter test from that test case in Automate.

For AI assistants (MCP)#

An assistant connected through the SmartestQA MCP server works with repairs through smartest_heal. Connect one from Device Farm › Test Agents › Connect to Claude.

ActionWhat it does
summaryHow many repairs wait, on how many test cases.
listRepairs by status (default PENDING; also APPROVED, REJECTED, PROMOTED, ALL), optionally for one testCaseId, up to 100 per call (default 20).
approve / rejectOne recordId at a time. The first call only returns a preview with the old and new locator and the effect. The assistant must show it to you and repeat the call with confirm=true after you agree. No bulk decisions.
Example prompts
List the auto-heal repairs waiting for review in project "Customer Portal" and tell me which ones the AI chose.

Show me repair 1482 and explain whether the new locator points at the same button.

Reject repair 1482, then suggest a stable locator for that action from the code in /web/src/checkout.

Run failures and run history are read with smartest_runs; the assistant can fix a locator with smartest_action_write.

Limits#

ItemLimit
Runs until a locator repair confirms itself3 (one per run)
AI acceptancescore ≥ 85 and lead ≥ 15
Candidates sent to the AI · shown in Details30 · 5
Desktop acceptanceconfidence ≥ 85, exactly one match
Waiting locator repairs per action1 (a newer one replaces it)
Assertion types that can be repairedEQUALS, CONTAINS, STARTS_WITH, ENDS_WITH (text)
Project typesWEB (all), DESKTOP (locators, AI only), MOBILE (none)
Playwright engineMain page only, no frames
Repair cards per page10 · 20 · 50
MCP list per call100
Action name from an AI step60 characters
Bulk approve, reject or convertNot available

Didn’t find your answer?Write to support with your project name and what you see on screen.

info@smartestqa.ai
© 2026 Selenity Technology · SmartestQAsmartestqa.aiThis page as MarkdownChecked against smarttest-fe predev 2a8c79cc, smarttest-be 633c2a03, smarttest-testsuite 017fcec, smartest-mcp-server b95acc0 (src/next/heal.ts, src/autoheal.ts)