- Docs
- Build tests
- Maintenance
- FAQ, MCP & limits
FAQ, MCP & limits
Answers to common questions about Maintain, how AI assistants decide repairs through MCP, and the module’s limits.
In short
On this page you can
- Find out whether a repair changes your test before anyone approves it.
- Learn what works without AI and what the Auto-Heal setting turns off.
- Decide repairs from an AI assistant with
smartest_heal. - Show self-healing to a prospect in five minutes.
- Check the fixed limits: 3 runs, score 85, lead 15, 30 candidates.
How to use it
- Filter the questions by Customers or Admins.
- Open a question for its answer and links.
- For MCP, connect an assistant, then ask it to list repairs waiting for review.
- Approve or reject through the assistant only after it showed you the preview.
Questions and answers#
Does a repair change my test before anyone approves it?Customers
What happens if nobody reviews the repairs?Customers
A locator repair whose new locator keeps working is confirmed by runs after 3 more runs. Assertion repairs and desktop repairs stay Waiting for review, the test cases keep the healed tag, and InSight keeps showing the line.
Can auto-heal pick the wrong element?Customers
It is designed not to. It refuses when the app looks broken, accepts an AI choice only at score 85 with a 15-point lead, requires a locator that matches only that element, and checks that the step did what it did on the last passing run. Still, review AI repairs in Details; that is why every repair waits for you. See the stages.
Will auto-heal hide real bugs?Customers
No. Server errors, error pages and ended sessions fail the step without a repair. Assertions pass only when the texts mean the same value: 50 TL and 55 TL still fail. Every repair is listed for review.
Does auto-heal work without AI?Customers
Can I turn auto-heal off completely?Admins
You can turn off the AI parts with Auto-Heal in Project Panel or per pipeline. The exact identity match and the whitespace rule always run in web runs; there is no setting for them in the app. Reject any repair you do not want.
Does a Dry Run change my locators?Customers
Yes. A repair made in a Dry Run is saved to the action and appears in Auto-heal like one from a pipeline. See Dry Run and pipeline runs.
Why are there no repairs in my mobile project?Customers
The runner does not repair Android or iOS steps today. Give important actions a second locator in Discover, preferably accessibility_id or resource-id.
Can I undo an approval or a rejection?Customers
No, decisions are final. To change the locators afterwards, edit the action in Discover › Action List.
Who can approve and reject, and is it logged?Admins
Admins and Users, in the projects they can open. The card shows who decided and when, and each decision is written to the company audit log under Auto-heal. See Who can decide.
What does auto-heal send to the AI?Admins
Only at the AI stage: a screenshot of the page with numbered boxes, the candidates’ tags, roles, names and nearby text, the step name and type, the old locators and the action’s recorded screenshot. For assertions, the two texts. Values typed into fields are never stored in the element identity. Run tests on a test environment with test data (see What is sent to the AI).
Does auto-heal call the AI on every run?Admins
No. For a locator, the AI is called only when all locators of a step failed and no exact match was found: once to choose among the candidates and at most once more to compare screenshots. For an assertion, only when the texts differ by more than whitespace. Within one pipeline run a repaired action is not repaired again. AI steps, in contrast, call the AI on every run, which is why converting them saves time.
Can I undo converting an AI step?Customers
There is no undo. The AI step was removed from the test case. Add a new AI step in Automate with the same instruction if you need it back; the action stays in the Action List.
Can I load test the calls I captured?Customers
Yes, in two steps: make an API test here, then generate a JMeter test from that test case in Automate.
For AI assistants (MCP)#
An assistant connected through the SmartestQA MCP server works with repairs through smartest_heal. Connect one from Device Farm › Test Agents › Connect to Claude.
| Action | What it does |
|---|---|
summary | How many repairs wait, on how many test cases. |
list | Repairs by status (default PENDING; also APPROVED, REJECTED, PROMOTED, ALL), optionally for one testCaseId, up to 100 per call (default 20). |
approve / reject | One recordId at a time. The first call only returns a preview with the old and new locator and the effect. The assistant must show it to you and repeat the call with confirm=true after you agree. No bulk decisions. |
List the auto-heal repairs waiting for review in project "Customer Portal" and tell me which ones the AI chose.
Show me repair 1482 and explain whether the new locator points at the same button.
Reject repair 1482, then suggest a stable locator for that action from the code in /web/src/checkout.Run failures and run history are read with smartest_runs; the assistant can fix a locator with smartest_action_write.
Limits#
| Item | Limit |
|---|---|
| Runs until a locator repair confirms itself | 3 (one per run) |
| AI acceptance | score ≥ 85 and lead ≥ 15 |
| Candidates sent to the AI · shown in Details | 30 · 5 |
| Desktop acceptance | confidence ≥ 85, exactly one match |
| Waiting locator repairs per action | 1 (a newer one replaces it) |
| Assertion types that can be repaired | EQUALS, CONTAINS, STARTS_WITH, ENDS_WITH (text) |
| Project types | WEB (all), DESKTOP (locators, AI only), MOBILE (none) |
| Playwright engine | Main page only, no frames |
| Repair cards per page | 10 · 20 · 50 |
| MCP list per call | 100 |
| Action name from an AI step | 60 characters |
| Bulk approve, reject or convert | Not available |
Didn’t find your answer?Write to support with your project name and what you see on screen.
info@smartestqa.ai