1. Docs
  2. Build tests
  3. Automate
  4. FAQ, MCP & limits

FAQ, MCP & limits

Answers to common questions about Automate, how AI assistants use this module, and its limits.

5 min read Updated CustomersAdmins

In short

On this page you can

  • Find quick answers about copying, undo, templates, dry runs and roles.
  • Let an AI assistant write and dry-run test cases through MCP.
  • Look up Automate’s limits.

How to use it

  1. Filter the questions by your role with the chips.
  2. Open a question for the short answer and the link to details.
  3. Developers: connect an assistant and use the example prompts.

Questions and answers#

Can I copy a test case?Customers

Not in the screen. Put the shared steps in a template and add it to each new test case, use Expand Scenarios for variations, or ask an AI assistant through MCP to write a new case from an existing one.

Is there an undo?Customers

Before you save: leave the test case with Exit without saving. After deleting: restore from the Archive. There is no undo for a saved change, and no version history to roll back to; History shows who changed what and when.

Why don’t my dry runs show up in Report?Customers

Dry runs are for trying while you build; they write no report and do not change the tree’s status icon. Run the test case in a pipeline in Schedule to keep results. See Run modes.

Should I record, add steps by hand, or let AI write them?Customers

Record the first time you automate a flow: it creates the actions. After that, Generate from Prompt can reuse those actions for new test cases, and the toolbar adds assertions, API and data steps. Always end with a proof step. See Record or add by hand?

If I change a template, what happens to the tests that use it?Customers

They all use the new steps from their next run. A template is a live reference, not a copy. Check its ×n badge first to see how many tests you affect.

Why did renaming a step change other test cases?Customers

A step’s name is its action’s name, and actions are shared. Renaming it renames the action everywhere. Tests keep working, because they point at the action by ID. See Rename a step.

How long should a test case be?Customers

There is no fixed maximum. Aim for one user goal in 5 to 20 steps; put repeated parts in templates. Short tests fail at a step you can find quickly.

Can one test case run with many sets of data?Customers

Yes, in a pipeline. Read values from a dataset with {{excel:name}}_column and set the execute count to {{excel:name}}_#. A dry run uses the first row. See Run a test case once per data row.

Where do I put passwords and tokens?Customers

In a data source, referenced as {{ds:name}}. Values typed into steps are stored in the test case and visible to the whole project. See Secrets and personal data.

Do mobile and desktop test cases work the same way?Customers

Mostly. The tree, templates, variables and saving are the same. Differences: Record opens an inspector and its steps need Save; there is no start page (mobile uses the configured app, desktop a Target Application); dry runs go to a device or a desktop agent instead of a browser; the step types differ. See Record mobile and desktop steps.

Which browser engine runs my web tests?Customers

Selenium or Playwright, chosen per project in the project settings (Script Language). Most steps behave the same. Python steps keep separate code for each engine, and a dry run refuses to start when the code for the active engine is empty.

Can two people work on the same test case?Admins

Not safely at the same time: there is no locking and the last save wins. Split work by test case or folder. See Two people editing.

Who can do what in Automate?Admins
ActionUserAdmin
Create, edit, save, dry run test cases and foldersYes, in projects they can accessYes
Archive itemsOnly items they created entirelyYes
Archive a module, move a module under another folderNoYes
Restore from the ArchiveWhat they archivedYes
Delete for good, Empty archiveNoYes
Turn AI on (Project Panel › AI Options)NoYes

Roles in general: Roles: who can do what and Roles & permissions.

Does archiving a test case free up licence capacity?Admins

No. Your company’s licence can limit how many test cases you create, and its counter counts test cases created, not test cases that exist. An Admin sees the licence in Company Settings › Billing.

What does Automate send to the AI provider?Admins

Prompts, test case names and steps, action and page names, page descriptions and notes; for AI steps and AI Analyze, information about the current page. Secret-looking values are masked in description requests. See What is sent to the AI.

For AI assistants (MCP)#

An assistant connected through the SmartestQA MCP server can build and run test cases for you. Connect it from Device FarmTest AgentsConnect to Claude; see Connect an assistant and Discover › MCP.

ToolWhat it does
smartest_contextWhere the project stands and the next step.
smartest_case_rulesThe rules for writing steps, which the assistant reads before writing.
smartest_write_caseWrites a planned scenario’s steps and dry-runs it. A red run returns the failed step; after the third red run the scenario is marked blocked and the assistant asks you how to go on.
smartest_run_case, smartest_run_statusRun a written scenario again, unchanged; read the result of a dry run that was still running.
smartest_runs, smartest_healWhy a run went red and its history; healed locators.
smartest_propose, smartest_applyOther changes to records: a preview and a diff first, applied only after you approve.

The assistant dry-runs on your own agent unless you tell it to use the server. Desktop scenarios run only on an agent. More prompts: Example prompts.

Example prompts
In project "Customer Portal", write a test case for "reset password with an expired link" in module Accounts and dry run it until it passes.

Why did test case 1484 fail last night? Fix the locator if it is a locator problem.

Add an assertion that checks the order total to every test case in folder Checkout that has no proof step.

Limits#

ItemLimit
Steps per test caseNo fixed limit
Step name, folder name255 characters
Description1,000 characters
Step timeout1–300 seconds
Stored variable value8,000 characters
Wait for recorded web steps20 seconds
Template list page size5 · 15 · 25 · 100
Dry-run screenshots kept1 hour
Dry-run live log5,000 lines or 2 million characters per run, 30 minutes, 50 runs per project
One live-log message4,000 characters
Run on Server queue20 waiting runs
Browser kept open for AI Analyze (agent)15 minutes
Recent runs listed10
Expand Scenarios2–4 variations
Generate Test Cases with AIUp to 30 per request
AI descriptions8 test cases per AI call, 600 per update, 60 steps per case read

Questions not answered here: write to info@smartestqa.ai.

Didn’t find your answer?Write to support with your project name and what you see on screen.

info@smartestqa.ai
© 2026 Selenity Technology · SmartestQAsmartestqa.aiThis page as MarkdownChecked against smarttest-fe predev 2a8c79cc, smarttest-be 633c2a03, smarttest-testsuite runner, smartest-mcp-server, research/automate-editor.md, research/automate-steps-and-runs.md