AI in Automate
AI in Automate writes steps from a sentence, turns a passing test into negative and edge cases, drafts test cases for a whole module, and writes each test case’s description. All of it builds on the actions you recorded.
In short
On this page you can
- Build a step list from one sentence with Generate from Prompt.
- Create 2–4 negative and edge variants with Expand Scenarios.
- Draft a folder of test cases with Generate Test Cases with AI.
- Get each test case’s description written from its steps.
- Ask the AI for locator suggestions on a failed dry-run step.
How to use it
- Ask an admin to turn on Enable AI and AI Document Analysis.
- Record the actions of the flow first; the AI reuses them.
- Open AI Assistant in the editor header and describe the test.
- Review the generated steps, then click Save.
- Dry run and fix what the AI got wrong.
Turn on AI#
An Admin turns on two options in ConfigurationProject PanelAI Options:
- Enable AI: the main switch.
- AI Document Analysis: needed for the AI Assistant and AI steps.
Until both are on, AI buttons are disabled with a tooltip such as “Enable 'AI Document Analysis' in Project Panel → AI Options”. The project also needs a working AI connection: either SmartestQA’s own AI or your own provider account and key. Without one, AI requests fail with “AI is not enabled or API key is missing for this project.”
The AI works from what SmartestQA knows
Generated steps use your recorded actions, and generated test cases use your pages, page purposes and notes from Discover. The better Discover is filled in, the more specific the results. With no actions at all: “No actions found for this project. Please record actions first.”
Generate steps from a prompt#
Open the AI Assistant
Open the test case and click AI Assistant in the editor header. Under Generate from Prompt: “Describe what to test in natural language — AI builds the steps.”
Describe the test
Write one test, with the data and the expected result:
Log in as test.user, search for order A-1029 and check that its status is Shipped.Generate
Click Generate Steps.
“Test case loaded into Details tab. Review and click Save to keep it.” The editor shows the new steps, name and description.
Review and save
Check every step and its values, then click Save. Nothing is saved before that.
Generate Steps replaces the whole test case
The generated steps, name and description replace what is in the editor. Use it on a new, empty test case. If you used it by mistake, leave the test case without saving (Exit without saving).
- The AI uses only actions that exist. If some parts of the request have none, you see “AI could only partially cover your request — missing actions for …”, and those parts become AI steps (on page
AI_Page, performing the action, stopping the test if they fail). Record the missing controls and generate again, or replace those AI steps by hand. - If nothing usable comes back: “AI could not produce any usable steps for this prompt. Try rephrasing with more detail.”
Expand a test case into variations#
Once a happy path passes, Expand Scenarios uses it as a seed and writes negative paths, edge cases and error handling.
Open the source test case
Open the passing test case and click AI Assistant.
Generate variations
Under Expand Scenarios click Generate Variations. The AI writes 2 to 4 variations, each marked Negative or Edge, with its steps.
Choose and name the folder
Tick the ones you want and give the folder a name (New folder name, default
Test Case Scenarios).Create
Click Create.
A new folder is created under the source’s folder, the new test cases are created in it, and the original test case is moved into it next to them.
The original moves
If the source test case was directly under the project root, the new folder is created at the root and becomes a module, also in Analyze and the App Map. Move it with drag and drop if that is not what you want.
Generate test cases for a module#
Choose the folder
Select the folder the new test cases should go under (or nothing, for the root).
Open the dialog
MoreGenerate Test Cases with AI. “AI analyzes your recorded actions and page flow to generate meaningful E2E test cases — complete with realistic test data and step parameters.”
Set the options
Option Choices Test type Full, Smoke, Functional, E2E, Regression Count Auto, 3, 5, 8, 10, 15, 20, 30 Step depth Auto, Short (3-5), Medium (5-8), Detailed (8-15) Verification Strong, Simple Test data Realistic, Boundary/Invalid Module All pages, or one module Pages (scope) The pages to cover, with their action counts Scenario types Happy, Negative, Edge, CRUD, E2E Include preconditions (login/setup) On or off Special focus (optional) Free text, for example “focus on the checkout flow and cart verification” Generate and review
Click Generate and wait (“Generating test cases with AI…”). Review the list; use Select all, Deselect all or Regenerate.
Create
Click Create.
“n test case(s) created in "folder"”. The folder is named by the AI (by default
AI Generated Test Suite), under the folder you selected; at the root it becomes a module.
Dry run each generated test case before you add it to a pipeline.
AI-written descriptions#
The Description (written by AI) field in Test Case Detail is read only. It is written when someone runs Update map in the Discover App Map: the preview says “Descriptions of n test cases will be written with AI (only those whose steps changed since; one AI call per 8 test cases)”, and Apply writes them.
- Only test cases whose name, steps, active state or parameters changed since the last description are rewritten.
- Up to 8 test cases per AI call and 600 per update; up to 60 steps per case are read; a description has up to 1,000 characters.
- Values that look like secrets are masked before they are sent. A batch that fails is tried again at the next update.
- Until a description is written, the field copies the test case name.
AI steps#
An AI step is written in plain language and performed by the AI on every run. There are four modes, shown in the step list as AI Ask (answer a question about the screen, for example “What is the total price on screen?”), AI Check (pass or fail, “Is the login button visible?”), AI Do (perform an action, “Type test@test.com in email field”) and AI Chat (text only, no screen).
- New AI steps stop the test when they fail.
- If AI is switched off for the project, an AI step fails and the test stops.
- AI steps are slower, cost an AI call each time and may answer differently from run to run. Folder summaries flag them (“Contains AI steps”). Replace them with ordinary actions once the flow is stable.
Parameters and outputs of AI steps: step reference.
AI Analyze on a failed dry run#
After a step fails in a dry run on your agent, AI Analyze on that step reads the live page and suggests locators. See AI Analyze.
What is sent to the AI#
- Generate from Prompt, Expand Scenarios, Generate Test Cases: your prompt or source test case, action names and pages, and page descriptions and notes from Discover.
- Descriptions: the test case name and its steps, with secret-looking values masked.
- AI steps and AI Analyze: the prompt and information about the current page.
Requests go to the AI provider configured for the project. Use test environments and test data. See also What is sent to the AI.
If something goes wrong#
- AI buttons are disabled, or “AI is not enabled or API key is missing…”
- “No actions found for this project. Please record actions first.”
- “AI could only partially cover your request…”
- Generate Steps replaced my steps
- Expand Scenarios moved my test case
- “AI request limit reached…” or “Could not reach the AI service…”
Didn’t find your answer?Write to support with your project name and what you see on screen.
info@smartestqa.ai