1. Docs
  2. Get started
  3. Quick Start
  4. Run your first test

Run your first test

Create a first test, try it with a dry run, then run it in a pipeline so it shows up in Report.

3 min read Updated CustomersAdmins

In short

On this page you can

  • Choose one of five routes to a first test, from recording to AI agents.
  • Record a web test in Automate with the browser extension.
  • Try it with Dry Run on the server or on your agent.
  • Run it in a pipeline so it appears in Report.
  • Understand why dry run results are not saved to reports.

How to use it

  1. Open First Test Case in Automate, or click New Test Case.
  2. Pick a Start Page URL and environment, click Record and do a short flow.
  3. Close the recording tab, name the test case and click Save.
  4. Click Dry Run and choose Run on Server or Run on your Agent.
  5. In Schedule, add it to a pipeline, click Save & Execute and open Report.

Create your first test#

There are five ways to get a first test. The setup guide calls them routes; finishing any one of them completes the Teach your app step.

RouteBest forWhat you do
Record it yourselfA first test in minutes; people who know the appMap pages with Discover Your App, then record a test case in Automate.
Write steps with the AI AssistantDescribing a flow in plain languageCreate a test case and let the AI Assistant write its steps.
Start from documents or screenshotsTeams with analysis documents or specsUpload documents or screenshots in Analyze and review the generated test cases.
Use your code assistant (MCP)Developers with Claude or another MCP clientConnect the assistant (Test Agents › Connect to Claude), open the code repository and ask it to build the suite.
Let AI agents exploreBroad first coverage with little effortRun an autonomous agent and approve the drafts it writes.

Record a web test#

  1. Open the test case

    Go to Automate and open First Test Case, or click New Test Case.

  2. Pick environment and start page

    Choose a Start Page URL and an environment. Without both, recording does not start (“Please select a Start Page URL.”).

  3. Record

    Click Record. Your app opens in a new tab and the extension adds every click, typed value and selection as a step. Log in and do the flow the way a user would.

  4. Finish and come back

    Close the recording tab and return to SmartestQA. The steps appear in the list.

  5. Name and save

    Give the test case a name and click Save. You do not have to write a description: the AI writes one from the steps.

    The test case shows your steps, each one linked to an action in Discover.

Record a short, complete flow first

Login, open one screen, check one value. Five to ten steps. A short test that passes teaches you more than a long one that fails at step 37.

Record with test data only

Recording saves screenshots of the controls you use, and the values you type are saved in the steps. Use a test user and test data. Never record with real customer data or a personal password.

Run it and read the report#

There are two ways to run a test, and only one of them leaves a report.

Dry RunPipeline run
Start it fromAutomateDry RunSchedule: a pipeline, with Save & Execute or a timer
Use it forTrying a test while you build itRuns you want to keep, share and compare
You seeA live log while it runsThe live log, then a full report
Saved in ReportNo: “Dry Run results are not saved to reports.”Yes, with screenshots, video and logs
Automate › Dry Run run-target choice showing Run on Server and Run on your Agent
Choosing where a dry run runs.
  1. Dry run

    In Automate click Dry Run and choose the run target: Run on Server (headless, no agent needed) or Run on your Agent (on your computer; you can watch the browser). Watch the live log.

  2. Put it in a pipeline

    Go to Schedule, create a pipeline, give it a name, add your test case and click Save & Execute.

  3. Read the report

    Open Report. Each run lists its test cases and steps with status, screenshots, video and logs.

    You have a passing, reported test. The Run & report step of the setup guide is done.

Didn’t find your answer?Write to support with your project name and what you see on screen.

info@smartestqa.ai