1. Docs
  2. Get started
  3. Core concepts

Core concepts

SmartestQA has a small set of building blocks that every screen reuses. Learn them once and every module reads the same way: what you capture, what you build from it, where it runs and what it leaves behind.

16 min read Updated CustomersAdmins

In short

On this page you can

  • Learn the building blocks: company, project, action, test case, pipeline, report.
  • See how DEV, TEST and PRP and the start page URL decide what a test opens.
  • Tell a dry run from a pipeline run, and a test agent from the Smartest Server.
  • Find where each AI feature lives, from Enable AI to MCP assistants.
  • Jump from each concept to the page that explains it in detail.

How to use it

  1. Create a project of the right type and add a start page URL with a TEST address.
  2. Capture pages and actions in Discover, then write requirements in Analyze.
  3. Build test cases from those actions in Automate and try them with Dry Run.
  4. Put the test cases in a pipeline in Schedule and run it on the server or an agent.
  5. Read the report, review repairs in Maintain and work the tasks that failed runs created.

At a glance

The account
A company holds users, the licence and projects. One project tests one application: WEB, MOBILE or DESKTOP.
Order of work
The main menu, top to bottom: Discover, Analyze, Automate, Schedule, Report, Maintain.
What you build
Actions (controls with locators), requirements, test cases made of steps, and pipelines that run them.
Where tests run
The Smartest Server, a test agent on your machine, or a device farm for phones.
What stays
Every pipeline run writes a report. A dry run shows a live log only.
AI
Optional per project. It names controls, writes requirements and steps, explains failures and repairs locators. Assistants such as Claude connect through MCP.

Start from your question#

How the pieces connect#

Read this list top to bottom once. Each line builds on the one before it, and each concept links to its own section below.

  1. A company has users with a role, a licence and one or more projects.
  2. A project tests one application. Its start page URLs hold one address per environment: DEV, TEST and PRP.
  3. Discover captures the app: pages, the actions on each page with their locators, the App Map and your Notes.
  4. Analyze holds what the app must do: main requirements split into testable sub-requirements.
  5. Automate builds test cases. Their steps point at actions, so fixing one locator fixes every test that uses it. Test cases link to sub-requirements.
  6. Test data feeds the steps: variables, Data Source keys, datasets, database and Kafka connections. API requests become API steps.
  7. A pipeline in Schedule runs a list of test cases (or requirements) in one environment, by hand, on a schedule, from CI or from a trigger.
  8. The run executes somewhere: on the Smartest Server, on a test agent or on a device farm.
  9. Every pipeline run writes a report. InSight adds the numbers up across runs. A dry run writes no report.
  10. When a locator breaks, auto-heal repairs it and a person reviews the repair in Maintain. Failed runs can create tasks.
  11. AI helps at every step when it is on, and MCP lets an assistant such as Claude do the same work with your approval.

Where each thing lives in the menu#

Menu itemWhat you keep thereExplained in
InSightThe home dashboard: run results and test health across the project.InSight
DiscoverActions, App Map pages, modules and Notes.Discover
AnalyzeRequirements, their folders and project documents.Analyze
AutomateTest cases, templates and their folders.Automate, Step reference
SchedulePipelines, scheduled runs, triggers and queues.Schedule
ReportOne report per pipeline run.Report
MaintainAuto-heal reviews, AI steps and network captures to convert.Maintenance
AgentsAutonomous AI agents of the project.Autonomous Agents
Data SourceKeys, datasets and database and Kafka connections.Data Source
API RequestsSaved HTTP requests in collections and folders.API Requests
Device FarmTest agents, and for mobile: nodes, devices and the App Library.Agent Pool & Device Farm
Deploys & ReposCode repositories and the deploys your CI reports.Deploys & Repos
TasksDefects and work items, many created by failed runs.Tasks
ConfigurationProject Panel, Company Settings, Integrations and Logs (Admins).Project settings, Company settings, Integrations, Logs

Company and project#

Company
Your account. Users, projects, the licence and billing belong to it. Whoever signs up becomes its first Admin. See Create your account.
Project
One application under test. Actions, requirements, test cases, pipelines, reports, data and integrations all belong to one project. See Create a project.
Sample project
A finished example project created once per company, with actions, tests, a pipeline and reports. It does not count against the licence. See Sample project.
Licence limits
Your company’s licence can limit users, projects, test cases, requirements, pipelines, pipeline runs and parallel threads. Counters count what you create, so deleting does not give capacity back. At a limit you see “Company license limit has been reached…”. An Admin sees the licence in Company Settings › Billing.

Project types: WEB, MOBILE, DESKTOP#

The type decides how you capture the app and where its tests can run. API, database, mail and load-test steps work in every type, so you do not need an extra project for them.

TypeForCapture withTests run on
WEBAnything that opens in a browser.The Chrome or Edge extension: Discover Your App and Record Actions.The Smartest Server or your test agent.
MOBILENative or hybrid Android and iOS apps; one project covers both.The Appium inspector on a device or emulator.Your agent with a local device, SmartTest Device Farm, AWS Device Farm or BrowserStack.
DESKTOPA desktop application. The platform is set in Desktop Configuration (Windows by default, or macOS).The Desktop Recorder through the agent.Only your agent, on the machine where the app is installed.

More: Which project type?, Mobile & desktop capture, Mobile Config, Desktop Configuration.

Environments and start page URLs#

Environment
Which installation of your app a recording or run uses: DEV, TEST or PRP, PRP being the production-side environment. Test cases follow the pipeline’s environment, so one set of tests serves all three. See Start page and environment.
Start page URL
A named address with one value per environment, where web recordings and runs begin. One of them is the project’s default. An environment with no address is greyed out (“No URL defined for this environment”). See Add a start page URL and Start pages.
Environment-specific key
A Data Source key whose name ends in _DEV, _TEST or _PRP. A run in that environment uses it instead of the key without the suffix, for example a different API address per environment. See Environment variants.
Deploy environment
Deploys & Repos also accepts names you add, such as STAGING. Pipelines still run on DEV, TEST or PRP only. See Deploys & Repos.

PRP is real

Tests on PRP create orders, send e-mails and change data in production. Record and build on TEST with a test user, and run only agreed read-only checks on PRP.

What SmartestQA knows about your app (Discover)#

Page
A name that groups actions, such as checkout. Pages are created by naming them; there is no separate create step. See Page names.
Action
One control on one page, for example Click Sign In on page login, with a type, its locators, an optional Navigates To page and a screenshot. Steps point at actions by ID, so a fixed locator fixes every test that uses it. See Actions.
Locator
How a test finds the control: id, name, css, xpath, accessibility_id, resource-id, class, linktext or tagname. Recorded actions keep fallback locators. See Locator chip.
App Map
Every page as a record: purpose, preconditions, expected outcomes, risk, roles, screenshots and where it leads. Update map builds it from your actions. The AI reads it when it writes requirements and tests. See App Map.
Module
A business area such as Orders. A module groups pages in the App Map, and the same module is the top-level folder in Analyze and Automate. See Modules.
Note
One rule or fact about the app that no scan can see, such as “Orders over €10,000 need a second approver”. Active notes are sent to the AI. See Notes.

Requirements (Analyze)#

Main requirement
A feature or user flow, for example Pay with a saved card. It is a row in the requirement tree. See Write requirements.
Sub-requirement
One testable condition with an expected result inside a main requirement. Test cases are linked here, and coverage is counted over sub-requirements. See Link test cases.
Assessment
An AI review of a main requirement and its subs, with a Quality Score from 0 to 100. The score rates the text, not whether it is tested. See AI assessment.
Requirements from documents
The Analyze with AI wizard turns a spec or screenshot into requirements and test cases. See Generate from documents with AI.

Test cases and steps (Automate)#

Test case
An ordered list of steps that runs on its own and produces a report when a pipeline runs it. See Test cases and folders.
Step
One thing the test does. An action step uses a shared action (click, type, select). A step-type step has its own dialog: Assertion, Python, JavaScript, AI, API, Network, Database, Mail, Kafka, Global parameter, JMeter or k6. See Step reference.
Template
A test case used as one step inside other test cases, such as a login. Editing it changes every test that uses it. See Templates.
Recording
The fastest way to start a test: the extension (web), the inspector (mobile) or the Desktop Recorder writes the steps and the actions they use. See Record and add steps.
Archive
Where removed test cases and folders go. Restoring brings them back, but not into the pipelines they were removed from. See Archive.

Test data: variables, Data Source and datasets#

Steps never need a hard-coded value. They read it by name when the step runs:

Written asComes fromExplained in
{{name}} or {{tc:name}}A value an earlier step saved in this run (a Get step, an API extraction, a mail OTP, a code step).Data and variables
{{ds:key}}A Data Source key: a typed value, a dataset cell or a database query. Encrypted keys hold passwords and tokens.Variables (keys), Secrets
{{excel:name}}_columnA column of a dataset uploaded from Excel or CSV. A pipeline can run the test once per row; a dry run uses the first row.Datasets, Run once per row
Data Source
The project’s store of keys, datasets and connections. Tests reference them by name, so nothing secret sits in a step. See Data Source.
Connection
A database (JDBC) or Kafka login that Database and Kafka steps use. The password lives in the connection, never in a test. See Connections.
Global parameter
A value a step stores under a project-wide name. Later runs can read it, which is why a skipped step can leave a value from an earlier run. See Where a variable gets its value.
Pipeline parameter
A value a step types or selects, changed for one pipeline only. The test case in Automate keeps its own value. See Pipeline Parameters.

A name that no source provides is typed as written, for example {{orderNo}}. See The test typed {{name}} literally.

API requests#

The API Requests library holds saved HTTP calls in collections and folders. You import them from Postman, Swagger / OpenAPI or curl, try them with Send from the SmartestQA server, and add them to test cases as API steps. Each API step keeps its own copy of the request, so later edits in the library do not change existing tests. See API Requests and The step’s own copy.

Pipelines (Schedule)#

Pipeline
A named, saved run plan: a run list plus the settings to run it. Some messages call it a scheduler. See Schedule.
Run list
What the pipeline runs, in order: test cases, or requirements (their linked test cases), plus other pipelines as sub-pipelines. Test cases and requirements cannot be mixed. See Add content.
Phases
Setup runs once at the start, Before before each test, Test is the check itself, After after each test, Teardown once at the end. If Setup fails the tests are skipped; Teardown always runs. See Setup, Before, After and Teardown.
Period
When the pipeline starts by itself: Real-time (never by itself), Daily, Weekly, Monthly or Custom (Cron). See Schedule a pipeline.
Run Environment
Where the run executes: a SmartestQA server, your agents, or a mobile device farm. See Run Environment.
Trigger and queue
A trigger starts a pipeline when a webhook is called, an e-mail arrives or a file lands in a folder. A queue runs one pipeline run per work item. See Triggers & queues.

A pipeline copies the project’s run settings when it is created and keeps its own copy after that. See How settings reach a run.

Run modes: dry run, pipeline run, agent#

A test case runs in one of two modes. An agent is not a third mode: it is a place where either mode can run.

Dry runPipeline run
Started fromDry Run in the Automate editorA pipeline: Run or Save & Execute, a schedule, CI/CD, a trigger or a queue
Use it toTry a test while you build itKeep results, run nightly, gate a release
ReportNo. Live log only; screenshots are deleted after one hourYes, with screenshots, logs and video
Dataset rowsFirst row onlyOne run per row when the pipeline asks for it
Retries, stop, status iconNone; cannot be stopped; icon unchangedThe pipeline’s own Retry Count; can be stopped; icon updated

Full comparison: Run modes. A scheduled run is simply a pipeline run started by its Period. See Run, stop and follow.

Where tests run: server, agent, runner, device farm#

Smartest Server
SmartestQA’s own test servers. Run on Server runs the browser there, headless, with nothing to install. It reaches apps open to the internet, not your internal network. Runs wait in a fair queue when all slots are busy. See Run on Server.
Test agent
The SmartestQA client on a computer you control (also called a host). It runs tests there and sends logs, screenshots and video back. Every user has a personal Localhost agent; shared agents serve the whole company. See Test Agents.
Runner
The part of SmartestQA that executes a test case step by step, finds elements, retries and heals. It works on the Smartest Server and inside your agent. Run logs quote it as “the runner”.
Device Farm
Phones, tablets and emulators connected through nodes, with an App Library of builds to install. You can run tests on a device or take it for a live session. Mobile projects can also use AWS Device Farm or BrowserStack. See Agent Pool & Device Farm.
Your situationRun on
Web app reachable from the internetThe Smartest Server
Web app behind a VPN or firewall, or steps that reach an internal database, mailbox or APIAn agent inside that network
Phone or emulator on your deskYour agent, with Appium
Real devices you do not ownSmartTest Device Farm, AWS Device Farm or BrowserStack
Desktop applicationYour agent on the machine with the app

Do you need an agent at all? See Quick Start.

Two kinds of “agent”

A test agent (Device Farm › Test Agents) is a program that runs your tests on a machine. An autonomous agent (menu Agents) is an AI job that reviews runs, writes reports and proposes changes. They are unrelated; this documentation always says which one it means.

Results: report, live log and InSight#

Report
The stored record of one pipeline run: its verdict, every test case attempt, each step’s status, log, screenshot and durations, and video. You can export it as PDF or share it by e-mail. See Report.
Live log
The Test Run panel that streams while a run is going. It is not kept. Dry runs have only this. See The report record and the live log.
Attempt
One execution of one test case inside a run. Data rows and automatic retries each add an attempt, which is why a Result can say 5/6 for three test cases. See Result chip.
AI verdict
The AI’s classification of a failed step: Real Bug, Test / Infrastructure Issue, Flaky or Unclassified, with a confidence. See Verdicts.
InSight
The home dashboard. Execution shows how runs went in a period; Test Health shows how much of the application the tests prove: requirement, scenario, risk and action coverage. See InSight.

Auto-heal and Maintain#

When every locator of an action fails, the runner looks for the element again. If it finds one clear match, it writes the new locator to the action at once, so the run goes on and every test that uses the action is fixed. Each repair waits as a card in Maintain › Auto-heal until a person approves or rejects it; rejecting puts the old locators back. The runner refuses to heal when the app itself looks broken, so real bugs still fail.

Tasks and deploys#

Task
One item of work on a Kanban board, such as a defect. With Auto Task Creation on, a failed pipeline run creates a task with the failed steps and a link to the report. Tasks can sync with Jira or Azure DevOps. See Tasks and Tasks from failed runs.
Task status
The board column: Open, In Progress, Resolved, Closed, plus statuses your company adds. Statuses are shared by all projects. See Statuses.
Deploy
A record your CI sends when a version reaches an environment. SmartestQA shows which version runs where and whether a pipeline tested it. See Deploys & Repos.

AI features and MCP#

AI is switched per project in Project Panel › AI Options: Enable AI turns it on, and one switch per capability turns each feature on or off. The project uses SmartestQA’s AI account or your company’s own provider key. See AI Options.

WhereWhat the AI doesExplained in
DiscoverNames controls and helps pick locators while you record.Discover Your App
AnalyzeTurns documents into requirements and scenarios; scores requirements.AI wizard, Assessment
AutomateWrites steps from a prompt, expands a case into variations, writes code; AI steps act and check in plain words.AI in Automate, AI steps
ReportClassifies failed steps and reviews a whole run.AI failure analysis
MaintainChooses the element when a locator repair needs it.How auto-heal decides
AgentsAutonomous agents work on a rhythm or after a deploy and leave reports and proposals; nothing is applied without a click.Autonomous Agents
Your AI assistantClaude or another MCP client works in SmartestQA in a chat. Every change is shown as a diff first and written only after you approve it.MCP & AI assistants, Safe changes

What is sent to the AI provider is listed on each module’s FAQ, for example Discover and MCP.

Roles and administration#

RoleCanCannot
AdminEverything in the company: projects, users, Project Panel, Company Settings, Integrations, Logs.Assign Super Admin.
UserTest work in the projects they were given: Discover, Analyze, Automate, Schedule, Report and the rest of the main menu.Create projects or users; open Project Panel, Company Settings, Integrations or Logs.
Super AdminSmartestQA operators only.Not a customer role.

Item by item: Roles & permissions. Every change, from the app, MCP or an API key, is recorded in the audit log.

Didn’t find your answer?Write to support with your project name and what you see on screen.

info@smartestqa.ai
© 2026 Selenity Technology · SmartestQAsmartestqa.aiThis page as MarkdownChecked against smarttest-fe predev 2a8c79cc (menu-items/pages.js, translations/en.json), smarttest-be 633c2a03; built from the module pages of these docs, which are verified against the code