---
title: "How do I build, run and schedule a regression suite?"
section: "Use cases"
description: "Build a regression suite in SmartestQA: add test cases to a pipeline, set Setup and Teardown, run it on a server, schedule it daily, weekly, monthly or with cron, and read each run in Report."
url: https://docs.smartestqa.ai/en/use-case-regression.html
audience: [customer, admin]
updated: 2026-11-12
keywords: "regression suite regression testing nightly run schedule tests pipeline daily weekly monthly cron setup teardown same session save and execute scheduled runs next run report export pdf ci cd automated regression regression suite example regression by hand cron job test scripts manual regression before release"
---
# Use cases: How do I build, run and schedule a regression suite?

A regression suite in SmartestQA is a pipeline: a saved list of test cases that runs in a fixed order. Build it in Schedule, run it once with Save & Execute, set its Period to Daily, Weekly, Monthly or Custom (Cron), and read each run in Report.

## In short

**On this page you can**

- Collect the test cases of a regression suite in one pipeline.
- Log in once in Setup and clean up in Teardown.
- Run the suite now, on a schedule, or from CI/CD.
- Stop a run safely and see every past run.
- Read each run in Report, export it as PDF or send it by e-mail.

**How to use it**

1. Open Schedule and click New Pipeline ; name it.
2. Click Add Test Case , tick the cases and click Add to pipeline .
3. Set the login case to Setup and the clean-up case to Teardown .
4. Click Save & Execute and check the report.
5. Choose a Period , set Start Date Time and End Date Time , click Save .

## The short answer

You do not need a script or a separate scheduler. A **pipeline** holds the test cases of the suite, where they run and with which settings. It starts when you click **Run**, at the times its [**Period**](https://docs.smartestqa.ai/en/schedule-timing.html#period) sets, or from your CI/CD. Every run is kept in **Report** with its steps, screenshots and logs.

## The problem this solves

Many teams run their regression checks by hand before a release: a list of flows in a document, and a few people who click through them. As the application grows, the round takes longer and gets shortened under time pressure.

Teams that automate often end up with scripts on one machine and a scheduler entry that starts them. Then the questions change: did it run last night, where is the result, and who cleans up the test data when a script stops halfway?

## An example

This is an illustration, not a customer story. A team runs a booking site and wants its twelve main flows checked every night.

1. A tester opens **Schedule**, clicks [**New Pipeline**](https://docs.smartestqa.ai/en/schedule-create.html#create) and names it. With **Add Test Case** the tester ticks the twelve cases and clicks **Add to pipeline**.
2. The login case gets the phase **Setup** and the case that deletes the test bookings gets **Teardown**. The first case after the login gets [**Same session**](https://docs.smartestqa.ai/en/schedule-create.html#same-session), so it continues in the login’s browser.
3. The tester keeps [**Run on Server**](https://docs.smartestqa.ai/en/quick-start-connect.html#agent) and clicks [**Save & Execute**](https://docs.smartestqa.ai/en/quick-start-first-test.html#step-6). The report opens. Its Summary shows the verdict **Failed** and lists the one failed test case first, with its [**Failure reason**](https://docs.smartestqa.ai/en/report-summary.html#test-cases-table). The tester fixes it and runs the pipeline again until it passes.
4. The tester sets [**Period**](https://docs.smartestqa.ai/en/schedule-timing.html#period) to **Daily**, [**Start Date Time**](https://docs.smartestqa.ai/en/schedule-timing.html#dates) to tomorrow at 02:00, an [**End Date Time**](https://docs.smartestqa.ai/en/schedule-timing.html#dates), and clicks **Save**.
5. [**Scheduled Runs**](https://docs.smartestqa.ai/en/schedule-timing.html#scheduled-runs) lists the pipeline. **Next run** reads 05:00, three hours later than intended, because the server reads the time in its own time zone. The tester sets the time three hours earlier and saves again.
6. The next morning the night’s run is in **Report**, and the pipeline’s History lists it on top.

The suite now starts by itself every night, and the result is in one place for the whole team.

## By hand, with your own scripts, and with SmartestQA

| Question | Regression by hand | Your own scripts and a scheduler | SmartestQA |
| --- | --- | --- | --- |
| What is the suite? | A list of flows in a document. Fine for a short list | A folder of scripts in a repository | A pipeline: a saved list of test cases that runs in a fixed order |
| Who starts it? | The testers, before a release | A scheduler entry or a CI job you set up | **Run**, the pipeline’s [**Period**](https://docs.smartestqa.ai/en/schedule-timing.html#period), or your CI/CD |
| Where does it run? | On the testers’ machines | On a machine or runner you maintain | On SmartestQA servers with [**Run on Server**](https://docs.smartestqa.ai/en/quick-start-connect.html#agent), or with [**Run on Your Agent**](https://docs.smartestqa.ai/en/schedule-where.html#agent) |
| Is test data cleaned up after a failure? | If the tester remembers | If the script handles it | Teardown runs once at the end and always runs |
| Where is the result of last night? | In notes or a chat message | In the log or report the scripts write, if someone keeps it | In **Report**, with steps, screenshots and logs; reports are not deleted automatically |
| What if a planned run is missed? | It is done later, or not | Depends on the scheduler | There is no catch-up: the run is skipped and the next one starts at its normal time |

## Steps

1. **Create the pipeline**

   Open **Schedule** › **Pipelines** and click [**New Pipeline**](https://docs.smartestqa.ai/en/schedule-create.html#create). SmartestQA creates the pipeline immediately and opens its page. Name it after what it proves.
2. **Add the test cases**

   Click **Add Test Case**. Filter by name, tick cases or a whole folder, then click **Add to pipeline**. To run everything linked to a requirement instead, use **Add Requirement**. A pipeline holds test cases or requirements, not both. See [Add test cases, requirements or pipelines](https://docs.smartestqa.ai/en/schedule-create.html#add-content).
3. **Set the order and the phases**

   Drag the rows into the order they should run. Every row starts as a **Test**. Hover a row, click its phase chip and choose **Setup** for the login case and **Teardown** for the clean-up case. Setup runs once at the start; if it fails, the tests are skipped. Teardown runs once at the end and always runs. See [Setup, Before, After and Teardown](https://docs.smartestqa.ai/en/schedule-create.html#phases).
4. **Keep one browser session where it helps**

   Turn on [**Same session**](https://docs.smartestqa.ai/en/schedule-create.html#same-session) for a case that should continue in the previous case’s browser, for example after the login. See [Continue in the same browser session](https://docs.smartestqa.ai/en/schedule-create.html#same-session).
5. **Run it once**

   Keep [**Run on Server**](https://docs.smartestqa.ai/en/quick-start-connect.html#agent), which needs no setup, or choose [**Run on Your Agent**](https://docs.smartestqa.ai/en/schedule-where.html#agent). Click [**Save & Execute**](https://docs.smartestqa.ai/en/quick-start-first-test.html#step-6).

   The run is queued and its report opens. Make sure the suite passes before you schedule it: a schedule repeats whatever the pipeline does, failures included.
6. **Put it on a schedule**

   A new pipeline has the Period [**Real-time**](https://docs.smartestqa.ai/en/schedule-timing.html#period): it runs only when something starts it. Choose **Daily**, **Weekly**, **Monthly** or [**Custom (Cron)**](https://docs.smartestqa.ai/en/schedule-timing.html#cron). [**Start Date Time**](https://docs.smartestqa.ai/en/schedule-timing.html#dates) is the first run, and its time is the time of every run. [**End Date Time**](https://docs.smartestqa.ai/en/schedule-timing.html#dates) is when the schedule stops. Click **Save**. See [Set a schedule](https://docs.smartestqa.ai/en/schedule-timing.html#set-schedule).
7. **Check the next run**

   Open **Schedule** › [**Scheduled Runs**](https://docs.smartestqa.ai/en/schedule-timing.html#scheduled-runs). The pipeline is listed with its Period and **Next run**, shown in your own time.
8. **Read the result**

   Open **Report** and find the run. The Summary sorts test cases as failed, passed on retry, running, not run, then passed, and shows the first failed step next to each. [**Export PDF**](https://docs.smartestqa.ai/en/report-summary.html#export-pdf) and [**Share Report**](https://docs.smartestqa.ai/en/report-summary.html#share-report) are enabled once the run has finished. See [Run summary, export & share](https://docs.smartestqa.ai/en/report-summary.html).

## Watch it

The video builds a pipeline of four test cases with Setup, Same session and Teardown, runs it on a server, stops a run, and schedules the pipeline for every night at 02:00.

Video: [Schedule](https://docs.smartestqa.ai/media/videos/MOD-SCHEDULE.mp4) (3:06)

Build a pipeline, run it, stop it, and put it on a nightly schedule.

Video transcript: Schedule lists every pipeline with its test cases, run environment and last result. New Pipeline creates an empty pipeline and opens it. Name it after what it proves. Add Test Case opens the project's test cases. Tick Log in as test user: it will be the Setup. Filter the tree to find the rest: three cases that add, edit and delete a note. Tick all three. The counter now shows four cases selected. Select a case to preview its steps before you add it. Add to pipeline puts the ticked cases into the run list. Every row starts as a normal Test. Hover a row and click its phase chip. Setup runs once at the start. Before and After wrap every test. Teardown runs once at the end. The login becomes Setup: if it fails, the tests are skipped. The delete case becomes Teardown. Teardown always runs, so the note is cleaned up even after a failure. Same session: the case continues in the previous case's browser. No new browser, no start page. Turn it on for every case that should carry on where the login left off. The run list now reads: Setup, two tests, Teardown, all in one browser session. Run on Server needs no setup: the first free server in the shared pool takes the run. Save & Execute saves the pipeline and starts the run. The run is queued, then the report opens on its own. Test Execution Logs follow the run live: the queue, the opened page, then each step. Every step lands in the log as it runs, until the Teardown finishes. Find the pipeline, hover its row and click Run. Run queues the pipeline and opens its report as soon as it starts. Back in Schedule, Last Run Date shows Running, with Stop run beside it. Stop run asks first: the running case finishes, the rest are skipped, and Teardown still runs. The cell shows Stopping until the run ends. Open the pipeline and click History for every past run, with result and status. A new pipeline runs on demand: its Period is Real-time. Open Period to schedule it: Daily, Weekly, Monthly or a cron expression. Choose Daily. The date fields become editable. Start Date Time is the first run, and its time is the time of every run. Here: 02:00. End Date Time is when the schedule stops: at most one year ahead. Save. From now on the pipeline starts by itself every night. Scheduled Runs lists every scheduled pipeline with its period and Next run, in your own time.

To see how a finished run is read, watch the [Report lesson](https://docs.smartestqa.ai/en/videos.html#video-mod-report): a passed run and a failed run, down to the step that broke.

## What you get

- **A suite that starts by itself.** Daily, weekly or monthly at the time of [**Start Date Time**](https://docs.smartestqa.ai/en/schedule-timing.html#dates), or at the times a cron expression of five fields sets.
- **A fixed order with clean-up.** Five phases, always run as Setup → Before → Test → After → Teardown. Teardown always runs, so test data is removed even after a failure.
- **Three ways to start it.** By hand with **Run** or [**Save & Execute**](https://docs.smartestqa.ai/en/quick-start-first-test.html#step-6), by the schedule, and from a terminal or CI/CD with one command.
- **A report per run.** A report appears within seconds of the run starting and fills in live. Each step keeps its log, a screenshot and its duration. Reports are not deleted automatically.
- **History.** The pipeline’s History lists every run, newest first, with its result and status. InSight shows the last 14 runs of each pipeline in one strip.
- **Retries when you want them.** [**Retry Count**](https://docs.smartestqa.ai/en/project-settings-test-management.html#retry-math) re-runs a failed test 0 to 5 times.

## Questions and answers

### Q: What is a regression suite in SmartestQA?

A pipeline in Schedule. It holds test cases, or requirements whose linked test cases it runs, together with the run environment and the settings of the run. You can tag a pipeline; InSight reads the tags to show which kinds of suite a project has, such as **Smoke** and **Regression**. See [Tag a pipeline](https://docs.smartestqa.ai/en/schedule-create.html#tags).

_Audience: customer, admin_

### Q: How often can the suite run?

**Daily**, **Weekly** and **Monthly** run at the time of [**Start Date Time**](https://docs.smartestqa.ai/en/schedule-timing.html#dates). [**Custom (Cron)**](https://docs.smartestqa.ai/en/schedule-timing.html#cron) takes a five-field expression, for example `30 2 * * 1-5` for weekdays at 02:30. Choose an interval longer than one run takes. See [Cron expressions](https://docs.smartestqa.ai/en/schedule-timing.html#cron).

_Audience: customer, admin_

### Q: How long does a schedule keep running?

Until [**End Date Time**](https://docs.smartestqa.ai/en/schedule-timing.html#dates). For Daily, Weekly and Monthly the end can be at most one year from today; extend it before it passes to keep the schedule going. After the end date the pipeline stays and can still be run by hand. See [Start and end dates](https://docs.smartestqa.ai/en/schedule-timing.html#dates).

_Audience: customer, admin_

### Q: What happens if a scheduled run is missed?

There is no catch-up: the run is skipped and the next one starts at its normal time. A run on your own agent needs the agent online at that moment. See [Missed runs, restarts and long runs](https://docs.smartestqa.ai/en/schedule-timing.html#missed-runs).

_Audience: customer, admin_

### Q: What happens when one test fails in the middle of the suite?

With [**Continue On Failure**](https://docs.smartestqa.ai/en/project-settings-test-management.html#retry-math) on, the remaining tests still run. With it off, the first failed test skips the rest. In both cases Teardown runs. See [What happens when a phase fails](https://docs.smartestqa.ai/en/schedule-create.html#phase-failures).

_Audience: customer, admin_

### Q: Can my CI/CD start the suite?

Yes. The pipeline page gives a one-line command for your shell. The run starts like a click on **Run**, and CI can wait for the result and read an exit code: 0 passed, 1 failed or disconnected. See [Run from a terminal or CI/CD](https://docs.smartestqa.ai/en/schedule-runs.html#cli).

_Audience: customer, admin_

### Q: How does the team get the result?

Every run is in Report for everyone with access to the project. A pipeline can also send the test report by e-mail when a run ends, always or only on failure. See [E-mail notifications](https://docs.smartestqa.ai/en/schedule-settings.html#email).

_Audience: customer, admin_

### Q: Why is Next run a few hours away from the time I typed?

The server reads the time in its own time zone, and **Next run** is shown in yours. Trust the **Next run** column and move the start time by the difference. See [Time zone](https://docs.smartestqa.ai/en/schedule-timing.html#time-zone).

_Audience: customer, admin_

## Read more

- [Build a pipeline](https://docs.smartestqa.ai/en/schedule-create.html): content, order, phases, sub-pipelines and tags.
- [Schedule a pipeline](https://docs.smartestqa.ai/en/schedule-timing.html): Period, dates, cron and the Scheduled Runs tab.
- [Where it runs](https://docs.smartestqa.ai/en/schedule-where.html): servers, agents and environments.
- [Run, stop & follow](https://docs.smartestqa.ai/en/schedule-runs.html): run now, stop a run, history and CI/CD.
- [Report](https://docs.smartestqa.ai/en/report.html): find a run and read it down to the failed step.
- [How do I prove every requirement is tested?](https://docs.smartestqa.ai/en/use-case-coverage.html): what the suite proves.
