---
title: "CI/CD"
section: "Integrations"
description: "Run SmartestQA pipelines from GitHub Actions, GitLab CI, Jenkins, Azure Pipelines or any shell: the API key, the ready-made step, run states, JUnit report, exit codes and timeouts."
url: https://docs.smartestqa.ai/en/integrations-cicd.html
audience: [customer, admin]
updated: 2026-11-08
keywords: "ci cd github actions gitlab ci jenkins azure pipelines shell smartest-ci.sh smartest-ci.ps1 api key smartest_api_key pipeline_id junit exit code timeout trigger build fail"
---
# Integrations: CI/CD: GitHub Actions, GitLab CI, Jenkins, Azure Pipelines

The CI/CD page builds a ready-made step for your CI. The step starts a SmartestQA pipeline, waits for it, writes a JUnit report for the CI’s test tab and fails the build when a test fails.

## In short

**On this page you can**

- Start any pipeline of the project from GitHub Actions, GitLab CI, Jenkins or Azure Pipelines.
- Override pipeline parameters for one run, for example {"ENV":"uat"} .
- See each test case in the CI’s test tab from smartest-junit.xml .
- Fail the build when a test fails, with a link to the SmartestQA report.
- Use the same script from any CI that has bash and curl, or PowerShell.

**How to use it**

1. Under 1. Choose the pipeline pick the pipeline, optional parameters and a timeout.
2. Store an agent API key as the CI secret SMARTEST_API_KEY .
3. Under 3. Add the step pick your CI tool and click Copy .
4. Paste the step into your CI file and commit.
5. Run the CI job and open the report link it prints.

## How it works

1. **Start** — The step calls `POST /api/pipelines/run` with the API key and gets a run id
2. **Wait** — It checks `/api/ci/runs/id` every 15 seconds
3. **Report** — It downloads the JUnit file and prints the result and report link
4. **Exit** — 0 passed, 1 failed, 2 timeout or error

The run is a normal [pipeline](https://docs.smartestqa.ai/en/schedule-create.html) run: it appears in **Report**, sends the usual [notifications](https://docs.smartestqa.ai/en/integrations-notifications.html#events) and [webhooks](https://docs.smartestqa.ai/en/integrations-webhooks.html#events), and runs where the pipeline is set to run (server or agent). The run states follow the same rules as the Reports screen.

## Before you start

- **A pipeline** in the project. Without one the page says “This project has no pipelines yet. Create one first, then come back here.” with [**Go to Pipelines**](https://docs.smartestqa.ai/en/schedule-timing.html#scheduled-runs).
- **An agent API key.** Use the key of your own agent, or of a dedicated agent for CI (**Device Farm** › [**Test Agents**](https://docs.smartestqa.ai/en/agent-pool-agents.html#agent-list), [**Copy API Key**](https://docs.smartestqa.ai/en/agent-pool-agents.html#api-key); see [Install an agent](https://docs.smartestqa.ai/en/quick-start-connect.html#install-agent)). Runs start with the project access of the user the key belongs to.
- **Network.** The CI runner must reach the SmartestQA API address shown as `SMARTEST_URL`. The step needs `bash` and `curl` (or PowerShell on Windows); `jq` or `python3` are used when present.

> **Warning: Keep the key out of the file**
>
> Store the API key as a secret in your CI: GitHub repository secret, GitLab masked variable, Jenkins credential (Secret text, id `smartest-api-key`) or Azure Pipelines secret variable. Never paste it into the YAML. Anyone with the key can start pipelines in every project that user can open. A dedicated agent for CI lets you revoke CI access alone.

## Set up the CI step

1. **Choose the pipeline**

   Open **Integrations** › **GitHub Actions** (or GitLab CI or Jenkins). Under **1. Choose the [pipeline](https://docs.smartestqa.ai/en/schedule-create.html)** pick the **Pipeline**. Optionally enter **Parameters (optional, JSON)**, which override the [pipeline parameters](https://docs.smartestqa.ai/en/schedule-settings.html#pipeline-parameters) for this run only, and a **Timeout (min)** between 1 and 1440 (60 by default).
2. **Add the connection to your CI**

   Under **2. Add the connection to your CI** copy `SMARTEST_URL`. Create the CI secret `SMARTEST_API_KEY` with [the agent](https://docs.smartestqa.ai/en/agent-pool-agents.html) key (**Open Agents** takes you to the agents).
3. **Add the step**

   Under **3. Add the step** pick the **CI tool**: GitHub Actions, GitLab CI, Jenkins, Azure Pipelines or Shell / other CI. Click **Copy** and paste the snippet into your CI file.
4. **Run it**

   Commit and run the CI job.

   The log prints “SmartestQA: run started, suiteRuntimeId=…”, progress lines, then a summary with **Result**, counts and **[Report](https://docs.smartestqa.ai/en/report.html)**. The test tab shows each test case.

GitHub Actions step (generated, shortened)

```yaml
- name: 'SmartestQA · Nightly regression'
  env:
    SMARTEST_URL: 'https://api.example.com'
    SMARTEST_API_KEY: ${{ secrets.SMARTEST_API_KEY }}
    PIPELINE_ID: '318'
    SMARTEST_TIMEOUT_MIN: '60'
  run: |
    curl -fsSL "$SMARTEST_URL/install/smartest-ci.sh" -o smartest-ci.sh
    bash smartest-ci.sh
- name: SmartestQA test results
  if: always()
  uses: mikepenz/action-junit-report@v5
  with:
    report_paths: smartest-junit.xml
```

| CI tool | Where the key goes | Test report |
| --- | --- | --- |
| GitHub Actions | Repository secret `SMARTEST_API_KEY` | A check from `action-junit-report`, plus a summary table in the job summary |
| GitLab CI | Settings › CI/CD › Variables (masked) | `artifacts: reports: junit`, shown in the merge request |
| Jenkins | Manage Jenkins › Credentials, Secret text, id `smartest-api-key` | `junit` post step |
| Azure Pipelines | Secret pipeline variable | `PublishTestResults@2` |
| Shell / other CI | Your CI’s secret store, exported as `SMARTEST_API_KEY` | `smartest-junit.xml` in the work folder |

Windows agents use `smartest-ci.ps1` instead; the Jenkins and Azure snippets include the PowerShell line as a comment.

## Script settings

| Variable | Default | Meaning |
| --- | --- | --- |
| `SMARTEST_URL` | — | SmartestQA API address |
| `SMARTEST_API_KEY` | — | [Agent API key](https://docs.smartestqa.ai/en/agent-pool-agents.html#api-key), sent as `Authorization: Bearer` |
| `PIPELINE_ID` | — | The [pipeline](https://docs.smartestqa.ai/en/schedule-create.html)’s number |
| `SMARTEST_PARAMS` | none | JSON object of parameter overrides |
| `SMARTEST_TIMEOUT_MIN` | 60 | How long the step waits |
| `SMARTEST_POLL_SEC` | 15 | Seconds between status checks |
| `SMARTEST_JUNIT` | `smartest-junit.xml` | Report file; `-` writes none |

## Run states and exit codes

| State | Meaning | Exit code |
| --- | --- | --- |
| `QUEUED` | The run is waiting to start | — (keeps waiting) |
| `RUNNING` / `STOPPING` | Test cases are running, or a stop was requested | — (keeps waiting) |
| `PASSED` | Every test case passed | 0 |
| `FAILED` | At least one test case failed, or a stopped run left test cases unrun | 1 |
| `DISCONNECTED` | A test case has been running for more than 2 hours without news: [the agent](https://docs.smartestqa.ai/en/agent-pool-agents.html) connection was lost | 1 |
| Timeout or error | The step’s timeout passed, the trigger was refused, or the status could not be read 5 times in a row | 2 |

On a timeout the run keeps going in SmartestQA; only the CI step stops waiting. In the JUnit file each test case is one `<testcase>` named *name* (#*id*): failed and disconnected cases are failures with the first error, cases that did not run are skipped, and cases still running at the timeout are errors.

## Related

- To tell SmartestQA that a deploy happened (so it can react), see **[Deployments & Repos](https://docs.smartestqa.ai/en/deployments.html)**.
- To start a [pipeline](https://docs.smartestqa.ai/en/schedule-create.html) from another system without waiting, use a webhook trigger in **Schedule**.
- To announce the result in chat, add [Slack, Teams or WhatsApp](https://docs.smartestqa.ai/en/integrations-notifications.html).

## If something goes wrong

- [“trigger failed (HTTP 403)” or another code](https://docs.smartestqa.ai/en/integrations-troubleshooting.html#fix-ci-trigger)
- [“SMARTEST_API_KEY is not set” / “PIPELINE_ID must be a number”](https://docs.smartestqa.ai/en/integrations-troubleshooting.html#fix-ci-not-set)
- [“timed out after 60 min; the run continues in SmartestQA.”](https://docs.smartestqa.ai/en/integrations-troubleshooting.html#fix-ci-timeout)
- [“status endpoint unreachable”](https://docs.smartestqa.ai/en/integrations-troubleshooting.html#fix-ci-unreachable)
- [“Enter a JSON object, for example {"ENV":"uat"}.”](https://docs.smartestqa.ai/en/integrations-troubleshooting.html#fix-ci-params)
