- Docs
- Administration
- Integrations
- Xray & Zephyr Scale
Xray and Zephyr Scale
Xray and Zephyr Scale keep your test cases in Jira. These integrations exchange test cases with SmartestQA and carry run results back, so the test plan in Jira shows what automation found.
In short
On this page you can
- Send SmartestQA test cases to Xray Cloud as Tests, and pull Xray Tests in.
- Bring Zephyr Scale test cases in as planned scenarios, folder by folder.
- Turn the Jira issues linked to Zephyr cases into requirements in Analyze.
- Write every pipeline run to a Zephyr test cycle with your pass and fail statuses.
- See and remove the mappings between SmartestQA and Xray tests.
How to use it
- Create the Xray API key or the Zephyr access token in Jira.
- Click Setup, enter the key or token and run Test Connection.
- Enter the Jira project key and choose which directions to sync, then save.
- Click Send to Xray / Get from Zephyr for the first sync.
- Run a pipeline and open the Zephyr cycle or the Xray execution.
| Xray | Zephyr Scale | |
|---|---|---|
| Editions | Xray Cloud | Jira Cloud (US or EU region) and Jira Data Center |
| Credentials | Client ID and Client Secret | Zephyr API access token (Cloud) or Jira personal access token (Data Center) |
| Test cases in | Pull from Xray, up to 100 tests | Get from Zephyr, up to 5,000 cases, as planned scenarios |
| Test cases out | Send to Xray: every test case | Send to Zephyr: automated test cases written in SmartestQA |
| Requirements | No | Linked Jira issues become requirements |
| Run results | Exported on request (see results) | Automatically after each pipeline |
Xray#
Before you start#
- In Xray Cloud, open Global Settings › API Keys and create a key for a technical user. You get a Client ID and a Client Secret. The dialog links to How to get an API key?
- Note the key of the Jira project where Xray Tests should live, for example
PROJ.
Set up Xray#
Connection Settings
Click Setup. In Xray Integration Configuration keep Xray Cloud URL at
https://xray.cloud.getxray.app, enter Client ID and Client Secret, and click Test Connection.Connection Successful. Next works only after this.
Project Selection
Enter the Jira Project Key, choose the Default Test Type (Manual, Cucumber (BDD) or Generic) and, if you like, a Default Folder such as
/SmartestQA Tests.Sync Settings
Under Sync Direction choose Test Case → Xray, Xray → Test Case and Execution Results. Click Save.
“Settings saved successfully”, and the page shows Active.
First sync
Click Send to Xray (“Synchronization started. Test cases are being sent to Xray...”) or Pull from Xray. The mappings appear in the Mappings table.
What each sync does#
| Button | What happens |
|---|---|
| Send to Xray (SmartestQA → Xray) | Every test case of the project goes to Xray. A new one becomes a Test named [SmartestQA-TCid] name of the default type, with one Xray step per SmartestQA step (the step’s action name; data and expected result stay empty). A mapped one is updated. |
| Pull from Xray (Xray → SmartestQA) | Reads up to 100 Tests of the Jira project. Tests that SmartestQA created are skipped. A new Test becomes a test case with status New; a mapped one gets its name updated, and its description only if the test case had none. The message reports “Created” and “Updated” counts. |
A button that is off in the settings shows “This feature is disabled in settings”. Remove Mapping in the table unlinks one test case; the next Send to Xray creates a new Test for it.
Test execution results#
The Test Execution Results panel describes how results reach Xray: results of mapped test cases are sent as a Test Execution, with PASSED, FAILED, EXECUTING or TODO per test. Only test cases mapped to Xray are included, so sync them first.
In the version checked for these docs, an export runs when it is requested: an AI assistant connected through MCP exports a run with smartest_integrations (system xray, action export, by pipeline, run or test log). The panel’s Automatic Export line shows whether Execution Results is on; it does not start an export by itself.
| SmartestQA result | Xray status |
|---|---|
| Passed (passed, success, completed) | PASSED |
| Failed (failed, error) | FAILED |
| Running | EXECUTING |
| Anything else | TODO |
The execution comment lists up to three error messages. A run with no mapped test case returns “No Xray mappings found for tests in this log”.
No timed sync for Xray
The dialog stores Automatic Synchronization and Sync Interval (minutes) (15 to 1440), and the page shows them, but this version runs no background sync for Xray. Use the two buttons after you change test cases.
Remove#
Remove asks “Are you sure you want to remove the Xray integration? This will delete all mappings.” Tests in Xray and test cases in SmartestQA stay.
Zephyr Scale#
Before you start#
| Where Zephyr runs | Token | Also needed |
|---|---|---|
| Jira Cloud | Zephyr API access token: Zephyr Scale › your profile › Zephyr API access tokens | The Zephyr region (US or EU). For requirements, this project’s Jira integration, because issues are read through it. |
| Jira Data Center (on-premises) | Jira personal access token: Jira profile › Personal Access Tokens. Read access to the project, write access to send results. | The Jira address, for example https://jira.company.local. Zephyr is read from /rest/atm/1.0 and issues from /rest/api/2. |
Set up Zephyr Scale#
Connect
Click Setup. In Zephyr Scale settings pick Where Zephyr runs, then the region or the Jira address, the Jira project key and the token. Click Test Connection.
Connected with the project name.
Choose what moves
Three switches, all on by default: Bring linked Jira issues to Analyze, Send test cases written in SmartestQA to Zephyr, Write pipeline results to Zephyr. Set Status for passed and Status for failed to the names in your Zephyr (Pass and Fail by default).
Save and import
Click Save, then Get from Zephyr on the page. “Started — this page updates when it finishes.” appears and the page shows the summary when the job ends.
A line such as “Last import · time — new 120 · updated 0 · unchanged 0 · requirements 34”, with Details when there are warnings or errors.
What each direction does#
| Direction | Rules |
|---|---|
| Get from Zephyr | Each Zephyr test case becomes a planned scenario tagged planned and zephyr. The first folder level becomes the module (Zephyr when a case has no folder). Descriptions are cut at 1,000 characters. Running it again only updates cases that changed. Cases that SmartestQA sent to Zephyr are skipped. |
| Requirements | With Bring linked Jira issues to Analyze on, each linked issue becomes a sub-requirement under its epic or parent (a main requirement), and the scenario is linked to it. |
| Send to Zephyr | Sends scenarios that have active steps and are not templates, not planned and not from Zephyr. They get the label SmartestQA. Test cases that came from Zephyr stay owned by Zephyr: their name and steps are never written back. |
| After each pipeline | With Write pipeline results to Zephyr on, linked test cases get a result in a new test cycle named SmartestQA · pipeline · yyyy-MM-dd HH:mm (Data Center: a test run). Each run is written once. |
Only one import or export runs at a time per project; a second click returns “Another sync is still running”. A job that has not finished after 45 minutes no longer blocks new ones.
The Zephyr page#
- KPIs: Test cases from Zephyr, Test cases sent to Zephyr, Requirements from Jira, Runs written to Zephyr. See how they are counted.
- Connection Status: Integration active and the three switches. They save as soon as you click them.
- Synchronization: Get from Zephyr, Send to Zephyr, Last import, Last export. Hidden with “Turn the integration on to sync.” while it is off.
- Results written to Zephyr: the last 20 runs with Pipeline, Results, Status and Retry on failed writes. “Writing results is turned off.” when the switch is off.
Remove asks “Remove the Zephyr integration?”: the connection and its settings are removed. Test cases and requirements that came from Zephyr stay in SmartestQA.
Best practices#
Do
- Decide which tool owns each test case before the first sync.
- Keep Zephyr folders by business area: they become modules.
- Set up Jira first when you use Zephyr Cloud requirements.
- Check the first Zephyr cycle after a pipeline before you rely on it.
Don’t
- Connect Xray and Zephyr to the same Jira project from one SmartestQA project.
- Rename Xray Tests that SmartestQA created: their prefix marks them.
- Change the status names in Zephyr without updating Status for passed.
If something goes wrong#
- Xray “Authentication failed”
- No Test Execution appears in Xray after a run
- “The API token was rejected” / “The token has no access to this project”
- “Requirements were not imported: Zephyr Cloud needs the Jira integration…”
- “No test case in this run is linked to Zephyr”
- “Another sync is still running”
Didn’t find your answer?Write to support with your project name and what you see on screen.
info@smartestqa.ai