---
title: "Dry run and live log"
section: "Automate"
description: "Dry run a SmartestQA test case: pick the run target, read the live log and screenshots, diagnose failures, and compare dry runs with pipeline runs."
url: https://docs.smartestqa.ai/en/automate-dry-run.html
audience: [customer, admin]
updated: 2026-11-08
keywords: "dry run run on server run on your agent live log test run progress screenshots video ai analyze device farm appium desktop agent run modes pipeline run agent parity failed error skipped heal warning"
---
# Automate: Dry run and live log

A dry run tries the saved test case once and shows every step live, with a screenshot, without writing a report. Use it while you build; put the test case in a pipeline when you want results you can keep.

## In short

**On this page you can**

- Run the saved test case once with Dry Run , without a report.
- Choose Run on Server or Run on your Agent for web tests.
- Pick a local device or a Device Farm device for mobile tests.
- Follow every step live, with a screenshot per step.
- Diagnose a failed step from its message, locator and screenshot.

**How to use it**

1. Click Save , then Dry Run next to it.
2. Choose the run target and start.
3. Watch the Test Run panel until it says Completed .
4. Open the first red step: read the reason and look at its screenshot.
5. Fix the step, save and dry run again; then add the test case to a pipeline.

## Before you start

- **Save first.** **Dry Run** is disabled while the test case has unsaved changes (“Please save the test case before running”). The dry run runs the saved test case.
- **[Python steps](https://docs.smartestqa.ai/en/automate-steps-code.html) need code for your [engine](https://docs.smartestqa.ai/en/project-settings-test-management.html#engine).** Otherwise: “*Selenium* code is empty for Python step(s): …. Dry Run was not started.”
- **Something to run on.** Web needs nothing extra for [**Run on Server**](https://docs.smartestqa.ai/en/quick-start-connect.html#agent); your [own agent](https://docs.smartestqa.ai/en/agent-pool-agents.html) for [**Run on your Agent**](https://docs.smartestqa.ai/en/quick-start-connect.html#agent). Mobile needs a device through your agent or [Device Farm](https://docs.smartestqa.ai/en/agent-pool.html). Desktop needs the desktop agent on [Windows](https://docs.smartestqa.ai/en/agent-pool-local-setup.html#windows-desktop) or [macOS](https://docs.smartestqa.ai/en/agent-pool-local-setup.html#macos-desktop). See [Do you need an agent?](https://docs.smartestqa.ai/en/quick-start-connect.html#agent)

## Start a dry run

1. **Click Dry Run**

   Click **Dry Run**, just left of **Save** at the bottom of the editor.
2. **Choose the run target**

   The **Run target** dialog depends on the project type; see the sections below.
3. **Start**

   Confirm the target.

   “Test execution started successfully” and the **Test Run** panel opens with the live log.

![Choosing the run target in a web project.](https://docs.smartestqa.ai/media/shots/ATM-SHOT-06.png)

### Web projects

| Target | What it does | Use it when |
| --- | --- | --- |
| [**Run on Server**](https://docs.smartestqa.ai/en/quick-start-connect.html#agent) | “Runs headless on the server. No agent needed.” | Your app is reachable from the internet and you only need the log and screenshots. |
| [**Run on your Agent**](https://docs.smartestqa.ai/en/quick-start-connect.html#agent) | “Runs on your computer; you can watch the browser.” Disabled when your agent is [offline](https://docs.smartestqa.ai/en/agent-pool-numbers.html#agent-online): “Your agent is offline. Start the agent app to use this option.” | You want to watch, your app is behind a VPN or firewall, or a step needs your network ([database](https://docs.smartestqa.ai/en/automate-steps-database.html#run-time), [mail server](https://docs.smartestqa.ai/en/automate-steps-mail.html)). |

The dialog selects your agent when it is online, otherwise the server. Its footer reminds you: “Dry Run results are not saved to [reports](https://docs.smartestqa.ai/en/report.html).” A dry run on your agent always opens a visible browser; on the server it is always headless.

### Mobile projects

- **Dry Run (local)**: “Runs on a local device/emulator on this machine.” Choose a device under **Pick a device** or **Automatic (connected device)**. Under **Application** you can override the app package and activity for this run only: “Applies to this run only — not saved to the project settings.”
- [**Device Farm**](https://docs.smartestqa.ai/en/agent-pool.html#screen-tour): a SmartTest Device Farm device, [when your company has access to it](https://docs.smartestqa.ai/en/agent-pool.html#request-access). **Control the device** lets you tap on the device during the run (“Taps you make while the run is in progress can affect the test result.”).
- [Appium must be running](https://docs.smartestqa.ai/en/agent-pool-local-setup.html#android-emulator) for local runs: “Appium server not found — Appium server must be running to execute mobile tests. Open a new terminal and run "appium" to start it.”

The device is chosen when you run, not stored in the project. Pipelines [choose theirs in Schedule](https://docs.smartestqa.ai/en/schedule-where.html#mobile).

### Desktop projects

Desktop tests run only on an agent on the computer with the application, [Windows](https://docs.smartestqa.ai/en/agent-pool-local-setup.html#windows-desktop) or [macOS](https://docs.smartestqa.ai/en/agent-pool-local-setup.html#macos-desktop). Choose the agent under **Device (agent)**: **This computer** or a **Remote device**; offline agents are marked. With none installed: “No SmartTest-Desktop agent found. Install and start the desktop app on the computer where the application runs.”

## Read the live log

| Part | What it tells you |
| --- | --- |
| Status chip | **Live** while the run sends updates, **Completed** at the end, **Disconnected** when the connection to the live log dropped. |
| Progress | “Progress: *p*%” and “*c* / *t* steps completed”. See [how they are counted](https://docs.smartestqa.ai/en/automate-numbers.html#live-progress). |
| Row 0 | “Open *start page URL*”: opening the [start page](https://docs.smartestqa.ai/en/automate-editor.html#environment) (web). |
| Step rows | One per active step, in order, with a status badge, its messages and duration. Click a row to expand its details. |
| **Show screenshot** | The screenshot taken after the step. Dry runs take one for every step. |
| **Watch screenshots as video** | The screenshots in order as a slideshow (**Video**). |
| Result banner | “Test completed successfully!” or “Test failed”. |

| Badge | Meaning |
| --- | --- |
| PENDING | Not reached yet. |
| RUNNING | Running now. |
| SUCCESS | Done and passed. |
| HEAL | The control was not found with its locators; [auto-heal is looking for it](https://docs.smartestqa.ai/en/maintenance-how-healing-works.html#stages). |
| WARNING | Passed with a warning (for example a [network step whose error rule is “warn”](https://docs.smartestqa.ai/en/automate-steps-network.html#check)). Does not fail the test. |
| ERROR | The step did not pass. The live log shows both failed checks and execution errors as ERROR. |
| SKIPPED | Not run, because the test [stopped at an earlier step](https://docs.smartestqa.ai/en/automate-steps.html#stop-or-continue). |

Messages follow the project’s locale (English, Turkish or German; other locales get English). Very long messages are cut at 4,000 characters.

### Closing the panel

A dry run cannot be stopped, not even by an [AI assistant](https://docs.smartestqa.ai/en/mcp-tools.html). Closing the panel asks “Close panel?”: “Test execution will continue running in the background. Are you sure you want to close only this live log panel?” Click **Yes, close** to close only the panel.

Video: [Dry run and read the live log](https://docs.smartestqa.ai/media/videos/ATM-03.mp4) (2:30)

Dry run a web test on the server, open a failed step, look at its screenshot, fix the value and run again.

Video transcript: A test case is a list of steps. Before it goes into a pipeline, try it with a dry run. Dry Run asks where to run: on the server, headless with nothing to install, or on your own agent. Dry run results are not saved to reports. The Test Run panel follows the run live: progress, and a status for every step. When it ends, every step shows its result.

## Diagnose a failed step

1. **Open the first red step**

   Later steps are usually SKIPPED because of it. Expand the row.
2. **Read the message**

   Error messages have the form `<what failed>: <reason> | Locator: by=value | Detail: …`.
3. **Look at the screenshot**

   Was the page the one you expected? Was a dialog, cookie banner or spinner in the way?
4. **Fix and rerun**

   Change the value, the wait, the order or the locator ([Action Detail](https://docs.smartestqa.ai/en/automate-editor.html#action-detail)), save and dry run again.

| Reason in the message | Usually means | Try |
| --- | --- | --- |
| [Element not found](https://docs.smartestqa.ai/en/automate-steps-troubleshooting.html#fix-element-not-found) | The locator no longer matches, or the page was not the expected one | Check the screenshot; add a stable `id` or `name` locator in Action Detail |
| Timeout — element did not become ready in time | The page is slow, or something covers the control | Add a [wait](https://docs.smartestqa.ai/en/automate-steps-flow.html#waits) or a [visibility check](https://docs.smartestqa.ai/en/automate-steps-browser.html#visibility-check) before the step |
| Element not clickable / not interactable | An overlay, a disabled control, or the control is off screen | Close the overlay with a step; check the [step order](https://docs.smartestqa.ai/en/automate-editor.html#reorder) |
| Element is no longer valid on the page (stale) | The page redrew the control | Usually retried automatically; add a short wait if it persists |
| Frame not found | The control is inside an iframe that is not there | Check the page state in the screenshot |
| Invalid selector | A typo in a locator | Fix it in Action Detail |
| Browser window/tab is not available | The tab was closed or the browser crashed | Check the step before it |

A failed **check** (an [assertion](https://docs.smartestqa.ai/en/automate-steps-assertions.html), a visibility check, a [network check](https://docs.smartestqa.ai/en/automate-steps-network.html)) and an execution **error** (an exception, an API, database or mail problem, a configuration mistake) both show as ERROR in the live log. Assertions stop the test; a visibility check that fails marks its step and the test goes on. Each step type’s failure behaviour is in the [step reference](https://docs.smartestqa.ai/en/automate-steps.html).

### AI Analyze

On a failed step of a dry run on your [agent](https://docs.smartestqa.ai/en/agent-pool-agents.html), **AI Analyze** inspects the live page and suggests locators. For this the agent keeps the browser open for up to 15 minutes after the failure. On the server the browser is always closed, so the analysis cannot read the page. It needs AI turned on for the project; see [Turn on AI](https://docs.smartestqa.ai/en/automate-ai.html#enable-ai).

> **Warning: A dry run can repair locators for good**
>
> When all locators of a control fail, [auto-heal looks for the control](https://docs.smartestqa.ai/en/maintenance-how-healing-works.html). If it finds it, the new locator is written to the action, used by every test case, and a review is created in Maintenance. The editor reloads the test case when you close the panel. Review healed actions in [**Maintenance**](https://docs.smartestqa.ai/en/maintenance-auto-heal.html#review-a-repair).

## What a dry run sends

- The saved test case, with [inactive steps](https://docs.smartestqa.ai/en/automate-steps-flow.html#inactive-steps) left out and [templates](https://docs.smartestqa.ai/en/automate-steps-flow.html#template) expanded into their steps.
- The values stored in the steps; for [datasets](https://docs.smartestqa.ai/en/data-source-datasets.html#use-in-steps), the first row. [Pipeline parameter](https://docs.smartestqa.ai/en/schedule-settings.html#pipeline-parameters) overrides are not used.
- The test case’s [environment and start page](https://docs.smartestqa.ai/en/automate-editor.html#environment); the project’s [browser, engine and timeout settings](https://docs.smartestqa.ai/en/project-settings-test-management.html).
- [Retry count](https://docs.smartestqa.ai/en/project-settings-test-management.html#test-settings) 0: a failed dry run is not retried.

## Run modes: dry run, pipeline, agent

A test case can run in two modes. An **agent** is not a third mode but a place where either mode runs. Every step type is built to work in all three situations: dry run, pipeline run on the server, and runs on an agent.

|  | Dry run | Pipeline run |
| --- | --- | --- |
| Started from | **Dry Run** in Automate | A [pipeline in Schedule](https://docs.smartestqa.ai/en/schedule-create.html): [by hand](https://docs.smartestqa.ai/en/schedule-runs.html#run-now), [on a timer](https://docs.smartestqa.ai/en/schedule-timing.html), or [from CI](https://docs.smartestqa.ai/en/integrations-cicd.html) |
| Runs on | Web: the server or your own agent. Mobile: a local device or Device Farm. Desktop: a desktop agent. | The [server](https://docs.smartestqa.ai/en/schedule-where.html#server), [any agent the pipeline names](https://docs.smartestqa.ai/en/schedule-where.html#agent), a [self-hosted runner](https://docs.smartestqa.ai/en/schedule-where.html#server-choice), or [Device Farm](https://docs.smartestqa.ai/en/schedule-where.html#mobile) |
| Live log | Yes | Yes |
| Report, All Logs, screenshots kept | No. Screenshots are deleted after 1 hour. | Yes, [stored with the report](https://docs.smartestqa.ai/en/report-steps.html#screenshots) |
| [Tree status icon](https://docs.smartestqa.ai/en/automate-numbers.html#status-icon), [**Recent runs**](https://docs.smartestqa.ai/en/automate-editor.html#test-case-detail) | Not updated | Updated |
| Dataset rows | First row only | One run per row when the [execute count](https://docs.smartestqa.ai/en/schedule-settings.html#data-loop) asks for it |
| [Pipeline parameter values](https://docs.smartestqa.ai/en/schedule-settings.html#pipeline-parameters) | Not used | Used |
| Retries | None: Retry Count is always 0 | The pipeline’s own [Retry Count](https://docs.smartestqa.ai/en/schedule-settings.html#test-management), copied from the project when the pipeline was created ([details](https://docs.smartestqa.ai/en/project-settings-test-management.html#how-settings-reach-runs)) |
| Can be stopped | No | Yes |
| [Auto-heal](https://docs.smartestqa.ai/en/maintenance-how-healing-works.html#run-modes) | Yes | Yes |

> **Tip: When a test passes in a dry run but fails in a pipeline**
>
> Compare what differs: the pipeline’s parameter values, the [dataset row](https://docs.smartestqa.ai/en/data-source-datasets.html#loop), the run location (server or agent), the [environment](https://docs.smartestqa.ai/en/schedule-where.html#environment), and whether an earlier test case in the pipeline changed the data. A step that reaches a database or mail server works only where that server can be reached.

## Limits

| Item | Limit |
| --- | --- |
| Dry-run screenshots | Kept 1 hour (“Screenshot expired (kept for 1 hour).”) |
| Live-log buffer per run | 5,000 lines or 2 million characters, kept 30 minutes; 50 runs per project |
| One live-log message | 4,000 characters |
| Server queue | 20 waiting runs; beyond that “The test server queue is full. Please try again in a moment.” |

## If something goes wrong

- [Dry Run is disabled](https://docs.smartestqa.ai/en/automate-troubleshooting.html#fix-dry-run-disabled)
- [“To perform a Dry Run, you need to have the Agent installed locally…”](https://docs.smartestqa.ai/en/automate-troubleshooting.html#fix-agent-not-found)
- [“Your agent appears to be offline…”](https://docs.smartestqa.ai/en/automate-troubleshooting.html#fix-agent-offline)
- [“The test server queue is full…” or “…not reachable right now”](https://docs.smartestqa.ai/en/automate-troubleshooting.html#fix-server-busy)
- [“Appium server not found”](https://docs.smartestqa.ai/en/automate-troubleshooting.html#fix-appium)
- [The live log says Disconnected](https://docs.smartestqa.ai/en/automate-troubleshooting.html#fix-live-disconnected)
- [“Screenshot expired (kept for 1 hour).”](https://docs.smartestqa.ai/en/automate-troubleshooting.html#fix-screenshot-expired)
- [The tree icon did not change after a dry run](https://docs.smartestqa.ai/en/automate-troubleshooting.html#fix-status-unchanged)
