---
title: "Can agents watch my test suite for me?"
section: "Use cases"
description: "Autonomous QA agents in SmartestQA: install Daily triage, Daily QA report or Coverage gap report, let them run on a schedule, and review what each run did, what it cost and what it proposes."
url: https://docs.smartestqa.ai/en/use-case-agents.html
audience: [customer, admin]
updated: 2026-11-12
keywords: "autonomous agents ai agents qa agent daily triage coverage gap report daily qa report weekly management report release readiness scheduled agent proposals human in the loop test monitoring ai test triage agents example morning check by hand summary script agent proposals review apply dismiss"
---
# Use cases: Can agents watch my test suite for me?

Yes. Autonomous agents read your runs, reports, defects and app map on a schedule and report what they found: Daily triage groups yesterday’s failures by cause, and Coverage gap report lists what is not tested. An agent run on the SmartestQA server never changes anything: every change waits as a proposal for a person to apply.

## In short

**On this page you can**

- Have yesterday’s failed tests grouped by cause before the team starts work.
- Get a daily or weekly report by e-mail, Slack or Teams.
- See what is not tested, in priority order.
- Read what each run did, step by step, with its tokens and cost.
- Apply or dismiss the changes an agent proposes.

**How to use it**

1. Open Agents and click Install new agent .
2. Pick an agent, check Trigger , Reports and Limits , and install it.
3. Switch it on with the toggle on its card and click Run now once.
4. Open the run on the Runs tab and read Summary, Outcome and the steps.
5. Click Apply or Dismiss on each proposal.

## The short answer

Agents come after the order of work. Once you have tests and pipelines, an agent keeps watching them so nobody has to check every morning by hand. Each agent has instructions, a set of tools it may call, a trigger and limits. It runs on a schedule, after a deploy, or when you click [**Run now**](https://docs.smartestqa.ai/en/ai-agents-runs.html#run-now), and it leaves a run you can read from start to finish.

## The problem this solves

A suite that runs every night produces results every morning, and somebody has to read them. In many teams one person opens the night’s runs, looks at each red test, sorts them by cause and writes a summary in the team chat. When that person is away, the reds pile up.

Some teams write a script that posts the pass and fail counts to a channel. That tells the team that something is red, not why, and not which failures belong together.

## An example

This is an illustration, not a customer story. A team tests the web application of a delivery service with pipelines that run every night.

1. The QA lead opens **Agents**, clicks [**Install new agent**](https://docs.smartestqa.ai/en/ai-agents-catalog.html#install) and picks [**Daily triage**](https://docs.smartestqa.ai/en/ai-agents-catalog.html#daily). The QA lead checks **Trigger**, **Reports** and **Limits** and installs it. The message is “Agent created as paused. Review the settings, then activate it.”
2. The QA lead switches the agent on with the toggle on its card. The next day at 06:00 it runs: it groups the red test cases since the previous day by the same error on the same page, classifies each group and names the next action. It writes nothing.
3. A tester opens the run on the **Runs** tab and reads **Summary** and **Outcome**: seven red test cases in two groups, one classified as a locator change and one as a real bug.
4. The team also installs Library maintenance, which proposes tags and fixes, switches it on and clicks [**Run now**](https://docs.smartestqa.ai/en/ai-agents-runs.html#run-now). The run starts on the SmartestQA server. The server records each change the agent wants as a PROPOSAL step instead of making it, and the run shows “2 proposals are waiting. An applied proposal runs as you, with the same arguments.”
5. The tester opens each proposal, reads what will be sent, clicks **Apply** on one and **Dismiss** on the other. The applied call runs as the tester, with the tester’s permissions.

The morning sorting is done before the team starts work, and nothing in the test library changed without a person clicking **Apply**.

## By hand, with your own summary script, and with SmartestQA

| Question | A person checks every morning | Your own script that posts a summary | SmartestQA |
| --- | --- | --- | --- |
| Who reads the night’s failures? | One person, when there is time. Fine for a small suite | The script counts them; a person still reads them | [**Daily triage**](https://docs.smartestqa.ai/en/ai-agents-catalog.html#daily), every day at 06:00 |
| Are failures with the same cause grouped? | Yes, by the person’s judgement | Only as far as the script was written to do it | Red test cases are grouped by the same error on the same page, and each group is classified |
| Can it change my tests? | Yes, the person can | Only if you write that into the script | Not in a run on the SmartestQA server: every change is recorded as a proposal until a person clicks **Apply** |
| Can I see what it did? | Ask the person | Read the script’s log | Each run lists its steps, call by call, with duration, tokens and cost |
| What does it cost to run? | The person’s time each morning | The time to write and maintain the script | AI usage, capped per agent by [**Max steps per run**](https://docs.smartestqa.ai/en/ai-agents-settings.html#limits), [**Max cost per run (USD)**](https://docs.smartestqa.ai/en/ai-agents-settings.html#limits) and [**Daily budget (USD)**](https://docs.smartestqa.ai/en/ai-agents-settings.html#limits) |

## Steps

1. **Open the catalog**

   Open **Agents** in the main menu (the page title is [**Autonomous Agents**](https://docs.smartestqa.ai/en/ai-agents.html#screen-tour)) and click [**Install new agent**](https://docs.smartestqa.ai/en/ai-agents-catalog.html#install). Agents are grouped by rhythm: after each deploy, daily, weekly and on demand. See [The catalog](https://docs.smartestqa.ai/en/ai-agents-catalog.html#which-agent).
2. **Pick an agent**

   For a suite that runs every night, start with [**Daily triage**](https://docs.smartestqa.ai/en/ai-agents-catalog.html#daily): every day at 06:00 it groups every red test case since the previous day by the same error on the same page, classifies each group and names the next action. [**Coverage gap report**](https://docs.smartestqa.ai/en/ai-agents-catalog.html#weekly) runs every Monday at 07:00 and lists untested points on the app map and sub-requirements without test cases.
3. **Check the settings and install**

   The settings dialog opens with the agent’s defaults. Look at least at **Trigger** (when it runs and in which time zone), **Reports** (who gets the result) and **Limits**. Click Install.

   “Agent created as paused. Review the settings, then activate it.” The card appears on the Agents tab.
4. **Try it once**

   A paused agent cannot be run. Switch it on with the toggle on its card and click [**Run now**](https://docs.smartestqa.ai/en/ai-agents-runs.html#run-now). If the result is useful, leave it on; its trigger takes over.
5. **Read the run**

   Open the **Runs** tab and click the run. The top row shows the status, trigger, start, duration, steps, tokens, cost and the model. **Summary** is the agent’s conclusion in plain words; **Outcome** is the full result, as a report or as JSON. The steps list what the agent did, call by call. See [Read a run](https://docs.smartestqa.ai/en/ai-agents-runs.html#run-detail).
6. **Apply or dismiss proposals**

   When a run would change something, such as a locator, a defect, a tag or a test case, the server records the exact call as a proposal instead of making it. **Apply** runs it now, as you, with your permissions. **Dismiss** drops it. See [Apply or dismiss proposals](https://docs.smartestqa.ai/en/ai-agents-runs.html#proposals).

## Watch it

The video shows the review side: the four numbers at the top of Autonomous Agents, the Runs tab, and one run with its top row, Summary, Outcome and steps. It does not show an agent being installed.

Video: [Review an agent run](https://docs.smartestqa.ai/media/videos/AGT-03.mp4) (0:42)

What an autonomous agent did, what it cost, and what it concluded.

Video transcript: Autonomous Agents starts with four numbers: active agents, runs today, proposals to review and spend. The Runs tab lists every run with its agent, status, trigger, duration, steps and cost. Click a run to open its detail. The top row shows trigger, start, duration, steps, tokens, cost and the model that ran. Summary is the agent's conclusion in plain words. Outcome holds the full result, as a report or as JSON. Steps lists what the agent did, call by call. Proposed changes would wait here for you to apply or dismiss.

## What you get

- **Failures explained every morning.** Daily triage classifies each group of failures, for example as a real bug, a locator change, test data, the environment or flaky. It writes nothing.
- **The same picture for everyone.** Daily QA report and Weekly management report go to e-mail with a PDF, Slack, Teams, Discord or a webhook.
- **What is not tested.** Coverage gap report lists the gaps in priority order.
- **Checks after a deploy.** Deploy agents look for a new deploy every hour and run only when one has arrived.
- **Control.** New agents start paused. An agent can call only the tools ticked in its settings; anything else is refused and logged.
- **Limits.** Each agent has [**Max steps per run**](https://docs.smartestqa.ai/en/ai-agents-settings.html#limits), [**Max cost per run (USD)**](https://docs.smartestqa.ai/en/ai-agents-settings.html#limits) and [**Daily budget (USD)**](https://docs.smartestqa.ai/en/ai-agents-settings.html#limits); the run stops at whichever comes first.
- **A full record.** The Runs tab keeps the latest 100 runs of the project with status, trigger, duration, steps, proposals and cost, and each run shows the settings it used.

## Questions and answers

### Q: Can an agent change my tests without asking?

Not when the SmartestQA server runs it. Every call that would change something is recorded as a proposal, whatever the agent’s autonomy setting says, and nothing changes until a person clicks **Apply**. See [Suggest only or Apply automatically](https://docs.smartestqa.ai/en/ai-agents-settings.html#autonomy).

_Audience: customer, admin_

### Q: Which agent should I install first?

The catalog marks the ones SmartestQA recommends first as **Core**: Deploy verification, Deploy impact analysis, Daily triage, Daily QA report and Coverage gap report. Daily triage and the two reports need no repository. See [The catalog](https://docs.smartestqa.ai/en/ai-agents-catalog.html#which-agent).

_Audience: customer, admin_

### Q: How do I keep the AI cost of an agent under control?

With its limits. A run stops at [**Max steps per run**](https://docs.smartestqa.ai/en/ai-agents-settings.html#limits) or [**Max cost per run (USD)**](https://docs.smartestqa.ai/en/ai-agents-settings.html#limits), and when the agent has spent its [**Daily budget (USD)**](https://docs.smartestqa.ai/en/ai-agents-settings.html#limits) the next run is recorded as Budget blocked and does not start. See [Limits](https://docs.smartestqa.ai/en/ai-agents-settings.html#limits).

_Audience: customer, admin_

### Q: Can I write my own agent?

Yes. A [**Custom agent**](https://docs.smartestqa.ai/en/ai-agents-catalog.html#custom) takes its instructions in plain words in five boxes, starting with [**What should the agent do?**](https://docs.smartestqa.ai/en/ai-agents-settings.html#easy-mode). Safe rules such as “Never delete anything” are ticked by default. See [Instructions](https://docs.smartestqa.ai/en/ai-agents-settings.html#instructions).

_Audience: customer, admin_

### Q: Where does an agent run?

On the SmartestQA server with the project’s AI connection, or as a scheduled task in Claude Desktop. Some agents work on the server only. See [Runs on](https://docs.smartestqa.ai/en/ai-agents-settings.html#runs-on).

_Audience: customer, admin_

### Q: How does an agent know which pipeline is my regression suite?

By tag, not by name. Agents find pipelines by the tags `smoke`, `regression`, `api` and `performance` and do not touch your other pipelines. See [Tag a pipeline](https://docs.smartestqa.ai/en/schedule-create.html#tags).

_Audience: customer, admin_

### Q: Who can use agents?

Every role, in projects they can open. An agent can be refused tools that its user’s role has no permission for. See [Autonomous Agents](https://docs.smartestqa.ai/en/ai-agents.html).

_Audience: customer, admin_

## Read more

- [Autonomous Agents](https://docs.smartestqa.ai/en/ai-agents.html): what an agent is and how a run works.
- [Install an agent](https://docs.smartestqa.ai/en/ai-agents-catalog.html): every agent of the catalog and when it runs.
- [Agent settings](https://docs.smartestqa.ai/en/ai-agents-settings.html): instructions, tools, trigger, reports and limits.
- [Runs and proposals](https://docs.smartestqa.ai/en/ai-agents-runs.html): read a run, apply or dismiss what it proposes.
- [How do I build, run and schedule a regression suite?](https://docs.smartestqa.ai/en/use-case-regression.html): the pipelines the agents watch.
