1. Docs
  2. Build tests
  3. Report
  4. FAQ, MCP & limits

FAQ, MCP & limits

Answers to common questions about Report, how AI assistants use reports, and the limits that apply.

5 min read Updated CustomersAdmins

In short

On this page you can

  • Find out how long reports, logs and screenshots are kept.
  • Learn who can see, stop, export and share runs.
  • Understand why counts include retries and iterations.
  • Let an AI assistant read runs, logs and screenshots through MCP.
  • Check every limit: rows, logs, PDF images, AI input size.

How to use it

  1. Filter the questions by your role with the buttons above the list.
  2. Open a question; follow its link for the full procedure.
  3. For assistants, connect one and try the example prompts in the MCP section.
  4. Check the limits table before you plan very large runs.

Questions and answers#

Why is my dry run not in Report?Customers

A dry run only streams a live log, which is not stored. Reports come from pipeline runs. Add the test case to a pipeline in Schedule and run it. See the report record and the live log.

How long are reports kept?Customers

Reports, step logs and screenshots are not deleted automatically. Two things expire: the server error logs behind Last 5 Error Logs after 5 days, and dry-run screenshots after 60 minutes (those never reach Report anyway).

Can we delete a report?Admins

Not from the project, not even as Admin. If a run captured data it should not have, write to info@smartestqa.ai with the project and report ID. Removing a report also removes its steps, screenshots and load-test results.

How do I send a report to someone without an account?Customers

Use Share Report: they get an email with the PDF. There is no public web link; a /report/<id> address only opens for people with access to the project.

Can I email a report to several people at once?Customers

The Share dialog takes one address per send. For a team, set the pipeline’s automatic email with several addresses in Email To separated by ;, Always or On Failure. See Automatic emails.

Who can see, stop, export and analyze runs?Admins

Everyone with access to the project: Admins for every project of the company, Users for the projects they were given. There is no separate report permission; both roles can stop runs, export, share, tag and run AI analysis. Changing the AI options needs an Admin (Project Panel). See roles.

Why does the Result say 5/6 when the pipeline has 3 test cases?Customers

The chip counts attempts: each data iteration and each automatic retry is one. Three test cases with a 3-row data source on one of them and one retry make six attempts. See Result chip.

Can I rerun only the failed test cases from the report?Customers

Report has no rerun button. Run the pipeline again in Schedule, or put the failed test cases in a pipeline of their own. Automatic retries are set in the pipeline’s retry count.

Which language is the PDF in?Customers

Your interface language when you click Export PDF or Share Report: English, Turkish, German, Spanish or Arabic. Automatic emails after a run are always in English.

Does AI analysis use credits or a quota?Admins

There is no separate quota for it. Each analysis is a call to the project’s AI account (SmartestQA’s, or your own with Own Account) and is logged with its token use in the AI usage of the project. Automatic analysis runs once per failed run.

What does the AI see?Admins

For a failed step: its log, its screenshot and the test case name. For a run review: all test cases and steps with logs, descriptions and AI notes, with e-mails, card numbers, IDs, phone numbers and tokens masked unless Share Sensitive Data is on. See What is sent.

The AI says Real Bug but it is our test. What now?Customers

A verdict is advice; it changes nothing by itself. Read the Evidence line and the screenshot, fix the test, and run again. Clear test case descriptions also help the run review tell expected from unexpected behaviour.

Why does my mobile run have a real video and my web run only Create Video?Customers

AWS Device Farm and BrowserStack record the device and deliver the video. Other runs keep screenshots, and your browser turns them into a video on demand. See Video.

For AI assistants (MCP)#

An assistant connected through the SmartestQA MCP server reads Report with the smartest_runs tool:

ActionReturns
listThe Reports list, newest first (optionally only failed).
reportOne run per test case, by report ID or run ID.
logOne test case run step by step: the failed step, locators and the failure screenshot.
case_history, pipeline_historyPass and fail across runs of one test case or pipeline.
active, runnersWhat runs now and runner health.
stop, mailStop a run, or email a report to an address you give. Both ask you first.

Connect an assistant from Device FarmTest AgentsConnect to Claude; see Discover › MCP.

Example prompts
Why did last night's "Nightly regression" run fail? Show me the failed steps and their screenshots.

Which test cases in project "Customer Portal" failed in more than two of the last ten runs?

Read the failed step of report #482, tell me whether it is a product bug or a test problem, and fix the test if it is the test.

Limits#

ItemLimit
Rows per page in the Reports list10 · 25 · 100
List refresh while a run is goingevery 2 seconds
Running or New before a run shows as Disconnected2 hours
Test cases per page in the Summary table25
Failure reasons loaded per page / length10 rows · 400 characters
Step log text storedabout 8,000 characters
Server error logs (Last 5 Error Logs, View Logs)kept 5 days
Recipients per Share Report1
PDF step-table thumbnails / image size60 images · 4 MB each
Video built from screenshots1 second per screenshot, not stored
Screenshot sent with a failure analysisup to 4 MB
AI Report Feedback input100,000 characters · 60 steps per run (failed steps always kept)
AI Report Feedback log text per step400 characters (passed) · 1,500 (failed)
AI Report Feedback output4 checks per scenario · 4 recommendations · 10 load tests
Chart with AIfirst 500,000 result rows
Number of reportsNo limit. Your company’s licence can limit pipeline runs; an Admin sees it in Company Settings › Billing.

Didn’t find your answer?Write to support with your project name and what you see on screen.

info@smartestqa.ai
© 2026 Selenity Technology · SmartestQAsmartestqa.aiThis page as MarkdownChecked against smarttest-fe predev 2a8c79cc, smarttest-be 633c2a03 (TestLogController, LogCleanupScheduler, SuiteRunFeedbackServiceImpl, CompanySecurityAspect, ApiKeyAuthenticationFilter), smartest-mcp-server b95acc0 (src/next/runs.ts)