1. Docs
  2. Build tests
  3. InSight
  4. FAQ, MCP & limits

FAQ, MCP & limits

Answers to common questions about InSight, how AI assistants work with these numbers, and the limits of every list.

4 min read Updated CustomersAdmins

In short

On this page you can

  • Find out what InSight counts and leaves out.
  • Learn which assumptions are fixed: 30 s per step, the 80% target, the maturity weights.
  • Use the copied MCP prompts and the smartest_status tool with an assistant.
  • Check list sizes and time windows before you trust an empty card.
  • Give a five-minute InSight demo.

How to use it

  1. Filter the questions by your role with the buttons above the list.
  2. Open a question; follow its link for the formula or the screen.
  3. For assistants, connect one and paste a copied prompt.
  4. Check the limits table when a list looks shorter than expected.

Questions and answers#

Do Admins and Users see different numbers?Admins

No. Anyone with access to the project sees the same InSight, and can save and delete Test Health reports. Access to a project is granted per user; see roles.

Are dry runs counted?Customers

No. Dry runs leave no report, so InSight never sees them. Pipeline runs, test cases started on their own, manual pipeline runs and load tests are counted. See the report record and the live log.

Do automatic retries change the success rate?Customers

Yes. Each attempt is a scenario run, so a test case that failed and then passed on a retry adds one failed and one passed run, and counts toward its flaky score. The pipeline run strip, however, judges each test case by its latest attempt. See Strip marks.

How do load tests count?Customers

In Execution they are runs like any other. In Test Health a JMeter or k6 step counts as a check, so a load test can prove a sub-requirement, but load tests are left out of the scenario funnel.

Why does “Agents online” include agents from other projects?Admins

Agents belong to the company, not to a project, and any project of the company can run on them. The count is the company’s.

Can we change the 30 seconds per step, the 80% target or the maturity weights?Customers

No, they are fixed: 30 s of manual work per step for Time saved, 80% as the pillar target, and weights of 35 / 25 / 25 / 15 for maturity. Quote them with the number when you report it.

How do I share InSight with someone outside SmartestQA?Customers

Download Execution as Excel or PDF, and print Test Health (live or a saved report) as PDF. Saved reports themselves are only visible inside the project.

Can I see Test Health for one module only?Customers

On screen, use Modules × coverage. An AI assistant can ask for one module’s status through MCP.

How far back can I look?Customers

Execution takes any range of up to 365 days. Progress can go back to the project’s first record; its past points are rebuilt and approximate. For an exact picture of a past day, save a Test Health report on that day.

Where are User Statistics and the weekly metrics chart?Customers

In the classic layout at /dashboard/default?classic=1. Its numbers are counted differently; see Classic view and its numbers.

For AI assistants (MCP)#

InSight is built to hand work to an assistant connected through the SmartestQA MCP server.

  • Copied prompts. Every Copy MCP prompt button copies a ready task. Test Health prompts start with a line such as SmartestQA project 42 · PROVE · sub:318, which the assistant reads as a command for that project, key and record, then works from there.
  • smartest_status returns the Test Health numbers for the project or one module: coverage, the funnel, the sub-requirements automation cannot prove, and the open work as keys in priority order.
  • smartest_runs reads runs and logs for the Execution questions; see Report › MCP.
  • With an API key or MCP sign-in, an assistant can read Test Health and Execution, but not Progress history or saved reports; those need a login in the app.

Connect an assistant from Device FarmTest AgentsConnect to Claude; see Discover › MCP.

Example prompts
What is the Test Health of project "Customer Portal"? Start on the first open item and tell me what you changed.

Which failure cluster in the last 7 days hits the most scenarios, and is it a product bug or a test problem?

List the risky sub-requirements in module Orders that no passing scenario proves, and write scenarios for the first three.

Limits#

ItemLimit
Execution period1–365 days, default 7
Pipeline run strip14 runs per pipeline, looking back at least 56 days
Failure clusters6 on the card · 100 in the dialog · signature 160 characters
Flaky scenarios10 · score from 5 runs · last 10 results shown
Slowest scenarios8
Results by requirement10
Failures by action type / by page6 each
Open tasks listed6
Weekly trend8 weeks of 7 days
Stuck runsRunning over 2 hours, started in the last 30 days
Fix these first3
Gaps, “check no result” list, not automatable list100 rows each
Long scenarioover 60 steps
Recent run (funnel, pipeline last run, stale)30 days
Autonomous agents cardlast 7 days
Progress pointsdaily up to 31 days, weekly beyond · default 90 days
Saved reports100 newest listed · name up to 200 characters

Didn’t find your answer?Write to support with your project name and what you see on screen.

info@smartestqa.ai
© 2026 Selenity Technology · SmartestQAsmartestqa.aiThis page as MarkdownChecked against smarttest-be 633c2a03 (quality/*, aspect/CompanySecurityAspect.java, middleware/ApiKeyAuthenticationFilter.java), smarttest-fe predev 2a8c79cc (pages/dashboard/**), smartest-mcp-server b95acc0 (src/next/status.ts, runs.ts)