1. Docs
  2. Resources
  3. Deploys & Repos
  4. How the numbers work

How the numbers work

Every number and state on the Deploys & Repos page, how it is calculated, and an example you can check against your own project.

5 min read Updated CustomersAdmins

In short

On this page you can

  • Understand which pipeline runs count as testing a deploy.
  • Read Tested after deploy and Running untested correctly.
  • Know why one green run after a red one makes a deploy Tested.
  • See why a custom environment lowers the tested ratio.

How to use it

  1. Find the deploy’s time and the next successful deploy to the same environment.
  2. List the pipeline runs in that environment between the two times.
  3. Take the newest run: running, broken or passed decides the chip.
  4. Compare with the KPI cards, which count the last 7 days.

Tested after: which runs count#

The tested-after windowShown in: cell chips, Tested after column, Tests after deploy
window   = from this deploy's time
           until the next SUCCESSFUL deploy of the same repository to the same environment (open if none)
runs     = pipeline runs of the project in the same environment (DEV, TEST or PRP)
           that started inside the window           listed: newest 10
per run  : total   = test case results in the run
           passed  = results Success or Completed
           failed  = results Failed or Error
           running = results Running or New
state    = newest run has running > 0   → Test running
           newest run has failed > 0    → n broken · passed/total
           otherwise                    → Tested · passed/total
           no run in the window         → Not tested
           environment not DEV/TEST/PRP → No pipeline environment

Example: web v2.14 reaches TEST at 10:00. The next TEST deploy of web is at 15:00.

Pipeline run (TEST)StartResultIn the window?
Nightly regression09:5540/40No: before the deploy, it belongs to the previous version
Smoke10:0518/20, 2 failedYes
Smoke (after a fix in the test)12:0020/20Yes, newest
Smoke15:1020/20No: belongs to the 15:00 deploy

The newest run in the window (12:00) decides: Tested · 20/20. Both runs are listed under Tests after deploy. A deploy that was broken first and then passed counts as tested.

Only pipeline runs count. Dry runs in Automate have no pipeline environment and are not matched. The match is by environment for the whole project: any pipeline in TEST tests every service deployed to TEST at that time. A failed or cancelled deploy has no window of its own; runs after it still belong to the last successful deploy.

Cell states#

What a cell in Currently running versions showsShown in: Overview › Currently running versions
records  = the newest 20 deploy records of this repository in this environment
current  = the newest SUCCEEDED record among them       → commit, version, age, test chip
pending  = the newest record, if it is not "current"    → "Deploy in progress" (status In progress)
                                                           "Last deploy failed · sha" (any other status)
empty    = no record: "No deploy yet" if the environment is switched on in the mapping, else "Not mapped"
ChipColourFrom
Tested · p/tGreenState Tested, newest run’s passed and total
f broken · p/tRedState broken, newest run’s failed, passed and total
Test running · pipelineBlueState running, newest run’s pipeline name
Not tested (· n deploys)AmberNo run in the window; streak below
No pipeline environmentGreyAn environment pipelines do not use

Example: TEST received a1b2c3d (Succeeded) at 10:00 and c4d5e6f (Failed) at 11:00. The cell shows a1b2c3d with its test chip, and below it “Last deploy failed · c4d5e6f”, because a1b2c3d is what still runs. A cancelled deploy also shows “Last deploy failed”.

Untested streak#

“Not tested · n deploys”Shown in: Currently running versions
n = number of the most recent SUCCESSFUL deploys, in a row, whose state is Not tested
    (looks at the newest 5 successful deploys and stops at the first one that is not Not tested)
label = "Not tested"              when n = 1
        "Not tested · n deploys"  when n > 1

Example: api in TEST: Monday’s deploy was tested; Tuesday’s, Wednesday’s and Thursday’s deploys had no pipeline run before the next deploy. n = 3: the cell reads Not tested · 3 deploys. Three versions went through TEST without a test. One pipeline run now makes Thursday’s deploy tested and the chip changes to its result.

The KPI cards#

The four cards at the topShown in: Deploys & Repos, above the tabs
Connected repositories = repositories of this project (deleted ones not counted)
Deploys in last 7 days = SUCCESSFUL deploys of the project in the last 7 × 24 hours
Tested after deploy    = tested ÷ Deploys in last 7 days, shown as "tested / deploys"
                         tested = those deploys whose state is Tested, broken or Test running
                         (Not tested and No pipeline environment do not count)
Running untested       = number of cells (repository × environment) whose current version is Not tested
                         amber when > 0

Example: two repositories, web and api. Successful deploys in the last 7 days:

Repository · environmentDeploysTestedCurrent version
web · TEST32Not tested (the newest one)
web · PRP11Tested
api · TEST221 broken
api · STAGING10No pipeline environment

Cards: Connected repositories 2 · Deploys in last 7 days 7 · Tested after deploy 5 / 7 · Running untested 1 (only web · TEST). The broken api deploy counts as tested: a pipeline did run against it. The STAGING deploy is in the 7 but can never be tested, so custom environments lower the ratio.

Commits and files#

“← 9f8e7d6 · 5 commits”, “5 commits · 12 files”Shown in: Deploy history, Deploy detail
previous version = the newest SUCCESSFUL deploy of the same repository to the same environment
                   before this one
commits          = commits after the previous version up to and including this one (from your Git provider)
files            = files changed between the two versions
same commit      → 0 commits, 0 files (a redeploy)
lists shown      : up to 100 commits and 300 files; the counts can be higher

Example: PRP ran 9f8e7d6. CI deploys a1b2c3d to PRP; on the branch there are 5 commits between them touching 12 files. History shows a1b2c3d ← 9f8e7d6 · 5 commits; the detail shows 5 commits · 12 files. Line counts (+18 −4) appear for GitHub, Gitea and Bitbucket; GitLab and Azure DevOps give the file status only.

Counts need an access token. Without one the history shows only ← 9f8e7d6, except for a redeploy, which always shows 0 commits.

Relative times and short commits#

“just now”, “12 min ago”, “3 h ago”, “2 d ago”Shown in: cells, Last deploy column
s = seconds since the deploy (or the last notification, in Last deploy)
s < 60                       → just now
m = round(s ÷ 60);  m < 60    → m min ago
h = round(m ÷ 60);  h < 24    → h h ago
otherwise                    → round(h ÷ 24) d ago
short commit = the first 7 characters of the commit SHA
Time sinceCalculationShown
45 ss < 60just now
89 sround(1.48) = 11 min ago
90 minround(1.5) = 22 h ago
23 h 40 minround(23.67) = 24, not < 241 d ago
36 hround(1.5) = 22 d ago

The page does not refresh the ages by itself; click Refresh. When CI sends no deployedAt, the deploy time is the moment the notification arrived.

Didn’t find your answer?Write to support with your project name and what you see on screen.

info@smartestqa.ai