1. Docs
  2. Use cases
  3. Use cases
  4. How do I keep passwords and API keys out of my tests?

How do I keep passwords and API keys out of my tests?

In SmartestQA a secret lives in Data Source, not in the test: an encrypted key in Data Source › Variables for a password, token or API key, and the connection in Data Source › Connections for a database or Kafka password. The step stores only a reference such as {{ds:adminPassword}}, and run logs and reports show •••••• in place of the value.

10 min read Updated CustomersAdmins

In short

On this page you can

  • Store passwords, tokens and API keys encrypted; only the creator of a value can show it again.
  • Reference a secret in any step as {{ds:key}} instead of typing it.
  • Keep database and Kafka passwords in the connection, not in the step.
  • See •••••• in the live log, the report and the API request log.
  • Rotate a secret in one place by typing the new value and saving.

How to use it

  1. Open Data Source › Variables and click New key.
  2. Name the key, keep Value comes from on Typed value and switch on Encrypt.
  3. Type the secret and click Save.
  4. In the test step, insert the key with the data picker: {{ds:adminPassword}}.
  5. Run the test and check that the log shows ••••••.

The short answer#

Do not type a secret into a step. Step values are stored with the test case and shown to every project member who opens it. Create a key in Data Source › Variables, switch on Encrypt, and use the key in the step as {{ds:key}}. The value is stored encrypted, only its creator can show it again in the key panel, and the runner receives it for the test run. Before a run writes a log line, a report entry or a live message, it replaces every known secret with ••••••.

Database and Kafka passwords belong in the connection under Data Source › Connections. Steps pick the connection by name, so the password never appears in the step or the test case.

The problem this solves#

Tests need real credentials: the password of a test user, an API token, a database password. The shortest way is to type the value into the step or the script. From then on everyone who opens the test can read it, and it can show up in logs and reports. Many teams also keep a shared document or spreadsheet of test accounts and copy from there.

When a password changes, someone has to find every test that contains it. When a colleague leaves the team, nobody can say which passwords that person has seen. What is missing is one place per secret, tests that hold only its name, and runs that do not print the value.

An example#

This is an illustration, not a customer story. A team tests an online banking web application, and the password of the test user and an API token are typed into the steps of about forty test cases.

  1. The QA lead installs the Test data audit agent from the catalog on Agents and clicks Run now. It finds the passwords and tokens written into steps and says where each should move. It changes nothing.
  2. In Data Source › Variables the QA lead clicks New key, names it adminPassword, leaves Value comes from on Typed value, switches on Encrypt, types the password and clicks Save. The field shows “•••••• (set, encrypted)” and the list shows the kind Encrypted. A second key, apiToken, holds the token.
  3. A tester opens the login steps and picks the key with the picker icon next to the value. The value is now {{ds:adminPassword}}. The API steps get {{ds:apiToken}} as the Bearer token.
  4. The tester starts a Dry Run. The runner receives the real value and uses it in the step; the live log shows ••••••.
  5. Later the password of the test user is changed. The tester opens the key, types the new value and clicks Save, without seeing the old one. The QA lead, who created the value, can still show it with the eye icon.

The password is now in one key, not in forty test cases, and every test that references the key uses the new value on its next run.

Typed into the test, a separate secret store with your own scripts, and SmartestQA#

QuestionTyped into the test or scriptA separate secret store with your own scriptsSmartestQA
Where is the value?In the test, next to the stepsIn the store; the script fetches it at run timeIn an encrypted key in Data Source › Variables; the step holds {{ds:key}}
Who can read it?Everyone who can open the testWhoever the store allowsOnly its creator can show it again. Everyone else, Admins included, sees “•••••• (set, encrypted)” and can replace it
Does it appear in logs and reports?It can, unless you filter the outputThat depends on whether your script masks itThe live log, the run report and the API request log show ••••••. A screenshot of a page that displays a secret still shows it
How do I rotate it?Find and edit every testChange it in the storeType the new value in the key and save; tests use it on their next run
Does an AI assistant see it?Yes, when it reads the testThat depends on your setupThe AI receives key names and whether they are encrypted, never the value. Through MCP an assistant creates the key empty and you fill it in

Steps#

  1. Create the key

    Go to Data Source › Variables and click New key. Type the name the steps will use, for example adminPassword. Leave Value comes from on Typed value: a dataset or database value cannot be encrypted.

  2. Switch on Encrypt

    Click Encrypt next to the value. The hint says “Store the value encrypted: it is never shown again and only the test run reads it.”

  3. Type the secret and save

    The field hides what you type. Add a Description so others know what the key is for, and click Save.

    The field now shows “•••••• (set, encrypted)”. In the list the key’s kind is Encrypted and its value “•••••••• set”.

  4. Use the key in the step

    In Automate, select the step and click the picker icon next to the value (“Data Sources - Global Parameters - Datasets”). Pick the key. The step’s value is now {{ds:adminPassword}}. For an API step, put {{ds:apiToken}} in the Bearer token or a header.

  5. Put database and Kafka passwords in the connection

    In Data Source › Connections, click New DB or New Kafka Connection. The Password is required for a new connection, stored encrypted and never sent back to the screen. Do not put the password in the Database Url: it would be visible to everyone in the list.

  6. Run and check the log

    Start a Dry Run. The runner receives the real value and uses it in the step; the live log shows ••••••.

  7. Rotate when needed

    Open the key, type the new value and save. Anyone who can edit keys can replace a secret without seeing the old one. Leave the field empty and save to keep the stored secret.

Where each kind of secret belongs#

SecretPut it inThe step uses
Test user password typed into a login formAn encrypted key, for example adminPassword{{ds:adminPassword}} as the Type value
API token or API keyAn encrypted key, for example apiToken{{ds:apiToken}} in the Bearer token or a header
Database passwordThe DB connectionThe connection, picked by name
Kafka SASL passwordThe Kafka connectionThe connection, picked by name
Test mailbox passwordThe email integration under IntegrationsThe Mail step’s mailbox

Who and what can see an encrypted value#

WhereWhat appears
The key list“•••••••• set”, or “no value”. Also for its creator.
The key panel, for the creatorAn eye icon: “Show the value (only you, its creator, can)”.
The key panel, for anyone else, Admins includedOnly “•••••• (set, encrypted)”. No eye icon. They can replace the value.
Test runsThe runner receives the real value and uses it in the step.
Live log, run report, API request log••••••
Send previews on the screen•••••• is sent in place of the value. Check secured calls with a Dry Run.
The AI (App Map, generation)Key names and “encrypted”, never the value.
AI assistants through MCPNever the value. They cannot write one either.
The audit logField values of screens that can hold secrets are not recorded; secret field names show ***.

What you get#

  • One place per secret. A step stores the key’s name, not the value. Change the key and every test that uses it picks up the new value on its next run.
  • Encryption on the server. Encrypted keys and connection passwords are stored with AES-256-GCM.
  • Masking in runs. The value of every encrypted key read during the run, the Authorization, token and API key headers of API steps, and text typed into a password field are replaced with ••••••.
  • An audit of what slipped through. The Test data audit agent finds passwords, tokens and personal data written into steps, and empty data sources, and says where each should move. It changes nothing.
  • AI assistants that never handle a password. Through MCP, an assistant creates the key empty (encrypted for secrets) and asks you to fill it in SmartestQA.
  • Masked AI reviews. With Share Sensitive Data off, e-mail addresses, card numbers, national ID numbers, phone numbers, secrets and tokens are masked before the run review is sent to the AI.

Three limits to know

Masking hides values, not screenshots: a screenshot of a page that displays a secret still shows it. Values shorter than 3 characters are not masked. Datasets are not secret: every project member can open a dataset and see all its cells, so keep passwords out of them.

Questions and answers#

Where do I put the password for a login test?Customers

In an encrypted key in Data Source › Variables, for example adminPassword. The Type step’s value is then {{ds:adminPassword}}, and the password itself is not in the test case. See Encrypt a value.

Can an admin see an encrypted value?Customers

No. Only the person who created an encrypted value can show it, with the eye icon in the key panel. An Admin sees “•••••• (set, encrypted)” and can replace the value, but cannot read it. See Who can see an encrypted value.

Do passwords show up in test logs or reports?Customers

No. A run replaces every known secret with •••••• in the live log, the report and the API request log. In the report’s list of variables, values whose name looks like a password or token are masked too. A screenshot of a page that displays a secret still shows it. See Secrets in runs.

What happens if someone types a password straight into a step?Customers

The value is stored with the test case and visible to everyone in the project. Values typed into fields whose name looks like password, secret, token or PIN are shown as •••••• in run logs, but the test case still holds the text. Move it to an encrypted key. See Secrets and personal data.

How do I find passwords that are already written into steps?Customers

Install the Test data audit agent from the catalog on Agents. It is an on-demand agent that you start with Run now. It finds passwords, tokens and personal data written into steps and says where each should move, and it changes nothing. See On demand agents.

How do I rotate a password or token?Customers

Open the key in Variables, type the new value and click Save. Every test that references the key uses the new value on its next run. For a connection, type the new password in the field labelled “Password (leave empty to keep current)”. See Replace or remove a secret.

Does the AI see my passwords?Customers

No. The AI receives key names, their kind, description and whether they are encrypted, never the value. For the AI review of a run, switch Share Sensitive Data off in the project’s AI Options to mask sensitive values before they are sent. See What is sent to the AI.

Can an AI assistant connected through MCP read or write a password?Customers

No. The assistant never asks you for a password and never writes one. When a test needs a credential, it creates the key empty, encrypted for secrets, and you fill it in SmartestQA. It reads connections with their credentials masked. See Other safeguards.

What happens when I copy an encrypted key or switch Encrypt off?Customers

The secret does not travel. A copy is a plain key without a value: type the secret again and switch on Encrypt. If you switch Encrypt off, the field is empty; saving it empty leaves the key without a value. See My key lost its value.

Where does the database password go?Customers

Into the connection in Data Source › Connections. It is stored encrypted and never sent back to the screen: the field stays empty when you open the connection. A failed Test Connection masks it in its message. See Add a database connection.

Read more#

Didn’t find your answer?Write to support with your project name and what you see on screen.

info@smartestqa.ai