- Docs
- Resources
- Data Source
- Use it in tests
Use Data Source in tests
Steps never copy a value from Data Source; they store a reference and the run looks it up. This page shows the syntax, the steps that use connections, and what happens at run time.
In short
On this page you can
- Reference a key in any step value as {{ds:key}}.
- Read a dataset column as {{excel:name}}_column.
- Run SQL in a DB step and keep its columns as {{tc:name}} variables.
- Send or wait for Kafka messages through a saved Kafka connection.
- Know which machine opens the database or broker connection.
How to use it
- In a step value, click the data source picker icon and pick a key or dataset.
- For SQL, add a DB step, choose its Configuration and list the Output Values.
- Run a Dry Run on the machine that can reach your systems.
- Read the step log: a DB or dataset key that fails is named in the error.
- Schedule the test in a pipeline with the environment it should use.
Reference syntax#
| You write | Comes from | The run gets |
|---|---|---|
{{ds:loginEmail}} | A key in Variables | The key’s value: typed, from a dataset cell or from its SQL query, for the run’s environment. |
{{loginEmail}} | The older form without prefix | A variable of the test case with that name wins. The key is read only when no step produced the name and the reference is the whole value; inside longer text, such as a URL, it is not read. New references are written with the prefix (resolution order). |
{{excel:users}}_email | A dataset column | The current loop row, or the first row. |
{{excel:users}}_# | A dataset | Its number of rows. |
{{tc:orderNo}} | A test case variable, not Data Source | A value produced earlier in the run, for example by a DB step’s Output Values, a Get step or an API extractor. |
A reference can sit inside other text: {{ds:apiBase}}/orders/{{tc:orderNo}}. Key names are case-sensitive in practice: write them exactly as in the list.
Insert a reference with the picker#
Wherever a step accepts a reference, a small table icon with the tooltip “Data Sources - Global Parameters - Datasets” opens the list of keys, global parameters and datasets. Pick one and it is inserted at the cursor. You find the icon in step values, the Bearer token of API steps, the JavaScript and Python editors, Mail steps, AI prompts and pipeline parameters.
What happens at run time#
- When a step runs, each
{{ds:…}}is looked up by name in the project. - If the run has an environment and a variant exists (
loginEmail_TESTin a TEST run), the variant is used. See Environment-specific keys. - The value is read from its source: the stored text, the dataset cell, or the SQL query, which runs at that moment on the machine that runs the test.
- An encrypted value is handed to the runner only. It is used in the step and replaced by
••••••in the live log, the report and the API request log. See Secrets in runs. - If no key has that name, the step gets no value: depending on the step, it uses the text
{{ds:…}}as written or an empty text. Check the step’s log for the value it used.
A DB-bound or dataset-bound key that cannot be read fails the step with “Could not resolve DB data source 'key': reason” or “Could not resolve Excel data source 'key': reason”. It never falls back to an old value. Fixes: DB, Excel.
DB steps#
A DB step runs one SQL statement on a DB connection. Add it from the Automate step toolbar (under More…); the dialog is called DB Requests.
| Field | What to enter |
|---|---|
| Test Step Name | What the step does, for example “Read new user”. |
| Configuration | The DB connection. With none in the project the list shows “Configuration not found”. |
| SQL Query | Any statement. SELECT needs explicit columns, no *. |
| Output Values | Variable names, comma-separated, one per column in order. |
SQL Query SELECT firstname, lastname FROM "user" WHERE id = 1
Output Values firstname, lastname
firstname → 1st column, lastname → 2nd column
Later steps: Enter firstname {{tc:firstname}}
Enter lastname {{tc:lastname}}- Output values are matched to columns by position, not by name.
- The query may contain references, for example
WHERE email = '{{ds:loginEmail}}'. They are filled in before the query runs. - The statement has 30 seconds. Changes are committed.
- An error stops the test case. The log explains the usual causes, for example “Could not connect to the database. Check host, port, credentials and that the database is reachable.” or 'The column "{name}" was not found in the table. Column names are case-sensitive…'.
- The connection’s password never appears in the step or the test case.
Kafka steps#
Kafka steps (Produce and Consume) need a Kafka connection, chosen in Kafka Configuration. Without one you see “No Kafka configuration found. Create one on the Data Source page (Kafka tab).”: create it under Data SourceConnections.
- Produce sends a message to a topic and waits for the broker’s acknowledgement.
- Consume waits for a message that matches Match Path (JSONPath) and Match Value, for at most Timeout (seconds) (default 10, at most 300). Without a match path the first message matches.
- Topic, message, key and match value accept references such as
{{tc:transferId}}.
The full field list is part of the step reference in Automate.
API and Mail steps#
- API steps put tokens and passwords in references:
{{ds:apiToken}}in the Bearer token or a header. See API Requests in tests. - Mail steps read a test mailbox. The mailbox is not a Data Source connection: it is set up under Integrations (Email).
Where connections are opened from#
The database or broker sees a connection from the machine that does the work. That is not always the same machine.
| What you do | Connects from |
|---|---|
| Test Connection on a DB or Kafka connection | The SmartestQA server |
| DB step, DB-bound key or Kafka step in a Dry Run with Run on Server, or in a pipeline on the server | The SmartestQA server |
| The same with Run on your Agent, an agent or a self-hosted runner | That machine |
| A mobile run on AWS Device Farm | Next to the device, in the cloud: the system must be reachable from the internet |
| Send on the API Requests page | The SmartestQA server |
Database behind a firewall
If the database is only reachable inside your network, run the tests on an agent inside that network (Do you need an agent?). Test Connection can still fail, because it is tried from the SmartestQA server. In that case check the connection with a Dry Run on the agent instead.
If something goes wrong#
Didn’t find your answer?Write to support with your project name and what you see on screen.
info@smartestqa.ai