---
title: "Use it in tests"
section: "Data Source"
description: "How SmartestQA test steps read Data Source: {{ds:key}} and {{excel:name}}_column syntax, the picker, DB and Kafka steps, run-time resolution, environments and where connections are opened from."
url: https://docs.smartestqa.ai/en/data-source-in-tests.html
audience: [customer, admin]
updated: 2026-11-08
keywords: "ds excel tc placeholder syntax variable reference picker db step output values kafka step produce consume run resolution environment agent server where connects"
---
# Data Source: Use Data Source in tests

Steps never copy a value from Data Source; they store a reference and the run looks it up. This page shows the syntax, the steps that use connections, and what happens at run time.

## In short

**On this page you can**

- Reference a key in any step value as {{ds:key}} .
- Read a dataset column as {{excel:name}}_column .
- Run SQL in a DB step and keep its columns as {{tc:name}} variables.
- Send or wait for Kafka messages through a saved Kafka connection.
- Know which machine opens the database or broker connection.

**How to use it**

1. In a step value, click the data source picker icon and pick a key or dataset.
2. For SQL, add a DB step, choose its Configuration and list the Output Values .
3. Run a Dry Run on the machine that can reach your systems.
4. Read the step log: a DB or dataset key that fails is named in the error.
5. Schedule the test in a pipeline with the environment it should use.

## Reference syntax

| You write | Comes from | The run gets |
| --- | --- | --- |
| `{{ds:loginEmail}}` | A key in [Variables](https://docs.smartestqa.ai/en/data-source-variables.html) | The key’s value: typed, from a dataset cell or from its SQL query, for the run’s environment. |
| `{{loginEmail}}` | The older form without prefix | A [variable of the test case](https://docs.smartestqa.ai/en/automate-steps-variables.html#syntax) with that name wins. The key is read only when no step produced the name and the reference is the whole value; inside longer text, such as a URL, it is not read. New references are written with the prefix ([resolution order](https://docs.smartestqa.ai/en/automate-test-data.html#resolution)). |
| `{{excel:users}}_email` | A [dataset](https://docs.smartestqa.ai/en/data-source-datasets.html) column | The current loop row, or the first row. |
| `{{excel:users}}_#` | A dataset | Its number of rows. |
| `{{tc:orderNo}}` | A test case variable, not Data Source | A value produced earlier in the run, for example by a DB step’s [**Output Values**](https://docs.smartestqa.ai/en/automate-steps-database.html#outputs), a Get step or an API extractor. |

A reference can sit inside other text: `{{ds:apiBase}}/orders/{{tc:orderNo}}`. Key names are case-sensitive in practice: write them exactly as in the list.

### Insert a reference with the picker

Wherever a step accepts a reference, a small table icon with the tooltip “Data Sources - Global Parameters - Datasets” opens the list of keys, [global parameters](https://docs.smartestqa.ai/en/automate-steps-variables.html#global-parameter) and datasets. Pick one and it is inserted at the cursor. You find the icon in step values, the Bearer token of [API steps](https://docs.smartestqa.ai/en/automate-steps-api.html), the JavaScript and Python editors, [Mail steps](https://docs.smartestqa.ai/en/automate-steps-mail.html), AI prompts and [pipeline parameters](https://docs.smartestqa.ai/en/schedule-settings.html#pipeline-parameters).

## What happens at run time

1. When a step runs, each `{{ds:…}}` is looked up by name in the project.
2. If the run has an environment and a variant exists (`loginEmail_TEST` in a TEST run), the variant is used. See [Environment-specific keys](https://docs.smartestqa.ai/en/data-source-variables.html#environments).
3. The value is read from its source: the stored text, the dataset cell, or the SQL query, which runs at that moment on the machine that runs the test.
4. An encrypted value is handed to the runner only. It is used in the step and replaced by `••••••` in the [live log](https://docs.smartestqa.ai/en/automate-dry-run.html#live-log), the report and the API request log. See [Secrets in runs](https://docs.smartestqa.ai/en/data-source-secrets.html#in-runs).
5. If no key has that name, the step gets no value: depending on the step, it uses the text `{{ds:…}}` as written or an empty text. Check the step’s log for the value it used.

A DB-bound or dataset-bound key that cannot be read fails the step with “Could not resolve DB data source '*key*': *reason*” or “Could not resolve Excel data source '*key*': *reason*”. It never falls back to an old value. Fixes: [DB](https://docs.smartestqa.ai/en/data-source-troubleshooting.html#fix-db-key-no-rows), [Excel](https://docs.smartestqa.ai/en/data-source-troubleshooting.html#fix-excel-resolve).

## DB steps

A DB step runs one SQL statement on a [DB connection](https://docs.smartestqa.ai/en/data-source-connections.html#add-db). Add it from the Automate [step toolbar](https://docs.smartestqa.ai/en/automate-add-steps.html#toolbar) (under **More…**); the dialog is called [**DB Requests**](https://docs.smartestqa.ai/en/automate-steps-database.html#add-db-step).

| Field | What to enter |
| --- | --- |
| [**Test Step Name**](https://docs.smartestqa.ai/en/automate-editor.html#rename-step) | What the step does, for example “Read new user”. |
| **Configuration** | The DB connection. With none in the project the list shows “Configuration not found”. |
| [**SQL Query**](https://docs.smartestqa.ai/en/automate-steps-database.html#sql-rules) | Any statement. SELECT needs explicit columns, no `*`. |
| [**Output Values**](https://docs.smartestqa.ai/en/automate-steps-database.html#outputs) | Variable names, comma-separated, one per column in order. |

Example · from the step’s “How it works?” guide

```text
SQL Query      SELECT firstname, lastname FROM "user" WHERE id = 1
Output Values  firstname, lastname

firstname → 1st column, lastname → 2nd column
Later steps:   Enter firstname  {{tc:firstname}}
               Enter lastname   {{tc:lastname}}
```

- Output values are matched to columns by position, not by name.
- The query may contain references, for example `WHERE email = '{{ds:loginEmail}}'`. They are filled in before the query runs.
- The statement has 30 seconds. Changes are committed.
- An error stops the test case. The log explains the usual causes, for example “Could not connect to the database. Check host, port, credentials and that the database is reachable.” or 'The column "{name}" was not found in the table. Column names are case-sensitive…'.
- The connection’s password never appears in the step or the test case.

## Kafka steps

Kafka steps (**Produce** and **Consume**) need a [Kafka connection](https://docs.smartestqa.ai/en/data-source-connections.html#add-kafka), chosen in **Kafka Configuration**. Without one you see “No Kafka configuration found. Create one on the Data Source page (Kafka tab).”: create it under **Data Source** › **Connections**.

- **Produce** sends a message to a topic and waits for the broker’s acknowledgement.
- **Consume** waits for a message that matches [**Match Path (JSONPath)**](https://docs.smartestqa.ai/en/automate-steps-kafka.html#consume) and [**Match Value**](https://docs.smartestqa.ai/en/automate-steps-kafka.html#consume), for at most **Timeout (seconds)** (default 10, at most 300). Without a match path the first message matches.
- Topic, message, key and match value accept references such as `{{tc:transferId}}`.

The full field list is part of the step reference in **Automate**.

## API and Mail steps

- **[API steps](https://docs.smartestqa.ai/en/automate-steps-api.html)** put tokens and passwords in references: `{{ds:apiToken}}` in the Bearer token or a header. See [API Requests in tests](https://docs.smartestqa.ai/en/api-requests-in-tests.html).
- **[Mail steps](https://docs.smartestqa.ai/en/automate-steps-mail.html)** read a [test mailbox](https://docs.smartestqa.ai/en/integrations-email.html). The mailbox is not a Data Source connection: it is set up under **Integrations** (Email).

## Where connections are opened from

The database or broker sees a connection from the machine that does the work. That is not always the same machine.

| What you do | Connects from |
| --- | --- |
| **Test Connection** on a DB or Kafka connection | The SmartestQA server |
| [DB step](https://docs.smartestqa.ai/en/automate-steps-database.html), DB-bound key or [Kafka step](https://docs.smartestqa.ai/en/automate-steps-kafka.html) in a [Dry Run](https://docs.smartestqa.ai/en/automate-dry-run.html) with [**Run on Server**](https://docs.smartestqa.ai/en/quick-start-connect.html#agent), or in a [pipeline](https://docs.smartestqa.ai/en/schedule-create.html) on the server | The SmartestQA server |
| The same with [**Run on your Agent**](https://docs.smartestqa.ai/en/quick-start-connect.html#agent), [an agent](https://docs.smartestqa.ai/en/agent-pool-agents.html) or a self-hosted runner | That machine |
| A mobile run on [AWS Device Farm](https://docs.smartestqa.ai/en/project-settings-mobile.html#aws) | Next to the device, in the cloud: the system must be reachable from the internet |
| **Send** on the [API Requests](https://docs.smartestqa.ai/en/api-requests.html) page | The SmartestQA server |

> **Tip: Database behind a firewall**
>
> If the database is only reachable inside your network, run the tests on an agent inside that network ([Do you need an agent?](https://docs.smartestqa.ai/en/quick-start-connect.html#agent)). **Test Connection** can still fail, because it is tried from the SmartestQA server. In that case check the connection with a Dry Run on the agent instead.

## If something goes wrong

- [Test Connection works, the run cannot connect](https://docs.smartestqa.ai/en/data-source-troubleshooting.html#fix-works-in-test-fails-in-run)
- [A step uses `{{ds:…}}` literally or gets an empty value](https://docs.smartestqa.ai/en/data-source-troubleshooting.html#fix-literal-placeholder)
- [“No Kafka configuration found…”](https://docs.smartestqa.ai/en/data-source-troubleshooting.html#fix-kafka-config-not-found)
- [“The column … was not found in the table”](https://docs.smartestqa.ai/en/data-source-troubleshooting.html#fix-db-step-column)
