1. Docs
  2. Use cases
  3. Use cases
  4. How do I test sign-up with a one-time code sent by e-mail?

How do I test sign-up with a one-time code sent by e-mail?

In SmartestQA you connect a dedicated test mailbox under Integrations › Email (IMAP/SMTP), then add a Mail step with the operation Extract OTP right after the step that sends the code. The step waits for the new e-mail, saves the code in a variable such as otp, and the next step types {{tc:otp}} into the code field.

10 min read Updated CustomersAdmins

In short

On this page you can

  • Finish a sign-up, two-factor login or password reset that sends a code by e-mail.
  • Read a 6-digit code by default, or any format with your own pattern.
  • Wait up to 600 seconds for the e-mail, checked every 5 seconds.
  • Never reuse an old code: only new, unread e-mails count.
  • Follow a confirmation link instead of a code with Extract Link.

How to use it

  1. Open Integrations › Email (IMAP/SMTP), click Setup, fill the mailbox and click Save & Test.
  2. In the test case, after the step that sends the code, click More › Mail.
  3. Choose Operation Type Extract OTP, fill Sender Filter and set Output Variable to otp.
  4. Add a Type step with {{tc:otp}} on the code field.
  5. Run it where the mailbox can be reached.

The short answer#

A test cannot finish a sign-up without reading the e-mail that carries the code. In SmartestQA the Email integration connects one mailbox to the project, and a Mail step with the operation Extract OTP reads it. Put the step right after the click that sends the code. It checks the mailbox every 5 seconds, finds the newest matching e-mail, takes the first text that matches the pattern (by default a 6-digit number) and saves it in the variable you name. The next step types {{tc:otp}} into the code field.

Only new mail counts: unread messages that arrived after the step started, with a 30-second margin. The e-mail is then marked as read, so an old code from an earlier run is never used. Mail steps exist in web, mobile and desktop projects.

The problem this solves#

Sign-up, two-factor login and password reset all stop at the same point: “enter the code we sent you”. Many teams leave these flows to manual testing, where a tester opens the mailbox and copies the code. Others ask the developers for a fixed code or a switch that turns the check off in the test environment. Then the test passes, but the real path is never tested: the e-mail is sent, it arrives, and its code is accepted.

Teams that script it write their own mailbox reader. It has to connect, search, wait, pick the code out of the text, and not take the code from the previous run. What is missing is a step that waits for the new e-mail, takes the code and hands it to the next step.

An example#

This is an illustration, not a customer story. A team tests an HR portal where a new employee signs up with an e-mail address and a 6-digit code.

  1. The QA lead sets up a dedicated test mailbox with IMAP on and an app password. Under Integrations › Email (IMAP/SMTP) the QA lead clicks Setup, fills the form and clicks Save & Test: “All connections successful”.
  2. A tester records the sign-up form up to the button that sends the code, with the address of the test mailbox.
  3. The tester clicks More › Mail, chooses the Operation Type Extract OTP, fills Sender Filter with part of the portal’s sender address, sets Output Variable to otp and clicks Add.
  4. The next step is a Type step on the code field with the value {{tc:otp}}, then the click that verifies it.
  5. In a Dry Run the mail step starts at 10:00:00 and checks the mailbox every 5 seconds. The e-mail arrives at 10:00:20. The step saves the code, marks the e-mail as read, and the Type step enters the code.
  6. One night the portal sends no e-mail. The step ends with “Timeout: No email with OTP found within 60 seconds”, as ERROR, and the test case stops there.

The sign-up test now runs to the end without a person copying a code, and a missing e-mail shows up as its own error.

By hand, with your own mailbox script, and with SmartestQA#

QuestionReading the code by handYour own mailbox scriptSmartestQA
Who reads the code?A person, in the mailboxA script you write and maintainA Mail step with the operation Extract OTP
How is an old code avoided?The person looks at the time of the e-mailYour own logic, such as deleting mail or comparing timesOnly unread e-mails that arrived after the step started count, with a 30-second margin; the e-mail is then marked as read
How long does it wait?As long as the person waitsAs long as the loop you wroteIt checks every 5 seconds for 60 seconds by default, up to 600
Where is the mailbox password?With the personOften in the script or its configurationStored encrypted in the email integration; the step only names the integration
Can it run at night without a person?NoYesYes, where the mailbox can be reached. A mail server inside your network needs a run on an agent inside it

Steps#

  1. Prepare a test mailbox

    Use a dedicated mailbox for tests, never a person’s mailbox: tests read and mark its e-mails. Turn IMAP on for it and create an app password, because the normal password usually does not work. See Before you start.

  2. Connect the mailbox

    Go to IntegrationsEmail (IMAP/SMTP) and click Setup. In Add Email Integration enter an Integration Name, pick the Email Provider (Gmail, Outlook / Office 365, Yahoo Mail or Custom IMAP), and enter the Email Address and the app password. For Gmail, Outlook and Yahoo the servers are Auto-filled. Click Save & Test. See Set up the mailbox.

    “All connections successful”, with “IMAP connection successful (n mail)” and “SMTP connection successful”.

  3. Build the sign-up steps up to the code

    Record or add the steps that open the form, type the address of the test mailbox and click the button that sends the code. If the form needs a password, keep it in an encrypted key and type {{ds:key}}, not the password itself. See Where secrets belong.

  4. Add the mail step

    Click More › Mail. In Add Email Step fill Test Step Name, for example Get OTP code, pick the Email Integration and choose the Operation Type Extract OTP. See Add a mail step.

  5. Fill the parameters

    Sender Filter is required: text to search in the sender address. Subject Filter narrows it, for example Verification code. Leave OTP Pattern (Regex) for a 6-digit code, set Timeout (seconds) if 60 is not enough, and name the Output Variable, for example otp. Click Add.

  6. Type the code

    Add a Type step on the code field with the value {{tc:otp}}, then the click that verifies it.

  7. Run it

    Start a Dry Run. When the step runs, the reference is replaced by the code, and the live log shows the resolved value. The mailbox is reached from the machine that runs the test, so choose a run environment that can reach the mail server.

Example · two-factor login
4  Click         Send code
5  Extract OTP   Sender Filter: shop.example.com   Output Variable: otp
6  Type          Code field   value: {{tc:otp}}
7  Click         Verify

Extract OTP: fields, defaults and limits#

FieldRequiredDefaultMeaning
Sender FilterYesNoneText to search in the sender address.
Subject FilterNoNoneText to search in the subject. It can contain variables, for example Code for {{tc:userEmail}}.
OTP Pattern (Regex)No\b(\d{6})\bThe code to find. The default is a 6-digit number. For a 4-digit code use \b(\d{4})\b.
Timeout (seconds)No60Maximum waiting time, 1 to 600.
Output VariableYesNoneThe variable that stores the code. Later steps read it as {{tc:name}}.

Which e-mail the step picks#

RuleValue
An e-mail matches whenIt is unread, it was received no earlier than 30 seconds before the step started, and the filters match
The mailbox is checkedEvery 5 seconds
Several e-mails matchThe newest one is used
Several numbers in the e-mailThe first text that matches the pattern is used
After the code is readThe e-mail is marked as read
The step gives up atStep start plus Timeout (seconds)
On timeout or a mailbox errorThe step ends as ERROR and stops the test case

Example: the step starts at 10:00:00 with Timeout 60. A matching e-mail received at 09:59:45 counts, because it is within 30 seconds. One received at 09:59:00 does not. If none arrives by 10:01:00, the step ends with “Timeout: No email with OTP found within 60 seconds”. See Which emails a step sees.

What you get#

  • A sign-up test that runs to the end. The code goes from the e-mail into the form with two steps: Extract OTP and Type.
  • No stale codes. Unread mail that was already in the inbox is ignored, and a used e-mail is marked as read so the next run cannot pick it up again.
  • Any code format. The pattern is a regular expression; the default finds a 6-digit number.
  • Five mail operations. Extract OTP, Extract Link, Wait for Mail, Get Mails and Send Mail, in web, mobile and desktop projects.
  • Four providers. Gmail, Outlook / Office 365, Yahoo Mail and Custom IMAP; the first three fill their servers by themselves.
  • The mailbox password kept out of the test. It is stored encrypted in the email integration, and the step only picks the integration by name.
  • A value you can reuse. The code is a variable of the test case: every step after the mail step can read it.

Questions and answers#

Does it work with Gmail and Outlook?Customers

Yes. The SmartestQA Email integration offers Gmail, Outlook / Office 365, Yahoo Mail and Custom IMAP. IMAP must be turned on for the mailbox, and you sign in with an app password, because the normal password usually does not work. See Before you start.

Can I use my own mailbox for the test?Customers

Do not. Use a dedicated mailbox for tests, never a person’s mailbox: the mail steps read its e-mails and mark them as read. See the checklist.

What if my code has 4 digits, not 6?Customers

Change OTP Pattern (Regex) in the Extract OTP step. The default \b(\d{6})\b finds a 6-digit number; for a 4-digit code use \b(\d{4})\b. See Fields.

Can the test pick up an old code from an earlier run?Customers

No. Extract OTP looks only at unread messages that arrived after the step started, with a 30-second margin for mail that arrived just before. After reading the code it marks the e-mail as read. See Which emails a step sees.

What happens if the e-mail does not arrive in time?Customers

The step ends with “Timeout: No email with OTP found within 60 seconds”, with your own timeout in place of 60. It ends as ERROR and stops the test case. Raise Timeout (seconds), up to 600, or check the filters. See the fix.

Our mail server is only reachable inside our network. Does it still work?Customers

Yes, on an agent. Mail steps connect from where the test runs, and a run on the SmartestQA server cannot reach a mail server that is only inside your network. Run those tests on an agent inside it. See Which one to choose.

Where is the mailbox password kept?Customers

In the email integration under Integrations, stored encrypted. It is not part of any step: the mail step only names the integration. When you edit the integration, “Leave blank if you don't want to change it” applies. See The Email page.

Why does the Mail dialog say the email integration has not been configured?Customers

The project has no mailbox yet. The dialog says “Email integration has not been configured yet. Please configure email from the Integrations page first.” Set one up under Integrations › Email (IMAP/SMTP), then open the dialog again. See the fix.

Why did the test type {{tc:otp}} into the field instead of the code?Customers

No value was found for the reference, so it was not replaced. Check the spelling of the Output Variable and that the Extract OTP step ran before the Type step. See the fix.

Read more#

Didn’t find your answer?Write to support with your project name and what you see on screen.

info@smartestqa.ai