How do I test a mobile app on real devices from my browser?
In SmartestQA you open Device Farm › Devices, click Use device on an Available phone and then Take device: its screen appears in your browser and your clicks become taps. Your build comes from the App Library, and dry runs in Automate and pipelines in Schedule run on the same Device Farm devices.
In short
On this page you can
- See and control a real phone or an emulator in the browser.
- Upload an APK, AAB, IPA or simulator
.ziponce and install it on farm devices. - Hold a device so that nobody else and no run can use it while you work.
- Run a dry run or a pipeline on Device Farm devices.
- Read temperature, battery, memory and CPU of an Android run.
How to use it
- Open Device Farm › App Library, click Add application and Upload.
- Open Devices and filter for Available.
- Click Use device on a card, then Take device.
- Click to tap, drag to swipe, use System Keys, then click Release.
- For automated runs, choose SmartTest Device Farm in the run settings and check Recent runs.
The short answer#
Real devices are in Device Farm, in the main menu. With the Device Farm right on your company’s licence and a MOBILE project, the Devices tab shows one card per phone, tablet, emulator or simulator. Click Use device on an Available card and then Take device. The screen is streamed to your browser, a click is a tap and a drag is a swipe. The device is held for you until you click Release.
Your build comes from the App Library. Automated tests use the same devices: a dry run in Automate or a pipeline in Schedule that targets SmartTest Device Farm installs the app you picked and runs on the devices you chose. Without the Device Farm right you can still open a phone attached to your own agent with Use Device Live, or run on devices in your AWS Device Farm or BrowserStack account.
The problem this solves#
Many teams test their mobile app on a handful of phones kept in a drawer. A tester has to find the phone, charge it, connect it and install the newest build by hand. A colleague in another office cannot use that phone at all. Emulators fill the gap, but they do not behave like a real phone in every respect.
The automated tests often live somewhere else, on other devices, with their own upload step and their own results. What is missing is one set of devices that a person can use live and that automated runs use too, one place for the build, and a record of which run used which device.
An example#
This is an illustration, not a customer story. A team builds a food delivery app for Android, and its testers sit in two offices.
- The QA lead, who is an Admin, opens Device Farm › App Library, clicks Add application, drops the APK and clicks Upload. The message is “file name: added to library”.
- In Devices the QA lead filters for Available, clicks Use device on a real phone and then Take device. The screen appears in the frame and the holder line says “You have this device”. The testers in the other office see the card as Busy.
- Install App puts the build on the phone. The QA lead places an order with clicks and drags, types the address on the on-screen keyboard of the device, and uses Rotate under System Keys to check the basket in landscape. A click on Release returns the device to Available.
- For the nightly suite, the QA lead opens the pipeline in Schedule, chooses SmartTest Device Farm as the Run Environment and picks three devices and the app on the Mobile Settings tab. The build is installed before the run and skipped when the same version is already on the device.
- The next morning Recent runs lists the runs on the farm devices. The metrics icon of a run opens Device Metrics with the Temperature, Battery, Memory and CPU charts.
Both offices now test on the same phones as the nightly pipeline, and the build is uploaded once.
Phones on your desk, a device cloud with your own scripts, and SmartestQA#
| Question | Phones on your desk | A device cloud with your own scripts | SmartestQA |
|---|---|---|---|
| Can I use a device by hand from my browser? | No, you hold it in your hand, which works fine for one person | Usually yes, in a live session | Yes: Use device, then Take device. Android takes taps, swipes and system keys; iOS is watch only for now |
| How do I know nobody else is using it? | Ask around | The provider’s own booking | A taken device shows Busy “In live use” and runs cannot take it |
| How does a new build get onto the device? | By cable or a download link, phone by phone | An upload step in your script before the run | Upload it to the App Library; Replace app for a new build. A run installs it and skips the install when the same version is already on the device |
| Do automated tests use the same devices? | Only when a script on someone’s computer drives them | Yes, with scripts you write and maintain | Yes: a dry run in Automate or a pipeline with SmartTest Device Farm as the Run Environment |
| What does a run leave behind? | The notes and screenshots a person took | The provider’s logs, separate from your test tool | A row in Recent runs; Android runs keep temperature, battery, memory and CPU charts |
Steps#
Check that you see the device tabs
Select a MOBILE project and open Device Farm. With the Device Farm right you see the tabs Test Agents, Nodes, Devices, Recent runs and App Library; Nodes is hidden for users of the cloud service. Without the right you see two tabs, and the second one has Contact us to enable it. Click it, add an optional message in Device Farm Request and click Send Request. See Ask for the Device Farm right.
“Request received · date” replaces the button.
Upload your build
An Admin opens Device FarmApp Library, clicks Add application, drops the file on “Drop an APK, AAB, IPA or simulator .zip here” and clicks Upload. See Upload a build.
“file name: added to library”. The row shows the package id (Android) or bundle ID (iOS) and the version read from the file.
Find a free device
Open Devices. The strip at the top counts Available, Busy and Offline devices. Search / pick a device... matches the name, model, UDID or node name. Filters narrow the list by Availability, Type (Real, Emulator), Brand, Operating system, OS version and Screen resolution. See Find a device.
Take the device
Click Use device on an Available card. The device view opens with an empty frame: “Take the device to mirror its screen and send input.” Click Take device. See Take a Device Farm device.
The screen appears in the frame, the holder line says “You have this device” and the card shows Busy to everyone else.
Install the app and use it
Install App under the phone installs a build from the App Library (Admins and on-premise users). Click to tap and click and drag to swipe. System Keys gives Back, Home, Recent Apps, Volume Up, Volume Down, Rotate and Lock Screen / Wake. The expand button in the frame’s toolbar makes the phone bigger without restarting the stream.
Release the device
Click Release, or close the device view.
The device returns to Available.
Run automated tests on the farm
For one test case, start a dry run in Automate on a Device Farm device; Control the device lets you tap on the device during the run. For a suite, open the pipeline in Schedule, choose SmartTest Device Farm as the Run Environment, and on the Mobile Settings tab choose the devices and the app per platform. The build is installed before the run and skipped when the same version is already on the device. See Mobile projects.
Check the run
Open Recent runs. Each run on a farm device is a row with its device, status, start time and minutes. The metrics icon opens Device Metrics with four charts: Temperature, Battery, Memory and CPU. See Recent runs.
Where the devices come from#
A mobile project can use four kinds of device. SmartestQA has two ways to go live in the browser: a Device Farm device and a device on your agent.
| Devices | What they are | You need | How you reach them |
|---|---|---|---|
| SmartTest Device Farm | Devices on Device Farm nodes: SmartestQA’s own real phones, and phones your company attached to its own nodes | The Device Farm right, a MOBILE project, your app in the App Library | Live: Devices › Use device › Take device |
| A device on your agent | A phone or emulator connected to the computer that runs your agent | The agent online, Appium on port 4723, the device listed under Connected Devices | Live: Test Agents › Connected Devices › Use Device Live |
| AWS Device Farm | Devices in your AWS account | AWS keys, a Device Farm project, the app uploaded; set in Mobile Config | Chosen as the run environment of a mobile pipeline |
| BrowserStack | Devices in your BrowserStack account | BrowserStack user name and access key, the app uploaded there; set in Mobile Config | Chosen as the run environment of a mobile pipeline |
Numbers and limits#
| Item | Value |
|---|---|
| App file types | .apk and .aab (Android), .ipa (real iPhones and iPads), .zip of the .app folder (iOS simulators) |
| Apps in the App Library | 30 per company, one stored build each |
| Live session lease | 180 seconds, renewed while the stream is open |
| Live reconnect attempts | 3, after 4 s, 10 s and 20 s |
| App install on a device | 6 minutes, then “timed out” |
| One farm run | 120 minutes, then closed as Failed |
| Recent runs loaded | 200 (users of the cloud service: 100) |
| Live control | Android: taps, swipes, system keys, restart. iOS: watch only |
| Run metrics | Android runs only. Emulators have no temperature or battery sensor |
The full table is in Limits; how the minutes of a run are counted is in Run minutes.
What you get#
- A phone in the browser. The stream starts as Preparing, then Connecting, then Live. The footer shows the codec:
h264orwebrtcfor Android,mjpegfor iOS. - A device that is yours while you work. A taken farm device shows Busy “In live use”, and runs cannot take it.
- One place for builds. The App Library holds up to 30 apps per company. Replace app puts a new build in the place of the old one, so pipelines that use the app get the new build without any change.
- Manual and automated testing on the same devices. Admins mark a device Automation, Manual or Dual use. Runs that pick a device automatically never take a Manual device.
- Network and reset controls for Admins. Airplane mode and Wi-Fi test how your app reacts to losing the connection. Set as baseline and Reset to baseline return a device to a clean starting point.
- Evidence per run. Recent runs lists each farm run with its status and minutes, and Android runs keep temperature, battery, memory and CPU charts.
- A safety net. If your browser or network dies, the device frees itself at the latest about three minutes later.
Questions and answers#
Do I need to own the phones?Customers
No. With the Device Farm right, a SmartestQA mobile project can run on SmartTest Device Farm, where the app runs on SmartestQA’s own real phones. A project can also run on devices in your AWS Device Farm or BrowserStack account. See Execution modes.
Can we connect our own phones to the Device Farm?Customers
Yes, with the Device Farm right. An Admin creates a node in SmartestQA and runs its install command on the computer the phones are attached to. Devices on your own nodes are reserved for your company. See Add a node.
What can I do without the Device Farm right?Customers
Your agents keep working. Connect a phone or start an emulator on the computer with your agent; in a MOBILE project it appears under Connected Devices in Test Agents, and Use Device Live opens its screen in the browser. Mobile tests can always run on your own agent with your own devices. See Open a device on your agent.
Can I control an iPhone live from the browser?Customers
Not yet. In a SmartestQA live session you can watch the screen of an iOS device or simulator, but taps, swipes and system keys work on Android only, and Restart is disabled for iOS. iOS devices need an agent or node on a Mac. See What you can do in a session.
Can I type on the phone with my computer keyboard?Customers
Not yet. In a live session a click is a tap and a drag is a swipe, but typing on your keyboard is not passed to the device. Use the on-screen keyboard of the device. See What you can do in a session.
What happens to the device if my browser crashes?Customers
The device is held for you on a lease of 180 seconds that renews while the stream is open. If the browser or the network dies, the device frees itself at the latest about three minutes later. An Admin can free it at once with Force release. See Controls.
What happens when the live stream drops?Customers
The window retries by itself up to three times and shows “Reconnecting… (n/3)”. After the third failed retry you see “Connection failed” and a Try Again button. Permission errors are not retried. See When the connection drops.
Do I have to install the app on the device before every run?Customers
No. A run that targets the Device Farm installs the app you picked from the App Library before it starts, and skips the install when the same version is already on the device. Install by hand only to explore a build live or to prepare devices. See Install on devices.
Do I upload the app again for every new build?Customers
Expand the app in the App Library and use Replace app. The confirmation is “file name: new version uploaded”, and package id, activity and version are read again from the new file. Adding the same app again creates a second app and counts against the limit of 30 apps. See Upload a new version.
Who can take a device, install apps and change the App Library?Customers
Admins and on-premise users can take any device of the company; users of the cloud service can take devices assigned to their company. Admins add, replace and delete apps in the App Library; users read it. Restart, the network switches, the baseline, the usage mode and Force release are for Admins. See Who can do what.
Read more#
- Agent Pool & Device Farm: the tabs, the building blocks and what each role sees.
- Live device sessions: both ways to go live, the keys and the connection states.
- Nodes, devices & runs: device states, filters, controls and Recent runs.
- App Library: upload, replace, fix app details and install on devices.
- Mobile Config: Local device, SmartTest Device Farm, AWS Device Farm and BrowserStack.
- Where a pipeline runs: the Run Environment of mobile pipelines.
- How do I build, run and schedule a regression suite?: the pipeline that runs on these devices.
Didn’t find your answer?Write to support with your project name and what you see on screen.
info@smartestqa.ai