DeviceFarmer vs TestingBot
DeviceFarmer
devicefarmer.github.io
TestingBot
testingbot.comSide by side
| Figure | DeviceFarmer | TestingBot |
|---|---|---|
| Price from | — | $20/mo |
| Free plan | — | — |
| Free trial | — | Yes |
| Open source | — | — |
| Web | Yes | Yes |
| Windows | — | Yes |
| macOS | Yes | Yes |
| Linux | Yes | Yes |
| iOS | — | Yes |
| Android | Yes | Yes |
| API | Yes | Yes |
| Self-hosted | Yes | — |
| Plans published | 0 | 8 |
| Spec score · Device Farm Software | 7.8 (#15) | 8.5 (#4) |
| API | The project documents a REST API for reserving and releasing devices, with OAuth 2.0 authentication. | TestingBot provides a public REST API and webhooks for custom integrations. |
| Deployment | The README describes installation via NPM and provides a Docker container; it says Linux or BSD distributions should work for production. | The pricing FAQ says Enterprise supports private cloud and that on-premise deployments are not supported. |
| Integrations | The README lists Android Studio and Chrome remote debugging support, while its API documentation describes using the API with Appium. | Official integrations include Slack, Jira, GitHub and CI/CD systems such as GitHub Actions, Jenkins, CircleCI, GitLab CI, Bitbucket Pipelines, Travis CI, TeamCity and Azure DevOps. |
| Security | The README says internal process communication is insecure and unencrypted, devices are not fully reset between uses, and multiuser deployments may expose sensitive data. | TestingBot states that each test uses a single-use machine and wiped device, with infrastructure in European data centers and GDPR and CSA STAR claims. |
Plans
TestingBot
- Live€20 a year
- Live$20 a month
- Automated€50 a year
- Automated$50 a month
- Pay As You Go€60, paid once
- Automated Pro$90 a month
- Automated Pro€90 a year
- EnterprisePriced on request
In their own words
DeviceFarmer
DeviceFarmer is a free, self-hosted web application for remotely debugging real smartphones, smartwatches, and other devices through a browser. It provides real-time screen viewing, rotation, keyboard input, copy and paste, multitouch, drag-and-drop APK installation, shell commands, logs, file browsing, and remote ADB connections. With adb connect, users can run local ADB commands or work in Android Studio and other IDEs while viewing the device online. Inventory tools show connection status, availability, hardware specifications, battery health, searchable attributes, and current users. Administrators can assign device sets to projects or organizations indefinitely, or book devices for users for limited periods. DeviceFarmer includes a REST API with OAuth 2.0 authentication, Prometheus metrics, Appium support, and Chrome remote debugging. Listed support covers Android 2.3.3 through 15, Wear 5.1, Fire OS, CyanogenMod, and other Android-based distributions. Installation uses NPM or Docker; Linux or BSD is suggested for production.
TestingBot
TestingBot provides a cloud testing grid for automated and manual testing on real browsers and physical mobile devices. Its options include live manual, visual, automated, and AI-assisted testing, with support stated for Selenium, Appium, Playwright, Cypress, Puppeteer, Espresso, XCUITest, and Maestro. Automated tests can run in parallel, with concurrency controlled through test configuration or a CI pipeline. Listed integrations include Jenkins, GitHub Actions, GitLab CI, CircleCI, Azure DevOps, Bamboo, TeamCity, Travis CI, Jira, and Slack. A public REST API and webhooks support custom integrations. TestingBot lists a free plan and a 14-day trial. Live costs €20 a year, billed annually, or €30 month-to-month; Automated costs €50 a year, billed annually, or €70 month-to-month. Pay As You Go costs €60, paid once payment, and Automated Pro costs €90 a year, billed annually, or €120 month-to-month. Enterprise pricing is by contact. TestingBot says sessions use isolated machines or devices wiped after testing, with test artifacts stored and processed in EU datacenters.