AWS Device Farm vs Web Platform Tests
AWS Device Farm
aws.amazon.comWeb Platform Tests
web-platform-tests.orgSide by side
| Figure | AWS Device Farm | Web Platform Tests |
|---|---|---|
| Price from | $200/mo | Free |
| Free plan | — | Yes |
| Free trial | Yes | — |
| Open source | — | — |
| Web | Yes | Yes |
| Windows | — | Yes |
| macOS | — | Yes |
| Linux | — | Yes |
| iOS | Yes | — |
| Android | Yes | — |
| API | Yes | — |
| Self-hosted | — | Yes |
| Plans published | 4 | 1 |
| Spec score · Cross-Browser Testing Software | 8.4 (#8) | 7.5 (#22) |
| Spec score · Browser Testing Software | 8.4 (#15) | 7.5 (#32) |
| Integrations | Service plugins and an API can initiate tests and retrieve results from IDEs and continuous integration environments such as Android Studio and Jenkins. | GitHub is used for issue tracking and test submissions, and pull requests receive automatic labels and reviewer notifications based on directory metadata. |
| Purpose | AWS Device Farm tests web and mobile applications across desktop browsers and real mobile devices hosted on AWS. | Web Platform Tests is a cross-browser test suite for the Web platform, intended to help browser projects check compatibility across implementations. |
| Security | Device Farm requires TLS 1.2 and recommends TLS 1.3 for network access through its published APIs. | The project documentation says SSL certificates for HTTPS-enabled domains are obtained through Let’s Encrypt. |
Plans
AWS Device Farm
- Desktop browser testing$0.01, paid once
- Real mobile devices — pay as you go$0.17, paid once
- Private devices$200 a month
- Unlimited test and access$250 a month
In their own words
AWS Device Farm
AWS Device Farm has no write-up on Specifiction yet; its figures are above.
Web Platform Tests
Web Platform Tests is an open-source suite for checking web platform compatibility across browser implementations. Anyone can run tests through the public wpt.live deployment in an internet-enabled browser, or clone the suite for local use on Linux, macOS, or Windows. The suite includes JavaScript API, rendering, crash, WebDriver, and manual tests. Its command-line `wpt` tool runs tests in a selected browser and can produce machine-readable JSON reports and screenshots. It can also save and load expected results for continuous integration and regression tracking. Regularly collected results from a range of browsers are archived at wpt.fyi. Local setup requires Python 3.9 or later and configuration of test domains in the system hosts file. Running tests in multiple browser processes may make results less deterministic and add noise compared with the default single-process mode. The project is free and open source, and documents Windows, macOS, and Linux setup.