AppVeyor vs Azure Test Plans
AppVeyor
appveyor.com
Azure Test Plans
azure.microsoft.comSide by side
| Figure | AppVeyor | Azure Test Plans |
|---|---|---|
| Price from | $29/mo | $6/mo |
| Free plan | Yes | Yes |
| Free trial | Yes | Yes |
| Open source | — | — |
| Web | Yes | Yes |
| Windows | Yes | Yes |
| macOS | Yes | Yes |
| Linux | Yes | Yes |
| API | Yes | Yes |
| Self-hosted | Yes | Yes |
| Plans published | 7 | 5 |
| Spec score · Continuous Integration Software | 9.5 (#1) | 9.4 (#3) |
| Spec score · Game Build Automation Software | 9.5 (#1) | 9.4 (#3) |
| Spec score · Ci Cd Platforms | 9.5 (#1) | 9.4 (#2) |
| Company facts | AppVeyor Systems Inc. is described by the company as a privately held Canadian corporation founded in 2011. | LinkedIn lists Microsoft as a software-development company headquartered in Redmond, Washington, with more than 10,001 employees and approximately 228,000 employees worldwide. |
| Founded | 2011 | Microsoft was founded in 1975. |
| Self-hosting | AppVeyor Server can be installed locally or in Docker and cloud environments on Windows, Linux or macOS. | The same test-management documentation covers Azure DevOps Services and Azure DevOps Server, including Server 2022. |
| What it does | AppVeyor provides hosted continuous integration and deployment for building, testing and deploying software. | Azure Test Plans lets teams plan, execute and track manual, exploratory and automated testing within Azure DevOps. |
Plans
AppVeyor
- AppVeyor ServerFree
- Open-sourceFree
- Basic$29 a month
- Pro$59 a month
- Premium$99 a month
- Pro$590 a year
- Premium$990 a year
Azure Test Plans
- Basic + Test Plans$52 a month
- BasicFree
- StakeholdersFree
- Stakeholders (Free)Free
- Basic$6 a month
In their own words
AppVeyor
AppVeyor provides continuous integration tools for Windows developers, with hosted service, self-hosted jobs and an Enterprise installation option for customer premises. A build pipeline can retrieve code, restore caches, install dependencies, build and test software, package artifacts, deploy, then run finalizer scripts. Projects can be configured through an appveyor.yml file or the user interface. Build matrices can create jobs across operating systems, variables, platforms, configurations, test categories and build clouds. Listed source control providers include GitHub, Bitbucket, GitLab, Azure DevOps, Kiln and Gitea. Deployment options include inline providers and environments triggered manually or through the API. The REST API covers teams, projects, builds, environments and deployments, with bearer-token authentication. Secure variables can be encrypted with account-specific keys and are not decoded during pull-request builds by default. Hosted builds use pristine virtual machines not shared with other builds, and VM state is not preserved. Every plan has a 60-minute maximum build-job execution time. Paid plans start at $29 a month.
Azure Test Plans
Azure Test Plans gives teams tools to plan, run, and track manual and exploratory testing for apps. For scripted tests, teams can track actionable defects and connect test cases and suites to user stories, features, or requirements. Exploratory sessions let testers design and run tests at the same time. During manual tests, Test Runner can capture screenshots, image action logs, and screen recordings. Reports, dashboard widgets, and customizable Power BI reports show plan progress and trends. The service integrates with Azure Pipelines for CI/CD testing and offers REST APIs and an extension contribution model. Manual testing is available in the web portal through supported browsers on major platforms; a free Test & Feedback extension supports exploratory testing in Chrome and Microsoft Edge. The free Stakeholders plan does not allow users to create or manage Test Plans. Basic is free for the first five users, then $6 per user per month; Basic + Test Plans costs $52 a month, billed at $52 per user per month. A 30-day trial is listed. Azure DevOps cloud-based load testing is deprecated; Microsoft points users to Azure Load Testing for managed load testing.