Azure Test Plans vs Rally Software
Azure Test Plans
azure.microsoft.comRally Software
rallysoftware.comSide by side
| Figure | Azure Test Plans | Rally Software |
|---|---|---|
| Price from | $6/mo | Free |
| Free plan | Yes | Yes |
| Free trial | Yes | — |
| Open source | — | — |
| Web | Yes | Yes |
| Windows | Yes | — |
| macOS | Yes | — |
| Linux | Yes | — |
| API | Yes | Yes |
| Self-hosted | Yes | Yes |
| Plans published | 5 | 1 |
| Spec score · Application Lifecycle Management Software | 9.5 (#1) | 8.0 (#6) |
| API | Public REST endpoints use dev.azure.com organization URLs and versioned _apis paths; the Test Plans API includes plan listing and requires the vso.test scope for read access. | Broadcom documentation describes Rally’s Web Services API and API keys for Rally SaaS; API keys are not supported on-premises. |
| Compliance | Azure DevOps lists ISO 27001:2022, ISO 27018:2019, ISO 26262:2023, HIPAA, BAA, EU Model Clauses, SOC 1 Type 2 and SOC 2 Type 2 among its certifications or commitments. | Broadcom’s SOC page lists Rally Software (SaaS) as in scope for SOC 1, SOC 2, and SOC 3 reports. |
| Integrations | Azure DevOps exposes integrations with GitHub, Docker, Slack, Sentry and more through its Marketplace extension ecosystem. | Rally integrates with Clarity and third-party tools through ValueOps ConnectALL, including ITSM, DevOps, Jira, PPM, DPM/SPM, and collaboration tools. |
| Purpose | Azure Test Plans provides planned manual and exploratory testing tools for apps. | Rally helps organizations plan, prioritize, manage, track, and measure work to deliver customer value. |
| Reporting | The product includes configurable test charts, progress reports, a Test Runs hub, dashboard widgets and Analytics-based reporting. | Rally provides real-time enterprise reporting and metrics including team burndown, velocity, productivity, responsiveness, and quality. |
| Security | Azure DevOps states that project data is protected in transit and at rest, uses the Microsoft Security Development Lifecycle and undergoes security penetration testing. | Broadcom’s Rally SaaS privacy notice lists access controls, encryption at rest and in transit, and change logging among its privacy-enhancing measures. |
Plans
Azure Test Plans
- Basic + Test Plans$52 a month
- BasicFree
- StakeholdersFree
- Stakeholders (Free)Free
- Basic$6 a month
In their own words
Azure Test Plans
Azure Test Plans gives teams tools to plan, run, and track manual and exploratory testing for apps. For scripted tests, teams can track actionable defects and connect test cases and suites to user stories, features, or requirements. Exploratory sessions let testers design and run tests at the same time. During manual tests, Test Runner can capture screenshots, image action logs, and screen recordings. Reports, dashboard widgets, and customizable Power BI reports show plan progress and trends. The service integrates with Azure Pipelines for CI/CD testing and offers REST APIs and an extension contribution model. Manual testing is available in the web portal through supported browsers on major platforms; a free Test & Feedback extension supports exploratory testing in Chrome and Microsoft Edge. The free Stakeholders plan does not allow users to create or manage Test Plans. Basic is free for the first five users, then $6 per user per month; Basic + Test Plans costs $52 a month, billed at $52 per user per month. A 30-day trial is listed. Azure DevOps cloud-based load testing is deprecated; Microsoft points users to Azure Load Testing for managed load testing.
Rally Software
Rally Software helps organizations plan work across teams and connect investment choices to delivery. It offers portfolio-level planning, iteration and release planning, capacity planning, and customizable Scrum and Kanban dashboards. Teams can associate work with enterprise objectives and track alignment, progress, dependencies, blockers, and risks across programs and release trains. Reporting includes team burndown, velocity, productivity, responsiveness, and quality. Rally describes AI features for forecasting outcomes, predicting risks, identifying bottlenecks, and modeling capacity, as well as generating and breaking down business outcomes into user stories and creating summaries. Integrations include Clarity, ConnectALL, Insights, Jira, CI/CD pipelines, ITSM, collaboration tools, and ServiceNow. It is available as SaaS or on-premises through Rally Anywhere, which can support data residency and sovereignty requirements. The listed free Starter Edition is limited to 50 named users and organizations in listed countries. Pricing for other options is available on request.