Azure Test Plans vs Cloudsmith
Azure Test Plans
azure.microsoft.com
Cloudsmith
cloudsmith.comSide by side
| Figure | Azure Test Plans | Cloudsmith |
|---|---|---|
| Price from | $6/mo | $149/mo |
| Free plan | Yes | Yes |
| Free trial | Yes | Yes |
| Open source | — | — |
| Web | Yes | Yes |
| Windows | Yes | Yes |
| macOS | Yes | Yes |
| Linux | Yes | Yes |
| API | Yes | Yes |
| Self-hosted | Yes | — |
| Plans published | 5 | 5 |
| Spec score · Artifact repository software | 9.4 (#1) | 8.9 (#3) |
| Compliance | Azure DevOps lists ISO 27001:2022, ISO 27018:2019, ISO 26262:2023, HIPAA, BAA, EU Model Clauses, SOC 1 Type 2 and SOC 2 Type 2 among its certifications or commitments. | iso27001, sso saml, two factor |
| Integrations | Azure DevOps exposes integrations with GitHub, Docker, Slack, Sentry and more through its Marketplace extension ecosystem. | Listed integrations include GitHub Actions, GitLab CI/CD, Jenkins, Azure DevOps, CircleCI, Buildkite, Argo CD, Terraform, Slack, Microsoft Teams, Datadog, Aikido, Zapier and VS Code. |
| What it does | Azure Test Plans lets teams plan, execute and track manual, exploratory and automated testing within Azure DevOps. | Cloudsmith is a fully managed, cloud-native artifact management platform serving as a single source of truth for software artifacts. |
Plans
Azure Test Plans
- Basic + Test Plans$52 a month
- BasicFree
- StakeholdersFree
- Stakeholders (Free)Free
- Basic$6 a month
Cloudsmith
- Free Open-SourceFree
- Pro$149 a month
- CorePrice not published
- EnterprisePriced on request
- UltraPriced on request
In their own words
Azure Test Plans
Azure Test Plans gives teams tools to plan, run, and track manual and exploratory testing for apps. For scripted tests, teams can track actionable defects and connect test cases and suites to user stories, features, or requirements. Exploratory sessions let testers design and run tests at the same time. During manual tests, Test Runner can capture screenshots, image action logs, and screen recordings. Reports, dashboard widgets, and customizable Power BI reports show plan progress and trends. The service integrates with Azure Pipelines for CI/CD testing and offers REST APIs and an extension contribution model. Manual testing is available in the web portal through supported browsers on major platforms; a free Test & Feedback extension supports exploratory testing in Chrome and Microsoft Edge. The free Stakeholders plan does not allow users to create or manage Test Plans. Basic is free for the first five users, then $6 per user per month; Basic + Test Plans costs $52 a month, billed at $52 per user per month. A 30-day trial is listed. Azure DevOps cloud-based load testing is deprecated; Microsoft points users to Azure Load Testing for managed load testing.
Cloudsmith
Cloudsmith has no write-up on Specifiction yet; its figures are above.