Domino Model Monitor vs NannyML
Domino Model Monitor
domino.ai
NannyML
nannyml.comSide by side
| Figure | Domino Model Monitor | NannyML |
|---|---|---|
| Price from | — | $99/mo |
| Free plan | — | Yes |
| Free trial | — | Yes |
| Open source | — | Yes |
| Web | Yes | Yes |
| Linux | — | Yes |
| API | Yes | Yes |
| Self-hosted | Yes | Yes |
| Plans published | 3 | 7 |
| Spec score · Machine Learning Model Monitoring Software | 7.5 (#11) | 8.8 (#3) |
| Drift monitoring | The monitoring platform tracks data drift and model quality degradation with integrated monitoring and alerts. | NannyML detects concept drift and multivariate and univariate data drift, and relates drift alerts to changes in model performance. |
| Founded | 2013 | Belgian company records state that NannyML was established on 19 March 2020. |
| Integrations | Domino's partner ecosystem includes Anaconda, Snowflake, Palantir, RStudio, SageMaker, MathWorks, Stata, Collibra, DataRobot, H2O.ai, Okera, Teradata, Trifacta, and Pinnacle 21. | NannyML Cloud is offered through both Azure Marketplace and AWS Marketplace. |
| Purpose | Domino Model Monitor automates model evaluation and alerts when production models degrade enough to require expert review. | NannyML monitors deployed machine-learning models and estimates model performance even when ground truth is delayed or absent. |
| Support | The Cloud and Premium plans include premium support with a two-business-day non-urgent SLA, while Enterprise includes one-business-day non-urgent support. | The pricing page lists email support for Starter, private Slack for Scale, and 24/7 support for Enterprise. |
Plans
Domino Model Monitor
- Domino CloudPriced on request
- EnterprisePriced on request
- PremiumPriced on request
NannyML
- Open SourceFree
- Starter (beta)$99 a month
- Starter$399 a month
- Scale$999 a month
- Pro (beta)$3,800 a month
- EnterprisePriced on request
- Enterprise (beta)Priced on request
In their own words
Domino Model Monitor
Domino Model Monitor evaluates production machine-learning models and alerts when degradation warrants expert review. It monitors data drift and model quality from a central dashboard showing status, summaries, customized alerts, and monitoring results. Automated prediction-data pipelines and metric generation support ongoing checks. When remediation is needed, teams can recreate a model’s development environment, retrain with production data, and redeploy a version. Models built in Python, R, SAS, MATLAB, TensorFlow, or DataRobot can be registered. The Elastic Monitoring Engine is designed to scale and ingest data from Amazon S3 and Hadoop-compatible storage, including Azure Blob Storage, Azure Data Lake, Google Cloud Storage, and HDFS. Domino supports API, web, and self-hosted use, including cloud VPCs, on-premises systems, and air-gapped environments. Pricing is on request, with Premium, Domino Cloud, and Enterprise plans listed.
NannyML
NannyML monitors deployed machine-learning models and estimates their performance when ground truth is delayed or unavailable. It detects concept drift as well as multivariate and univariate data drift, and connects alerts to changes in model performance. Data-quality checks and ranked drift alerts can help teams investigate features associated with performance issues. Users can relate model performance to monetary or other business outcomes. NannyML Cloud supports webhooks for retraining actions and an SDK for automating monitoring-data ingestion. Cloud is available through Azure and AWS marketplaces as SaaS or a managed application. With the managed-application option, the application and required infrastructure are provisioned within the customer's cloud subscription. For its Azure managed application, NannyML documents encrypted traffic and disks, but says automated backups were still being implemented and advises customers to safeguard model inputs and outputs elsewhere. The open-source library installs with pip or conda. Direct local dataset uploads through the Cloud UI must be smaller than 100 MB. Plans include a free self-managed Open Source option, Starter at $399 a month, and Scale at $999 a month.