FiboRisk Model Monitoring vs NannyML

FiboRisk Model Monitoring is web software for monitoring model performance, drift, bias, latency, overrides, and realized outcomes against an accepted model and policy version. Listed capabilities include performance monitoring, drift detection, bias and segment review, and alert management. Evidence objects connect datasets, models, policies, decisions, reviewers, and release states. Evidence records can version dataset and feature details, model and policy versions, acceptance status, reviewer role, and decision timestamps; records are attributable and exportable within the agreed delivery scope. The product is aimed at validation or audit teams and defines roles for validation, approval, operation, and audit. Delivery uses a dedicated tenant or agreed private deployment. Production access follows model and environment acceptance, with named-user invitations managed by customer administrators. Security, privacy, data, retention, transfer, encryption, export, and deletion controls are defined for each deployment boundary. Integrations and costs are confirmed in the statement of work.

NannyML monitors deployed machine-learning models and estimates their performance when ground truth is delayed or unavailable. It detects concept drift as well as multivariate and univariate data drift, and connects alerts to changes in model performance. Data-quality checks and ranked drift alerts can help teams investigate features associated with performance issues. Users can relate model performance to monetary or other business outcomes. NannyML Cloud supports webhooks for retraining actions and an SDK for automating monitoring-data ingestion. Cloud is available through Azure and AWS marketplaces as SaaS or a managed application. With the managed-application option, the application and required infrastructure are provisioned within the customer's cloud subscription. For its Azure managed application, NannyML documents encrypted traffic and disks, but says automated backups were still being implemented and advises customers to safeguard model inputs and outputs elsewhere. The open-source library installs with pip or conda. Direct local dataset uploads through the Cloud UI must be smaller than 100 MB. Plans include a free self-managed Open Source option, Starter at $399 a month, and Scale at $999 a month.