Cloudera Data Lake Service vs HPCC Systems

Cloudera Data Lake Service provides data governance, metadata management, access controls, and auditing for data lake workloads. It captures schema and metadata definitions that platform workloads discover or create, then supports search, lineage, chain-of-custody auditing, classification, profiling, and a business glossary. Security capabilities include role- and attribute-based policies, auditing of unauthorized access, and encryption at rest and in motion with key management. Apache Ranger can apply access policies at database, table, column, and file levels, including dynamic conditions such as time or geolocation. Apache Knox provides a protected single sign-on gateway to Data Lake and Cloudera Data Platform interfaces. A cluster includes Hive Metastore, Ranger, Atlas, and Knox. Cloudera supports light-duty and enterprise Data Lakes on AWS, Azure, and GCP. Listed capabilities also include SQL analytics, streaming ingestion, data sharing, and open table formats. The service has web and API access and supports hybrid deployment. The listed pricing note starts at $0.20, while the plan is also described as Contact Sales for pricing; cloud prices exclude infrastructure and networking costs.

HPCC Systems is a free, open-source platform for enterprise big-data processing and data-lake management. Its core components are ECL, a declarative and modular language, Thor for data refinement, and Roxie for data delivery. ECL's compiler optimizes programs for parallel processing and compiles them into C++. Thor handles ingestion, transformation, linking, and indexing; Roxie serves concurrent queries. Clusters can scale from two computers to more than a thousand commodity-hardware nodes. The cloud-native platform runs on Kubernetes and supports Azure Kubernetes Service and Amazon Elastic Kubernetes Service. Storage options include AWS S3, Azure Blob Storage, AWS Elastic Block Store, Azure Files, and Azure Disks. Security capabilities include end-to-end encryption, service meshes, OAuth 2.0 with Azure AD support, JWT, and configurable controls for data and services. The Machine Learning Library offers ECL-accessible algorithms for building and testing prediction models. Integrations include Spark, Kafka, Couchbase, Redis, Memcached, Pentaho, R, JDBC, Java APIs, and Tableau data connectors. ESP exposes queries through XML, HTTP, SOAP, and REST interfaces, and deployed queries can be called with REST/JSON. The platform is free to use under the Apache 2.0 license and is self-hosted.