Skip to content
SPECIFICTION
Get listed

HPCC Systems vs lakeFS

HPCC Systems

hpccsystems.com

lakeFS

lakefs.io

Side by side

FigureHPCC SystemslakeFS
Price fromFree—
Free planYes—
Free trial—Yes
Open sourceYes—
WebYesYes
WindowsYes—
macOSYes—
LinuxYes—
APIYesYes
Self-hostedYesYes
Plans published31
Spec score · Data Lake Software8.7 (#2)7.5 (#9)
Founded20002020
IntegrationsOfficial integrations include Spark, Kafka, Couchbase, Redis, Memcached, Pentaho, R, JDBC, Java APIs and Tableau data connectors.The integrations documentation lists tools including Spark, Trino, DuckDB, Databricks, MLflow, SageMaker, Vertex AI, Kubeflow, Airflow, and Airbyte.
PurposeHPCC Systems is an open-source, data-intensive supercomputing platform for enterprise big-data problems.lakeFS provides governed access to petabyte-scale multimodal data and records data changes for auditability.
StorageThe cloud-native storage plane supports AWS S3, Azure Blob Storage, AWS Elastic Block Store, Azure Files and Azure Disks.lakeFS supports AWS S3, Azure Blob, Google Cloud Storage, and on-premises object stores such as MinIO.

Plans

HPCC Systems

  • Community EditionFree
  • Open sourceFree
  • Enterprise subscriptionPriced on request

HPCC Systems pricing →

lakeFS

  • lakeFS EnterprisePriced on request

lakeFS pricing →

In their own words

HPCC Systems

HPCC Systems is a free, open-source platform for enterprise big-data processing and data-lake management. Its core components are ECL, a declarative and modular language, Thor for data refinement, and Roxie for data delivery. ECL's compiler optimizes programs for parallel processing and compiles them into C++. Thor handles ingestion, transformation, linking, and indexing; Roxie serves concurrent queries. Clusters can scale from two computers to more than a thousand commodity-hardware nodes. The cloud-native platform runs on Kubernetes and supports Azure Kubernetes Service and Amazon Elastic Kubernetes Service. Storage options include AWS S3, Azure Blob Storage, AWS Elastic Block Store, Azure Files, and Azure Disks. Security capabilities include end-to-end encryption, service meshes, OAuth 2.0 with Azure AD support, JWT, and configurable controls for data and services. The Machine Learning Library offers ECL-accessible algorithms for building and testing prediction models. Integrations include Spark, Kafka, Couchbase, Redis, Memcached, Pentaho, R, JDBC, Java APIs, and Tableau data connectors. ESP exposes queries through XML, HTTP, SOAP, and REST interfaces, and deployed queries can be called with REST/JSON. The platform is free to use under the Apache 2.0 license and is self-hosted.

Visit hpccsystems.com

lakeFS

lakeFS provides data version control and governance for teams working with petabyte-scale multimodal data. Its engine records changes and uses branches, commits, merges, and rollbacks to manage data versions. Branches create isolated environments without copying data or adding storage costs. Write-Audit-Publish workflows use automated hooks and human review to check changes before merging. Immutable commits, tags, and lineage connect experiments and agent actions to the data versions behind them. Listed controls include role-based access, geo-based policies, data immutability, SSO, and SCIM; lakeFS states that it is SOC 2 compliant and provides audit trails and lineage. It works with AWS S3, Azure Blob, Google Cloud Storage, and on-premises object stores such as MinIO. Integrations include Spark, Trino, DuckDB, Databricks, MLflow, SageMaker, Vertex AI, Kubeflow, Airflow, and Airbyte. Enterprise can run as managed lakeFS Cloud or as a self-managed deployment on-premises, in a private cloud, or in an air-gapped environment. Pricing is on request, and a 15-day trial includes full Enterprise access and support from lakeFS experts.

Visit lakefs.io