Hadoop Distributed File System vs Tahoe-LAFS
Hadoop Distributed File System
hadoop.apache.orgTahoe-LAFS
tahoe-lafs.orgSide by side
| Figure | Hadoop Distributed File System | Tahoe-LAFS |
|---|---|---|
| Price from | Free | Free |
| Free plan | Yes | Yes |
| Free trial | — | — |
| Open source | — | — |
| Web | — | Yes |
| Windows | Yes | Yes |
| macOS | Yes | Yes |
| Linux | Yes | Yes |
| API | Yes | Yes |
| Self-hosted | Yes | Yes |
| Plans published | 1 | 1 |
| Spec score · Distributed File System Software | 8.2 (#9) | 8.3 (#6) |
| Encryption | Configured HDFS encryption provides transparent end-to-end encryption for data at rest and in transit using encryption zones and the Hadoop Key Management Server. | The client encrypts each file with a unique key before uploading it to storage servers. |
| Fault tolerance | HDFS is highly fault-tolerant and designed to run on low-cost commodity hardware. | Its erasure coding lets a client recover files from a subset of shares even if some storage servers fail. |
| License | Apache Hadoop is licensed under the Apache License 2.0. | The project offers the software under GNU GPL version 2 or later, or the Transitive Grace Period Public Licence version 1.0 or later. |
| Purpose | HDFS is the primary distributed storage used by Hadoop applications. | Tahoe-LAFS is a decentralized file storage system that encrypts files, splits them into shares, and distributes those shares across storage servers. |
| Support | Apache Hadoop support is provided through project mailing lists maintained by non-paid volunteers. | The installation guide directs users who need help with deployment to the tahoe-dev mailing list or Libera.Chat IRC. |
Plans
Hadoop Distributed File System
- Apache Hadoop (open source)Free
In their own words
Hadoop Distributed File System
Hadoop Distributed File System (HDFS) is the primary distributed storage used by Hadoop applications. Its clusters use a NameNode to manage filesystem metadata and DataNodes to store data. HDFS is designed for fault tolerance on commodity hardware and high-throughput streaming access and batch processing, not low latency. It is tuned for files from gigabytes to terabytes, clusters with hundreds of nodes, and tens of millions of files in one instance. Administration includes shell-like commands, NameNode and DataNode web servers, fsck diagnostics, safemode, and a cluster balancer. Its permissions model shares much with POSIX and can optionally use POSIX ACLs; authentication supports simple operating-system identity or Kerberos. Configured encryption protects data at rest and in transit using encryption zones and the Hadoop Key Management Server. WebHDFS provides a REST API, and listed storage integrations include Amazon S3, Azure Blob Storage, Azure Data Lake Storage, and Tencent COS. HDFS is free to use under Apache License 2.0, with self-hosted cluster deployment.
Tahoe-LAFS
Tahoe-LAFS is free, open-source software for storing files across multiple servers. Before upload, a client encrypts each file with a unique key. Erasure coding distributes each file as shares, allowing recovery when some storage servers fail; the documented default uses 10 shares across at least 7 servers and can recover from any 3. A gateway checks integrity and ignores corrupted shares. Files can be immutable or mutable, with read-only or read-write access for mutable files and directories. The gateway gives access through HTTP(S) and SFTP, and a REST-style Web API supports programmatic operations. Users can run a gateway or operate a full grid of storage nodes and an introducer. Installation is documented for Windows, macOS, Linux, and BSD. The software is available under GNU GPL version 2 or later, or the Transitive Grace Period Public Licence version 1.0 or later. It is free at free. The public test grid is for experimentation, not reliable or large-scale storage.