Google Cloud Filestore vs Hadoop Distributed File System
Google Cloud Filestore
cloud.google.comHadoop Distributed File System
hadoop.apache.orgSide by side
| Figure | Google Cloud Filestore | Hadoop Distributed File System |
|---|---|---|
| Price from | $0.01/mo | Free |
| Free plan | — | Yes |
| Free trial | Yes | — |
| Open source | — | — |
| Web | Yes | — |
| Windows | Yes | Yes |
| macOS | — | Yes |
| Linux | Yes | Yes |
| API | Yes | Yes |
| Self-hosted | — | Yes |
| Plans published | 8 | 1 |
| Spec score · Distributed File System Software | 8.2 (#8) | 8.2 (#9) |
| Encryption | Filestore encrypts customer content at rest by default and supports customer-managed Cloud KMS keys for instances and backups on zonal, regional, and enterprise tiers. | Configured HDFS encryption provides transparent end-to-end encryption for data at rest and in transit using encryption zones and the Hadoop Key Management Server. |
| Purpose | Filestore is a fully managed shared NFS file service for applications that need persistent file storage. | HDFS is the primary distributed storage used by Hadoop applications. |
| Scale | The product page lists capacity up to 100 TiB, throughput up to 26 GiB/s, and up to 920K IOPS. | HDFS is tuned for gigabyte-to-terabyte files, hundreds of nodes per cluster, and tens of millions of files in one instance. |
Plans
Google Cloud Filestore
- Filestore Zonal - custom performance on (per IOPS)$0.01 a month
- Filestore Regional - custom performance on (per IOPS)$0.03 a month
- Filestore Zonal - custom performance on (instance capacity)$0.12 a month
- Filestore Regional - custom performance on (instance capacity)$0.21 a month
- Filestore Zonal - custom performance off$0.25 a month
- Filestore Regional - custom performance off$0.45 a month
- Filestore Zonal - custom performance on (per instance)$20 a month
- Filestore Regional - custom performance on (per instance)$40 a month
Hadoop Distributed File System
- Apache Hadoop (open source)Free
In their own words
Google Cloud Filestore
Google Cloud Filestore has no write-up on Specifiction yet; its figures are above.
Hadoop Distributed File System
Hadoop Distributed File System (HDFS) is the primary distributed storage used by Hadoop applications. Its clusters use a NameNode to manage filesystem metadata and DataNodes to store data. HDFS is designed for fault tolerance on commodity hardware and high-throughput streaming access and batch processing, not low latency. It is tuned for files from gigabytes to terabytes, clusters with hundreds of nodes, and tens of millions of files in one instance. Administration includes shell-like commands, NameNode and DataNode web servers, fsck diagnostics, safemode, and a cluster balancer. Its permissions model shares much with POSIX and can optionally use POSIX ACLs; authentication supports simple operating-system identity or Kerberos. Configured encryption protects data at rest and in transit using encryption zones and the Hadoop Key Management Server. WebHDFS provides a REST API, and listed storage integrations include Amazon S3, Azure Blob Storage, Azure Data Lake Storage, and Tencent COS. HDFS is free to use under Apache License 2.0, with self-hosted cluster deployment.