Google Cloud Managed Lustre vs OrangeFS

Google Cloud Managed Lustre is a fully managed parallel file system for artificial intelligence and high-performance computing workloads. Documented uses include HPC simulations, modeling and analysis, and machine-learning training and inference. It supports throughput up to 10 TBps and storage capacity up to 84,016,000 GiB (80.1 PiB). POSIX compliance is intended to preserve compatibility with existing applications, tools, and workflows. Instances can mount to Compute Engine virtual machines and Google Kubernetes Engine clusters. The service imports data from and exports data to Cloud Storage, including incremental transfers of new or changed files; only one transfer operation per instance can run at a time. Data is encrypted at rest and in transit to and within Google Cloud. Per-instance policies define maintenance windows and exclusion periods. Capacity is provisioned, and unused provisioned capacity is billed. Pricing is from $0.06/mo; hourly rates vary by month, and monthly charges use the hourly rate and hours in that month. Files can be up to 0.5 PiB, with 20 instances maximum per project. Lustre over TCP is supported, and managed backups are not available.

OrangeFS is a free, open-source parallel storage system for large workloads in high-performance computing, Big Data, streaming video, genomics, and bioinformatics. Its user-level servers and clients let applications access disk storage across multiple servers in parallel. Configurable file striping and distributed metadata, including directory metadata, support this layout. OrangeFS supports Linux, macOS, Windows, APIs, and self-hosted deployment. Linux integration uses a kernel client available in the mainline kernel since version 4.6, and the documentation recommends the upstream kernel module. Other interfaces include a native Windows parallel client, FUSE for Linux and Mac, a direct Linux application interface, Java JNI, Hadoop HCFS, WebDAV, and optimized MPI-IO; S3 is also listed among the file access protocols. Security options include capability-based security and default, key-based, and certificate-based modes. LDAP mapping can associate a client certificate subject with a user ID and primary group ID. CloudyCluster offers configurable deployments through Google Cloud and AWS marketplaces. The open-source software is free; commercial and research support is available by quote.