Amazon FSx for Lustre
Amazon FSx for Lustre is an Amazon Web Services FSx solution that offers scalable, parallel, high-performance file storage built on the Lustre file system.
Key points
- Supports fast processing workloads such as custom electronic design automation (EDA) and high-performance computing (HPC); delivers sub-millisecond latencies, up to multiple TB/s of throughput, and up to millions of IOPS.
- Storage classes: SSD (consistent sub-millisecond latency across the full dataset, up to TB/s throughput), Intelligent-Tiering (fully elastic, cost-optimized, sub-millisecond latency on frequently-accessed data with an optional SSD read cache), and HDD (single-digit ms latency, up to tens of GB/s, with an optional SSD read cache sized to 20% of HDD capacity).
- Deployment types: Scratch (data not replicated and does not persist through a file server failure — for shorter-term, temporary data processing) and Persistent (data is replicated and failed file servers are replaced — for longer-term, throughput-focused workloads).
- Stores datasets in Amazon S3 as files instead of objects and automatically updates them with the latest data to run the workload.
- Works with existing Linux-based applications without any changes.
- Offers network access control using POSIX permissions or Amazon VPC security groups.
- Provides data-at-rest and in-transit encryption.
- Can be backed up using AWS Backup.
- Part of the wider Amazon FSx family alongside Amazon FSx for Windows File Server, Amazon FSx for NetApp ONTAP, and Amazon FSx for OpenZFS.
- Example workflow: AWS Step Functions drives AWS Batch jobs on Amazon EC2, reading reference and input data from — and writing output data to — S3 buckets through FSx.