Amazon S3
Amazon Simple Storage Service (Amazon S3) is a Amazon Web Services key-based object storage service that stores objects across multiple Availability Zones in region-specific buckets, commonly used for static website hosting and for data that is written once and accessed many times, with fast transfer via Amazon CloudFront and archival tiers such as Amazon S3 Glacier
AI Practitioner focus
- S3 is the common durable store for raw/curated datasets, labels, model artifacts, evaluation sets, RAG documents, logs, and generated outputs. Apply versioning, lifecycle/retention, encryption, least privilege, and provenance.
- S3 Vectors is a newer managed capability for storing/querying vectors at scale; the current exam’s named vector examples still emphasize OpenSearch, Aurora/RDS PostgreSQL, and Neptune.
Key points
- Global service with region-specific buckets.
- Provides 99.999999999% (11 9’s) of content durability.
- Provides strong read-after-write consistency for all GET, PUT, and LIST requests, including overwrite PUTs and DELETEs, in every AWS Region since December 2020 (older CLF-C02 material still describes S3 as only eventually consistent for overwrite PUTs and deletes).
- Objects (files) are stored in a region-specific container known as a Bucket, and can range from 0 bytes to 5 TB.
- Object storage — S3 stores whole objects with metadata, not disk blocks or a POSIX file system; distinct from Amazon Elastic Block Store (EBS) (block storage for a single EC2 instance) and Amazon Elastic File System (EFS) (shared file storage mounted by multiple instances).
- Multipart upload — uploads objects in parts; suitable for objects 100 MB or larger.
- Versioning — retains multiple versions of objects to recover from accidental deletion or overwrite; must be enabled at both source and destination.
- Amazon S3 Transfer Acceleration — fast and secure transfer of objects over long distances with minimum latency, using Amazon CloudFront’s Edge Locations.
- Security models:
- User-based security — IAM policies.
- Resource-based security — bucket policies, bucket access control lists (ACL), object ACLs. S3 Object Ownership is now set to Bucket owner enforced by default on new buckets, which disables ACLs entirely; access is managed exclusively through IAM and bucket policies unless ACLs are deliberately re-enabled.
- Cross-account access to objects and buckets by assuming a role with specified privileges.
- An EC2 instance should access S3 by assuming an attached IAM role (instance profile), not by embedding or storing IAM access keys on the instance.
- Storage classes:
- S3 Standard — frequent data access.
- S3 Express One Zone — high-performance, single-Availability-Zone storage class for latency-sensitive applications, purpose-built for single-digit-millisecond access; stores data in a directory bucket rather than a general purpose bucket.
- S3 Intelligent-Tiering — automatically transfers data to other cost-effective access tiers based on changing access patterns.
- S3 Standard-IA — immediate but infrequent data access.
- S3 One Zone-IA — infrequent data access, stored in a single Availability Zone only; cheaper than Standard-IA but not resilient to loss of that AZ, and designed for lower availability (99.5%) than the multi-AZ storage classes (99.9%+) because it has no redundancy across Availability Zones.
- S3 Glacier Instant Retrieval — archive data needing millisecond retrieval, accessed roughly once a quarter.
- S3 Glacier Flexible Retrieval (formerly S3 Glacier) — long-term archive data, retrieval in minutes to hours.
- S3 Glacier Deep Archive — lowest-cost storage class, for long-term retention.
- Minimum storage duration charges — S3 Standard-IA and S3 One Zone-IA require a 30-day minimum; S3 Glacier Instant Retrieval and S3 Glacier Flexible Retrieval require a 90-day minimum; S3 Glacier Deep Archive requires a 180-day minimum. Deleting, overwriting, or transitioning an object out of these classes before its minimum has elapsed still incurs the full minimum-duration storage charge (a pro-rated early-deletion fee).
- Minimum billable object size — S3 Standard-IA, S3 One Zone-IA, and S3 Glacier Instant Retrieval bill every object as if it were at least 128 KB; smaller objects are still charged the 128 KB rate, so archiving very small objects into these classes can cost more than leaving them in S3 Standard.
- Replication types:
- Cross-Region Replication — replicates objects to a different AWS Region.
- Same Region Replication — replicates objects within the same AWS Region.