Amazon Kinesis
Amazon Kinesis is the family name Amazon Web Services uses for its streaming-data services: Amazon Kinesis Data Streams, Amazon Kinesis Data Firehose (renamed Amazon Data Firehose), Amazon Kinesis Data Analytics (renamed Amazon Managed Service for Apache Flink), and Kinesis Video Streams. The exam guide lists plain “Amazon Kinesis,” but questions really test whether you can pick the right family member for the job.
Key points
- Amazon Kinesis Data Streams (KDS) — real-time ingestion into shards, read by custom consumer applications you write. Choose it when you need low-latency, ordered, replayable streaming data (default 24-hour retention, extendable up to 365 days) and are willing to build the consumer logic yourself.
- Amazon Kinesis Data Firehose, now Amazon Data Firehose — a fully managed, serverless delivery service with no consumer code and no replay: it batches, optionally transforms, and reliably delivers streaming data near-real-time to a fixed destination such as Amazon S3, Amazon Redshift, or Amazon OpenSearch Service. Choose it when you just need data to land somewhere, not to run custom processing logic against it.
- Amazon Kinesis Data Analytics, now Amazon Managed Service for Apache Flink — the processing/analytics layer: runs Apache Flink applications (including plain SQL) directly against a stream to transform, aggregate, or analyze it in real time. Choose it when the job is computing something from the stream, not just moving or storing it.
- Kinesis Video Streams — securely streams and stores video from camera-equipped devices for playback, computer-vision analysis (e.g. with Amazon Rekognition), and security monitoring; the only family member built for video rather than structured/text data records.
- Typical pipeline shape: producers → Kinesis Data Streams (ingest + shard + custom consumers) → Amazon Managed Service for Apache Flink (real-time transform/analyze) → Amazon Data Firehose (managed delivery) → a destination like S3 or OpenSearch Service. Not every pipeline uses all three — Data Firehose can also read directly from Kinesis Data Streams, Amazon MSK, or CloudWatch Logs/Events without any Flink stage.
- The exam-critical distinction is Data Streams vs. Data Firehose: Data Streams requires you to write and run consumer code and supports replay/multiple consumers; Data Firehose is destination-only, hands-off, and has no replay — pick Data Streams when the question mentions custom processing or multiple independent consumers, and Data Firehose when it mentions simply loading data into S3/Redshift/OpenSearch with minimal management.
- The Data Firehose and Managed Service for Apache Flink renames (from Kinesis Data Firehose and Kinesis Data Analytics) changed only the displayed name — APIs, the AWS CLI, IAM policies, and CloudWatch metrics were unaffected, and the older CLF-C02 study material still uses the legacy “Kinesis” names for both.
- Distinct from Amazon SQS and Amazon SNS: the Kinesis family is for continuous, ordered streaming ingestion and analytics at scale, not point-to-point message queuing (SQS) or pub/sub fan-out notifications (SNS).