Amazon Kinesis Data Streams pricing

Shard hour: Shard is the base throughput unit of an Amazon Kinesis data stream. You specify the number of shards needed within your stream based on your throughput requirements. You’re charged for each shard at an hourly rate.

One shard provides an ingest capacity of 1 MB/second or 1,000 records/second.

When data consumers use enhanced fan-out, each shard provides up to 2 MB/second of data output for each consumer using it.

When data consumers do not use enhanced fan-out, each shard provides up to 2 MB/second of data output, regardless of the number of consumers processing data in parallel from a shard.

PUT Payload Unit (25 KB): A record is the data that your data producer adds to your Amazon Kinesis data stream. A PUT Payload Unit is counted in 25 KB payload “chunks” that comprise a record. For example, a 5 KB record contains one PUT Payload Unit, a 45 KB record contains two PUT Payload Units, and a 1 MB record contains 40 PUT Payload Units. PUT Payload Unit is charged a per-million PUT Payload Units rate.

Enhanced fan-out: Enhanced fan-out improves read parallelism by providing each data consumer with their own read throughput per shard (up to 2 MB/second). When consumers use enhanced fan-out, they incur hourly charges per consumer-shard hour and per GB of data retrieved.

Extended data retention: Kinesis Data Streams stores your data for 24 hours by default. Extended data retention gives you the option to extend the storage up to seven days. Once you start extended data retention, you’re charged for an additional rate on each shard hour incurred by your data stream.

Long-term data retention: Long-term data retention gives you the option to store your data stream up to 365 days. You’re charged extended data retention rates for the first seven days and then charged a GB-month rate of data stored beyond seven days up to your specified retention period.

Retrieval of long-term retention data: You’re charged a per-GB rate for retrieval of data stored beyond seven days if you are retrieving data using GetRecords API. You’re not charged for retrieval of data stored for the 24-hour default or for extended data retention up to seven days. You’re not charged for long-term data retrieval if enhanced fan-out consumer (SubscribeToShard API) is used.

Let’s say that your data producers put 100 records per second in aggregate, and each record is 35KB. In this case, the total data input rate is 3.4 MB/second (100 records/sec * 35KB/record divided by 1,024 = 3.4 MB/sec). For simplicity, we assume that the throughput and data size of each record are stable and constant throughout the day. Please note that we can dynamically adjust the throughput of the Amazon Kinesis stream at any time.

We first calculate the number of shards needed for your stream to achieve the required throughput. As one shard provides a capacity of 1 MB/sec data input and supports 1,000 records/sec, four shards provide a capacity of 4 MB/sec data input and support 4,000 records/sec. So a stream with four shards satisfies our required throughput of 3.4 MB/sec at 100 records/sec.

We then calculate our monthly Kinesis Data Streams costs using pricing in the US-East Region:

Shard Hour: One shard costs $0.015 per hour, or $0.36 per day ($0.015 * 24). Our stream has four shards so that it costs $1.44 per day ($0.36 * 4). For a month with 31 days, the monthly shard hour cost is $44.64 ($1.44 * 31).

PUT Payload Unit (25KB): As our record is 35KB, each record contains two PUT Payload Units. Your data producers put 100 records or 200 PUT Payload Units per second in aggregate. That is 267,840,000 records or 535,680,000 PUT Payload Units per month. As 1 million PUT Payload Units cost $0.014, your monthly PUT Payload Units cost is $7.499 ($0.014 * 535.68).

Optionally, we can choose to increase the data retention period of the stream from 24 hours to up to seven days. In this example, we assume that the data stream has extended data retention enabled throughout the whole month. Alternatively, you can choose extended data retention only in times when it’s critical to prevent data loss, which saves your cost on the extended retention.

Extended Data Retention (up to seven days): Extended data retention costs an extra $0.020 per shard hour. As our stream incurs 2,976 shard hours per month (4 shards * 24 hours/day * 31 days/month), your monthly extended data retention cost is $59.52 ($0.020 * 2,976)

Adding the shard hour and PUT Payload Unit costs together, the total Amazon Kinesis Data Streams costs are $1.68 per day, or $52.14 per month. For $1.68 per day, you have a fully managed streaming data infrastructure that enables us to continuously ingest 4 MB of data per second, or 337 GB of data per day, in a reliable and elastic manner. Additionally, you can increase your retention period from 24 hours to up to seven days for an extra $59.52 per month.

You can also optional retaining data longer than seven days and use enhanced fan-out. 

Long-term data retention (data older than seven days): Let’s assume that you want to retain your data stream for 30 days. You will incur extended data retention shard-hour pricing from 24 hours up to seven days (see above for calculations) and long-term data retention pricing beyond seven days to 30 days (a 23-day period). For long-term retention pricing of 23 days, it costs an additional $0.023/GB-month for storage and $0.021 per GB for data retrievals if you use the default consumer (GetRecords API). Based on a data input rate of 3.4 MB/second, you’ll accumulate 3.4 (input rate) * 1,024 * 1,024 * 3,600 = 12,902,400,000 bytes in an hour.

At the end of the month, you’ll have the following usage:

3,565,158 * 23 days * 24 hours/day = 7,122,124,800,000 byte-hours

Let’s convert that to GB-month:
7,122,124,800,000 byte-hours / 1,073,741,824 bytes per GB = 6,633 GB per month

So the total storage costs for 23-day retention = 6,633* $0.023 = $152.56 per month.

Long-term data retrieval (data older than seven days): If you retrieve all long-term data using a single consumer that uses GetRecords API to fetch records, you’ll incur retrieval charges only for the data stored beyond the first seven days. Since you’re retrieving 6,623 GB of data, you’ll be charged 6,623 * 0.021 = $139.08 at the end of the month. Long-term data retrieved using enhanced fan-out consumers incur enhanced fan-out data retrieval rates as described below.

Enhanced fan-out consumer-shard hours: Enhanced fan-out costs an extra $0.015 per shard, and this cost is additive per data consumer. As your stream incurs 2,976 shard hours per month (4 shards * 24 hours/day * 31 days/month), each consumer using enhanced fan-out throughout this period will incur a cost of $44.64 (2,976 * $0.015). Two consumers over the same period would incur a cost of $89.28.

Enhanced fan-out data retrievals: Data retrieved from a stream by a consumer using enhanced fan- out incurs a charge of $0.013 per GB. If, over a month, a two-shard stream ingested 50,000 MB of data, and two data consumers used enhanced fan-out and retrieved all of this data during the month, the total data retrieved would be 100,000 MB (50,000 MB * 2 consumers) or 97.6 GB. Since each GB costs $0.013, the total enhanced fan-out cost is $1.26 (97.6 * $0.013).