How do I select the best Amazon API Gateway Cache capacity to avoid hitting a rate limit?

Short description

Amazon API Gateway throttles requests to your API to prevent it from being overwhelmed by too many requests. Turn on API caching to reduce the number of calls made to your endpoint.

There are multiple API Gateway Cache sizes available. To select the appropriate cache size, run a load test on your API and then review the Amazon CloudWatch metrics.

Resolution

Turn on API Gateway caching

Turn on Amazon API Gateway caching for your API stage. The cache capacity depends on the size of your responses and workload.

Note: Cache capacity affects the CPU, memory, and network bandwidth of the cache instance. As a result, cache capacity can affect the performance of your cache.

After creating your cache, run a load test to determine if the cache size is high enough to prevent throttling.

Run a load test

Run a load test on your API. You can use AWS Distributed Load Testing to simulate the load test.

Run the load test for at least 10 minutes and mirror your production traffic. When the load test is running, monitor related CloudWatch metrics using the steps in the following section.

Monitor API metrics in CloudWatch

1. Open the CloudWatch console.
2. In the navigation pane, select Metrics.
3. Choose the ApiGateway metric.
4. Monitor Latency, 4XXError, 5XXError, CacheHitCount, and CacheMissCount metrics for the API that you are load testing against.
5. If you see an increase in Latency, 4XXError, 5XXError or CacheMissCount with a CacheHitCount decrease, then resize your API Gateway cache to a larger capacity.
6. If you see an increase in CacheHitCount and no corresponding increase in CacheMissCount, then resize your API Gateway cache to a smaller capacity.
7. After any changes to your cache's capacity, run the load test again until there are no sudden increases or decreases.

Related information

Amazon API Gateway pricing

Turn on API caching to enhance responsiveness

Amazon API Gateway dimensions and metrics