Rate Limiting: Defending Your APIs Without Compromising User Experience

APIs essentially act as gateways to your infrastructure. Without rate limits, these gateways remain wide open, potentially exposing your system to abuse, overload, and misuse. Rate limiting is not only a protective measure but also a tool to shape client behavior, distribute capacity fairly, and maintain system health, all without sacrificing usability.

When implemented effectively, rate limiting is unobtrusive. However, poor implementation can lead to considerable frustration for developers. This article provides an in-depth exploration of rate limiting, from its theoretical foundations to practical implementation in REST and GraphQL APIs, drawing on real-world strategies from Stripe, GitHub, Shopify, and others.

The Rationale Behind Rate Limiting

Your API is a communal resource that involves memory, CPU, database access, and bandwidth. A handful of misbehaving clients or a sudden surge in traffic can put your entire system at risk.

Rate limiting serves as a safeguard against potential threats such as:

• DDoS attacks
• Overenthusiastic testing or infinite retry loops
• Background jobs running out of control
• Mobile apps polling excessively
• Bots repetitively scraping your catalog

Therefore, rate limiting is both a safety mechanism and governance tool. The challenge lies in enforcing it transparently, fairly, and predictably.

Strategies and Algorithms for Rate Limiting

There are diverse strategies and algorithms for implementing rate limiting, each with its strengths and potential drawbacks:

1. Fixed Window

This strategy allows a specific number of requests per time unit (e.g., 100 requests per 60 seconds) and resets at fixed intervals.

Allow 100 requests per 60 seconds.
Resets every minute.

Strengths:

• Simplicity: It is straightforward to implement and understand.

Drawbacks:

• Burst Traffic: Traffic spikes at the reset boundary can overwhelm the system.

2. Sliding Window

Unlike the fixed window, this strategy tracks requests over a rolling window of time, for example, the last 60 seconds.

Strengths:

• Smooth Distribution: This approach reduces the risk of system overload due to burst traffic.
• Greater Fairness: It ensures a more equitable distribution of requests over time.

Drawbacks:

• Computational Cost: It requires more computational resources to track and maintain.

3. Token Bucket

This strategy assigns each user a bucket of tokens. Each request consumes a token, and tokens are replenished over time.

Strengths:

• Burst Allowance: It can accommodate short bursts of traffic.
• High Flexibility: It can be adjusted based on specific needs or scenarios.
• Real-world Adoption: Companies like Stripe and Cloudflare use this strategy.

4. Leaky Bucket

Similar to the token bucket strategy, the leaky bucket processes requests at a consistent rate, ensuring smooth output over time.

Strengths:

• Rate Control: It maintains a steady outgoing rate, which can be crucial for systems processing queue-like flows.

Enforcing Rate Limits

Rate limits can be enforced based on different parameters such as:

• IP address
• API key or authentication token
• User account ID
• Resource type (e.g., write-heavy vs. read-heavy operations)
• Endpoint

Well-designed APIs differentiate between read and write limits: read operations (GET) typically have higher limits due to their lower impact on system resources, while write operations (POST/PUT/DELETE) have lower limits due to their potential to significantly affect system state and performance.

Utilizing HTTP Headers for Client Communication

Communicating limit information to clients is essential in building trust and enabling smart retries. This information can be conveyed through HTTP headers in the server response:

HTTP/1.1 429 Too Many Requests
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 23
X-RateLimit-Reset: 1691334020
Retry-After: 30

Here's what each of these headers represents:

• X-RateLimit-Limit: The maximum number of requests allowed in a given time window.
• X-RateLimit-Remaining: The number of requests still available in the current time window.
• X-RateLimit-Reset: The Unix timestamp when the rate limit will reset.
• Retry-After: The recommended wait time (in seconds or as an HTTP date) before the client should attempt another request.

It's crucial to return a 429 Too Many Requests HTTP status code for rate limit violations to inform clients explicitly about the reason for the request denial.

Real-World Rate Limiting Systems

Let's look at how some big tech companies implement rate limiting:

Stripe

• Stripe applies different rate limits for public and secret keys.
• It responds with a 429 status code for rate limit violations and includes detailed headers.
• Rate limits vary based on the endpoint class (write, read, webhook).

GitHub

• For REST APIs, GitHub allows 5000 requests per hour per user or token.
• For GraphQL APIs, GitHub uses a point-based limit where the cost depends on the shape of the query.
• GitHub includes rate limit information in both HTTP headers and the rateLimit field in GraphQL.

Shopify

• Shopify uses a bucket-based rate limit per app.
• Its Admin API permits bursts of requests before throttling.
• Shopify includes the Retry-After header in its responses to guide clients on when to retry.

Frontend Considerations

When dealing with rate limits, frontend developers should:

• Use exponential backoff with jitter to handle retries smartly, reducing the likelihood of synchronized retry storms.
• Detect and display rate limit messages in the user interface to keep users informed.
• Warn users if they are approaching limits, for instance, by displaying an API quota meter.
• In B2B applications, make rate limits visible via developer dashboards.

Best Practices

Rate limiting should be designed with the following best practices in mind:

• Normalize limits based on customer plans (e.g., free vs. enterprise) to ensure fairness and sustainability.
• Provide soft warnings before enforcing hard blocks to give clients a chance to adjust their behavior.
• Log all violations with context (IP, token, endpoint) for audit purposes and to support future tuning of rate limits.
• If your system is at risk of DDoS attacks, consider applying rate limits before the authentication layer.
• Always notify clients when you drop their requests due to rate limits. Silent failures can lead to confusion and frustration.

Anti-Patterns

Avoid these common mistakes when implementing rate limiting:

• Using HTTP status codes 500 or 503 instead of 429 for rate limit violations.
• Not including rate limit headers, leading to mystery failures for clients.
• Applying rate limits without critical consideration (e.g., on password resets or help center requests).
• Using the same rate limit across all endpoints indiscriminately.
• Not differentiating between different types of clients (e.g., bots vs. browsers vs. mobile clients).

Conclusion: Design Limits with Empathy

Rate limiting is not just a traffic control mechanism but a crucial user experience touchpoint and system guardrail. Intelligently designed rate limits can encourage good client behavior, protect shared infrastructure, educate developers, and ensure predictable scalability.

As you design rate limits, approach it like you would when creating APIs: with transparency, intention, and empathy. Limits are inevitable in shared systems, but frustration is not.