Rate Limiting in Real Systems

Without rate limiting, a single client can starve everyone else. With it, you can shape traffic, protect backends, and enforce product tiers. The use cases below are organized by the specific failure they prevent, followed by the algorithm patterns that fit best.

Where it shows up in production

This flow is not a rule engine. It is a starting point that helps choose an algorithm based on the kind of failure you are trying to prevent.

Public API protection against abuse

Public APIs are exposed to bots, scraping, and misconfigured clients. The failure here is fairness: one client consumes shared capacity and makes the API unreliable for everyone else.

A good pattern is a burst-friendly limit for normal traffic plus a strict rolling cap on high-risk endpoints. Token Bucket lets legitimate bursts through, while Sliding Window Log enforces a hard ceiling on any rolling window.

References for real-world behavior and header patterns:

• GitHub API Rate Limiting: https://docs.github.com/en/rest/overview/resources-in-the-rest-api#rate-limiting
• Stripe Rate Limits: https://stripe.com/docs/rate-limits

Protecting critical backends

Databases, ML inference, and payment processors often fail under spikes rather than sustained load. The goal is a steady output rate. Leaky Bucket is the right fit because it smooths incoming traffic into a fixed drain rate.

Example: a query endpoint allows 5 requests/sec per tenant, but a queue depth of 50 absorbs bursts without overwhelming the database.

Reference:

• Envoy global rate limiting architecture: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_features/global_rate_limiting

Tiered pricing and quotas

SaaS plans live or die on predictable enforcement. The failure is inconsistent limits across tiers. You need clean separation of plans with runtime overrides.

Use per-tier configuration in YAML for defaults and MongoDB overrides for customer-specific adjustments. Fixed Counter works well for low tiers because it is fast; Token Bucket works well for standard tiers that need bursts; Sliding Window Log is best for enterprise SLAs where fairness matters.

Reference:

• AWS API Gateway usage plans: https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-api-usage-plans.html

Login, OTP, and password reset protection

Authentication endpoints are targets for brute force and credential stuffing. The failure is not just load; it is account compromise. Sliding Window Log is the safest choice because it enforces strict limits in any rolling period.

Use per-IP limits plus a second dimension like username, user ID, or API key where possible. Always return 429 with clear retry headers so legitimate clients can back off.

Reference:

• OWASP Authentication Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html

Internal service stability

Internal services spike during deploys, backfills, and batch jobs. The failure is often localized and temporary. Fixed Counter or Token Bucket with conservative defaults is usually enough, with overrides for critical dependencies.

Reference:

• Google SRE book on handling overload: https://sre.google/sre-book/handling-overload/

Choosing an algorithm quickly

flowchart TD
  A[Need bursts?] -->|Yes| B[Token Bucket]
  A -->|No| C[Need strict accuracy?]
  C -->|Yes| D[Sliding Window Log]
  C -->|No| E[Need constant output?]
  E -->|Yes| F[Leaky Bucket]
  E -->|No| G[Fixed Counter]

Practical guidance

Token Bucket is the safest default for external APIs. Leaky Bucket is for protecting slow or expensive backends. Fixed Counter is for internal services where simplicity and speed matter more than precision. Sliding Window Log is for security, billing, and SLA enforcement.