Architecture and Concepts

Why this has to be distributed

Rate limits break the moment you add a second instance. If each server keeps its own counter, a client can spread requests across instances and bypass the limit. The fix is shared state: every instance must evaluate limits against the same data.

This system does that with Redis. All algorithms run as Lua scripts inside Redis, so every request sees a single, consistent view of the limiter state.

The request flow in practice

Every request goes through a filter that calculates identity, resolves configuration, and delegates to Redis. That flow is the spine of the service.

HTTP Request
  → RateFilter (extract IP + endpoint)
  → RateLimitManager (resolve config)
  → Redis (Lua script runs algorithm)
  → Response with rate-limit headers

The filter runs once per request. The manager decides which algorithm and parameters apply. Redis executes the algorithm atomically and returns the decision along with counters used to set response headers.

Configuration resolution order

The system always picks the most specific configuration available, then falls back.

1) MongoDB override (IP + endpoint + tier)
2) YAML endpoint match
3) YAML fallback

MongoDB overrides let you adjust limits at runtime for specific clients without redeploying. YAML provides safe defaults for each endpoint. The fallback ensures every request still has a limit even if no endpoint matches.

Redis state and algorithm execution

Each algorithm uses a different data structure, but all of them live under the same key pattern: rate-limit:{ip}:{endpoint}.

Token Bucket and Leaky Bucket store a small hash with the current token or queue state and the last update timestamp. Fixed Counter stores a single string counter with a TTL. Sliding Window Log stores timestamps in a sorted set and trims entries outside the window.

Lua scripts are critical because they keep read-modify-write operations atomic. Without Lua, concurrent requests would race and the limits would drift.

MongoDB as a persistent override store

MongoDB is only used for configuration overrides, not for request decisions. The document includes the IP, endpoint, user tier, HTTP method, algorithm, and parameters. Once stored, it becomes the highest-priority configuration for that matching request.

This keeps runtime changes separate from baseline configuration and avoids frequent redeploys when you need to tweak a limit for a single tenant.

Headers and client behavior

When a request is rate-limited, the response includes three headers:

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 87
X-RateLimit-ResetOn: 1713091800000

Clients should back off until X-RateLimit-ResetOn when X-RateLimit-Remaining reaches 0. When the limit is exceeded, the response status is 429 Too Many Requests.

Failure modes and tradeoffs

The service depends on Redis for every decision. If Redis is down or unreachable, the rate limiter returns 503 Service Unavailable. This is intentional: it avoids silently allowing traffic when the limiter cannot evaluate a request. If your risk model requires fail-open behavior, that should be an explicit design choice at the gateway or in a custom fallback layer.

Sliding Window Log provides the most accurate enforcement, but it uses more memory because it stores one entry per request in the window. Token Bucket and Fixed Counter are cheaper when you need high throughput or low memory usage.

Where this fits in your system

If you are wiring the service into an existing Spring Boot app, the integration guide is the next practical step. If you are selecting algorithms or setting limits, the deep dives explain tradeoffs and concrete parameter choices.