Skip to main content

Rate-Limiting and Request-Id Propagation

Juneau REST servers ship two small, opt-in primitives for operational hardening:

  • A token-bucket RestGuard — RateLimitGuard — that throttles inbound traffic per key (default: client IP) and answers with 429 Too Many Requests + Retry-After when the bucket empties.
  • A @RestStartCall-friendly filter — RequestIdFilter — that mints (or honors) an X-Request-Id per request, stashes it on the request attributes, and echoes it back on the response so downstream logs, traces, and clients can correlate.

Both classes are purely additive — they are wired in through the existing @Bean / @RestStartCall extension points and do not change the behavior of any pre-existing handler.

RateLimitGuard — token-bucket throttling​

RateLimitGuard is a RestGuard that runs before the handler. It maintains a token bucket per key, refilled at a configurable steady-state rate, and rejects requests when the bucket is empty.

Builder surface​

RateLimitGuard.create()
    .permitsPerSecond(50)              // or permitsPerMinute(N) / permitsPerHour(N)
    .burst(100)                        // maximum bucket capacity (default = permitsPerSecond)
    .keyBy(req -> req.getRemoteAddr()) // default: client IP from req.getRemoteAddr()
    .xForwardedForAware(true)          // default: false; see warning below
    .exemptPaths("/healthz","/readyz","/livez")  // default: those three probe paths
    .whenLimitExceeded((req, info) -> log.warn("throttled key={} resetIn={}s", info.key(), info.secondsUntilReset()))
    .storage(RateLimitGuard.Storage.inMemory(100_000))  // default: in-memory, 100k key cap
    .build();
Builder methodPurpose
permitsPerSecond(int) / permitsPerMinute(int) / permitsPerHour(int)Steady-state refill rate. Pick one.
burst(int)Maximum bucket capacity (cap on instantaneous burst). Defaults to the per-second rate.
keyBy(Function<RestRequest,String>)Key resolver. Default: client IP from req.getRemoteAddr(). Returning null collapses traffic onto a shared "_" bucket — useful for global throttling.
xForwardedForAware(boolean)When true, prefer the first hop in X-Forwarded-For over getRemoteAddr(). See warning below.
exemptPaths(String...)Paths that bypass the guard entirely. Defaults to /healthz, /readyz, /livez.
whenLimitExceeded(BiConsumer<RestRequest,RateLimitInfo>)Optional callback fired immediately before the 429 is thrown. Use for logging / metrics.
storage(Storage)SPI for the bucket store. Default: in-memory with a 100k key cap and LRU-style eviction.

Wiring it into a resource​

Use a @Bean RestGuardList method to install the guard — the same mechanism used for RoleBasedRestGuard:

@Rest(path="/orders")
public class OrderResource extends BasicRestServlet {

    @Bean
    public RestGuardList guards(BeanStore bs) {
        return RestGuardList.create(bs).append(
            RateLimitGuard.create()
                .permitsPerSecond(100)
                .burst(200)
                .build()
        ).build();
    }

    @RestGet("/{id}")
    public Order get(@Path long id) { ... }
}

Response on rejection​

When the bucket is empty the guard throws 429 Too Many Requests with:

HTTP/1.1 429 Too Many Requests
Retry-After: 3
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 3
Content-Type: text/plain

Retry-After is in seconds and is always at least 1 (i.e. Math.max(1, secondsUntilNextToken)).

Advisory X-RateLimit-* headers​

Every passing request gets three advisory response headers:

HeaderMeaning
X-RateLimit-LimitThe configured burst capacity for this guard.
X-RateLimit-RemainingTokens remaining in the bucket after this request.
X-RateLimit-ResetSeconds until the bucket is back at full capacity.

These follow the de-facto convention popularized by GitHub, Twitter, and Shopify. They are advisory — there is no IETF standard for them yet, and the exact semantics differ slightly between providers. Clients should treat them as hints, not contractual guarantees.

Token-bucket math​

The default in-memory storage tracks one Bucket per key. Each bucket holds a tokens value (double) and a lastNanos timestamp from System.nanoTime(). On every tryAcquire:

  1. Refill: tokens = min(capacity, tokens + elapsedSeconds × permitsPerSecond).
  2. If tokens >= 1 → decrement and allow.
  3. Otherwise reject and compute secondsUntilNextToken = (1 - tokens) / permitsPerSecond (in seconds, ceiling).

System.nanoTime() is monotonic but not wall-clock — buckets reset their windows relative to monotonic deltas, which is correct for throttling and immune to wall-clock drift.

Distributed-deploy footgun​

The default storage is per-JVM. A 4-pod deployment with permitsPerSecond(50) effectively allows 200 RPS, not 50, because each pod has its own bucket. Options:

  • Accept the per-pod multiplier (often fine for coarse protection).
  • Plug in a shared Storage backed by Redis, Memcached, or similar via .storage(Storage).
  • Push throttling up to an ingress (NGINX, Envoy, API Gateway) and use this guard only as a defense in depth.

The default in-memory storage caps at 100k keys and evicts the least-recently-touched bucket when the cap is hit. This protects against unbounded growth from random keys (e.g. spoofed IPs).

X-Forwarded-For spoofing warning​

When xForwardedForAware(true) is set, the guard uses the first comma-separated hop in X-Forwarded-For as the rate-limit key. This header is trivially spoofed by any client, so:

  • Only enable this flag when the server runs behind a trusted reverse proxy (ELB, NGINX, Cloudflare, Envoy) that strips and re-writes X-Forwarded-For from external traffic.
  • If the server is internet-facing without such a proxy, leave xForwardedForAware(false) and rely on getRemoteAddr().

A spoofed header lets an attacker forge a fresh key per request and bypass the bucket entirely.

Exempt paths​

exemptPaths(...) is intended for liveness / readiness / startup probes, which:

  • Run on a tight schedule (often every 5–10 seconds) from a known set of IPs.
  • Must always succeed to avoid spurious pod-restart loops.

The defaults — /healthz, /readyz, /livez — match the paths served by Juneau's built-in health-probe classes (HealthMixin / HealthServlet / HealthResource, org.apache.juneau.rest.server.health). Override with your own list (or pass .exemptPaths() for none) when paths differ.

The path match is exact: it compares against RestRequest.getPathInfo() first, then getServletPath(). Both are checked verbatim — there is no prefix/glob matching in v1. Wildcard support is on the future-work list.

RateLimitInfo callback​

The optional whenLimitExceeded(BiConsumer<RestRequest, RateLimitInfo>) callback fires immediately before the 429 is thrown. The RateLimitInfo record carries:

FieldMeaning
key()The resolved rate-limit key (IP, user id, custom).
limit()The configured burst capacity.
remaining()Tokens remaining (0 on rejection).
secondsUntilReset()Seconds until the bucket is back at full capacity.
allowed()false on rejection.

Use this hook for metrics (statsd, micrometer), structured logging, or pushing throttle events onto a queue for downstream analysis.

Inspecting bucket state at runtime​

RateLimitGuard.Storage exposes an optional read-side SPI for operator-facing tooling: default Map<String, BucketState> snapshot(). It returns a point-in-time view of every per-key bucket the storage backend currently tracks, so operators can answer not only "what is the rate-limit policy?" but also "which clients are currently throttled and at what fill level?".

The method is default returning Map.of(), which makes the addition purely additive and binary-compatible: external storage backends (Redis-backed, DynamoDB-backed, JDBC-backed, …) that don't enumerate buckets cheaply keep compiling unchanged and inherit the empty-map default. The bundled in-memory storage overrides snapshot() to walk its ConcurrentHashMap<String,Bucket> and emit one BucketState per entry.

BucketState record​

Each entry in the returned map is a BucketState record describing a single per-key bucket:

FieldMeaning
key()The per-request key the bucket is registered under (IP, user id, custom).
tokens()The bucket's current fractional fill level (continuously refilled at permitsPerSecond).
remaining()Integer tokens currently available — floor(tokens). Mirrors X-RateLimit-Remaining.
throttled()true when tokens < 1.0 (the next request from this key would be rejected).
lastRequest()Wall-clock Instant of the bucket's last activity.

Vocabulary is intentionally token-bucket-shaped: the underlying model is continuous-refill, so there is no discrete "window start" to report. tokens (the fill level) and lastRequest (the last activity) together fully describe the bucket's state.

The lastRequest field is wall-clock-accurate because each Bucket carries a lastWallMillis field updated alongside the existing monotonic System.nanoTime() value. The 8-byte-per-bucket overhead is trivial (~800 KB worst-case at the default 100k-key cap) and avoids the drift you'd get from mapping a nanoTime delta back to wall-clock under JVM suspension or container pause.

RateLimitGuard accessors​

For programmatic access — operator dashboards, custom admin endpoints, integration tests — RateLimitGuard exposes the configured state plus a snapshot convenience:

MethodReturns
getCapacity()The bucket capacity (burst).
getPermitsPerSecond()The steady-state refill rate.
isXForwardedForAware()Whether X-Forwarded-For-aware keying is enabled.
getExemptPaths()The set of paths that bypass the guard.
getStorage()The configured Storage backend.
snapshot()Convenience for getStorage().snapshot().
var guard = RateLimitGuard.create()
    .permitsPerSecond(100)
    .burst(200)
    .build();

// ... requests flow through ...

Map<String, BucketState> live = guard.snapshot();
for (var b : live.values())
    if (b.throttled())
        log.warn("client {} is throttled (last seen {})", b.key(), b.lastRequest());

Consistency contract​

Each per-bucket read inside InMemoryStorage.snapshot() is consistent (the Bucket accessors are synchronized on the bucket monitor), but the snapshot as a whole is not a global point-in-time consistent view — the iterator walks ConcurrentHashMap weakly-consistent entries, and concurrent tryAcquire(...) calls on other buckets may interleave with the walk. This is the right trade-off for operator visibility: contention-free under load, accurate per-key, and inconsistent only in the rare "did this bucket change while I was reading the next one?" sense.

The returned Map is immutable (Map.copyOf(...)); callers can safely pass it across threads.

/admin/ratelimit endpoint​

The Juneau ops/introspection mixin pack ships AdminMixin, whose /admin/ratelimit endpoint already exposes snapshot() over HTTP for every registered RateLimitGuard bean — alongside the enriched config block (class, limit, permitsPerSecond, xForwardedForAware, exemptPaths). See the AdminMixin section of the ops-pack page for the full JSON response shape and a worked example.

Storage backends that don't override snapshot() emit "snapshot": [] for their bean entry on /admin/ratelimit — the SPI's default-empty contract is honored end-to-end.

RequestIdFilter — X-Request-Id mint / honor / echo​

RequestIdFilter is a thin pre-call filter that:

  1. Honors a valid incoming X-Request-Id header.
  2. Mints a fresh one when the header is absent or fails validation.
  3. Stashes the resolved id on the request attribute keyed by RestServerConstants.REQUEST_ID.
  4. Echoes the id on the response as X-Request-Id.

Builder surface​

RequestIdFilter.create()
    .idSupplier(() -> UUID.randomUUID().toString())   // default
    .validator(Pattern.compile("^[A-Za-z0-9-_]{1,128}$").asPredicate())  // default
    .attributeKey(RestServerConstants.REQUEST_ID)     // default
    .build();
Builder methodPurpose
idSupplier(Supplier<String>)How fresh ids are minted. Default: UUID.randomUUID().toString().
validator(Predicate<String>)Whether to honor a given incoming id. Default: ^[A-Za-z0-9-_]{1,128}$.
attributeKey(String)Where to stash the id on the request. Default: RestServerConstants.REQUEST_ID.

Wiring it in​

The idiomatic shape is a single @RestStartCall method that delegates to the filter:

@Rest(path="/orders")
public class OrderResource extends BasicRestServlet {

    private static final RequestIdFilter REQUEST_ID = RequestIdFilter.create().build();

    @RestStartCall
    public void onStart(HttpServletRequest req, HttpServletResponse res) {
        REQUEST_ID.apply(req, res);
    }

    @RestGet("/{id}")
    public Order get(@Path long id, RestRequest req) {
        log.info("loading order id={} requestId={}", id, req.getAttribute(RestServerConstants.REQUEST_ID).asString().orElse("?"));
        ...
    }
}

Mint / honor / reject​

Incoming X-Request-IdBehavior
absentMint via idSupplier, stash, echo on response.
present + matches validatorHonor as-is, stash, echo on response.
present + fails validatorDiscard, mint a fresh one via idSupplier, stash, echo on response.

Rejecting malformed ids is the safe default: an unvalidated header lands in logs, metrics labels, and trace contexts. The default predicate (^[A-Za-z0-9-_]{1,128}$) accepts UUIDs, ULIDs, traceparent ids, and most opaque tokens while excluding whitespace, control characters, and pathologically long values.

Re-entrancy​

If the filter is invoked twice on the same request (e.g. via a forward), the second call sees the id already on the request attributes and short-circuits — the previously minted/honored id is preserved and re-echoed. The attribute is not overwritten by subsequent calls.

Interaction with mixins and paths​

Both primitives compose cleanly with the new @Rest(mixins=...) and @Rest(paths=...) features (see REST Server — Mixins and Multi-Mount Paths):

  • RateLimitGuard installs through a @Bean RestGuardList method. When the resource inherits a mixin that also contributes a guard list, the standard @Bean resolution rules apply — the resource's local method overrides the mixin's, so the mixin's guards are lost unless the resource composes them explicitly. If both lists need to apply, append both inside the local method.
  • RequestIdFilter installs through a @RestStartCall method. Multiple @RestStartCall methods on the same resource (including those inherited from mixins) all run — Juneau collects them via reflection and invokes them in declaration order. There is no implicit override, so mixin-contributed @RestStartCall hooks coexist naturally with the request-id filter.

Scope notes​

The primitives cover the per-pod, in-process layer. Out of scope for v1:

  • Distributed bucket sharing. The Storage SPI exists explicitly to support a future Redis-backed backend; v1 ships only the in-memory default.
  • Wildcard / regex exempt paths. Exact-match only.
  • Retry-After HTTP-date form. Always seconds.
  • Trace context propagation. RequestIdFilter only handles X-Request-Id. W3C traceparent / tracestate and Zipkin b3 headers are not currently handled.
  • Per-operation throttle overrides. All operations on a resource share the same guard configuration. Pre-handler dispatch on annotation is not currently supported.

See also​

Resources​