Skip to main content

NG Client Security

The NG RestClient protects outbound calls against server-side request forgery (SSRF) and credential leaks across redirects. This page describes what is enforced, where, and how to opt out. The same URL policy class (org.apache.juneau.http.remote.RemoteUrlPolicy, in juneau-rest-common) is shared with the classic client.

Which calls are covered?

The SSRF guard (URL deny-list, pin-on-connect, per-hop redirect re-validation) applies to policy-covered calls: @Remote / @RemoteOp proxies and dynamic @Url arguments, or a request where you called remoteUrlPolicy(true, ...). An ordinary client.get("https://...") is treated as trusted application code and is not run through the deny-list.

URL policy​

RemoteUrlPolicy.requireAllowedUrl(url, allowPrivate) rejects a URL unless:

  • the scheme is http or https and the host is non-empty;
  • the host is not on the deny-list: localhost, localhost.localdomain, metadata.google.internal (and its subdomains), or an IP literal that is loopback, any-local, link-local, site-local (RFC 1918), IPv6 unique-local (fc00::/7), IPv4 0.0.0.0/8 ("this network"), IPv4 100.64.0.0/10 (carrier-grade NAT shared space, which stays denied), IPv4 special-purpose ranges 192.0.0.0/24 (IETF protocol assignments), 198.18.0.0/15 (benchmarking), 224.0.0.0/4 (multicast) and 240.0.0.0/4 (reserved, including 255.255.255.255), IPv6 multicast (ff00::/8), IPv6 Teredo (2001::/32) or IPv6 local-use NAT64 (64:ff9b:1::/48). The last two are denied outright, because the embedded IPv4 address is obfuscated or its embedding length is operator-defined. IPv6 addresses that embed an IPv4 address (IPv4-mapped, IPv4-compatible, IPv4-translated ::ffff:0:a.b.c.d, NAT64 64:ff9b::/96, and 6to4 2002::/16) are unwrapped and classified by the embedded IPv4 address. IPv6 zone ids ([fe80::1%25en0]) are ignored for classification, and an unparseable IPv6 literal is denied.

To re-permit a lab or appliance network (for example 198.18.0.0/15), set the JVM-wide system property RestClient.allowAddressRanges to a comma- or whitespace-separated CIDR list, such as -DRestClient.allowAddressRanges=198.18.0.0/15. It defaults to nothing allowed, and it is narrow by construction:

  • every entry must lie entirely inside one of the special-purpose ranges 192.0.0.0/24, 198.18.0.0/15, 224.0.0.0/4, 240.0.0.0/4 or ff00::/8; any other entry (including 0.0.0.0/0, ::/0, a private range or the CGNAT range) is rejected with an IllegalArgumentException;
  • it only re-permits addresses denied solely because they fall in one of those ranges. Loopback, link-local and metadata, private, CGNAT, "this network", unique-local, Teredo and local-use NAT64 addresses stay denied, as do IPv6 forms (such as 6to4) that embed such an address;
  • an IPv4 entry matches the IPv4 address or the IPv4 address embedded in an IPv6 form; an IPv6 entry matches only a native ff00::/8 address.
  • the allow applies to both the string check above and the resolved-address (DNS-pinned) check below. Entries are parsed strictly without any DNS lookup: dotted-quad IPv4 with octets 0-255 and no leading zeros, prefix lengths without leading zeros, and empty list items (such as a trailing comma) are ignored.

The same allow-list can be set for a single client with RestClient.Builder.allowAddressRanges(String...) (for example .allowAddressRanges("198.18.0.0/15")) on both the classic and next-generation clients and every transport. It has exactly the semantics and validation described above, and an invalid entry makes build() throw an IllegalArgumentException. The client's ranges are combined with the system property as a union, and a client without the setting is unaffected by another client's setting in the same JVM.

Relative values pass through, and the client's rootUrl is not re-checked. Bare decimal, octal, or hex IP spellings are not recognized as literals by the string check; they are caught by DNS pinning below.

Set allowPrivateUrls(true) on the builder, allowPrivateUrls=true on @Remote / @RemoteOp, or the system property RestClient.allowPrivateUrls=true to permit private addresses (for example, when calling internal services). The effective setting is the logical OR of the three.

DNS pinning​

A host-name check alone is defeated by DNS rebinding and by names that resolve to private addresses. When the guard is active, each transport resolves the host itself, picks the first address that is not denied (RemoteUrlPolicy.selectAllowedAddress), and connects to exactly that address. If every candidate is denied, the call fails with TransportException.

TransportPinning mechanism
JDK JavaHttpTransport (default)Rewrites the request URI to the pinned literal address and keeps the original Host header. It fails closed if pinning is not possible.
Apache HttpClient 4.5PolicyPinningDnsResolver on a dedicated policy-only client with its own connection pool.
Apache HttpClient 5PolicyPinningDnsResolver on a dedicated policy-only client with its own connection pool.
OkHttpPolicyPinningDns on a dedicated policy-only derived client (shares the connection pool and dispatcher with the normal client, but OkHttp only reuses a connection for a request with the same Dns).
JettyPolicyPinningSocketAddressResolver

Pinning is scoped to policy-covered requests, so normal traffic through the same client is unaffected, and a policy-covered request never reuses a connection that a normal request opened without the pin. The Apache HttpClient 4.5 and 5 transports send policy-covered requests through a second client of their own, whose separate connection pool resolves every connection through the pin. Jetty tags policy-covered requests, which gives them a separate connection pool of their own, and its resolver pins every connection opened for that pool. OkHttp's policy-only client shares the normal client's connection pool, but OkHttp reuses a connection only for a request with the same Dns, so the two never share a connection.

Transports that cannot enforce the policy​

If the guard is active and the transport reports supportsUrlPolicy() == false, RestRequest.run() refuses to send the request and throws TransportException ("Refusing to send a policy-covered @Remote request..."). All bundled transports and the mock transports support the policy. The Apache HttpClient 4.5, Apache HttpClient 5, and Jetty transports support it only with the client they create themselves; built with a caller-supplied client, they report supportsUrlPolicy() == false. The OkHttp transport reports false for a client with an explicit proxy, and refuses a policy-covered request that the client's proxy selector routes through a proxy, because the proxy resolves the target host itself. A custom HttpTransport must override supportsUrlPolicy() once it implements pinning and honors TransportRequest redirect handling.

Classic RestClient​

The classic client (org.apache.juneau.rest.client.classic.RestClient) guards @Remote calls with the same URL policy. The connection manager it builds keeps two separate pools: policy-covered calls lease from a pinned pool whose connections are all opened through PolicyPinningDnsResolver, and every other call leases from the ordinary pool. So a policy-covered call, or any of its redirect hops, never reuses a connection that an ordinary call opened without the pin. This holds for both the default and the pooled() connection manager. The pinned pool uses the ordinary pool's settings (pool limits, socket and connection configuration), so a createConnectionManager() override that configures the manager returned by super configures policy-covered calls too.

The connection manager the classic client builds applies the builder's TLS and pool settings to both pools: sslContext(...) (for example one with a custom trust store), sslHostnameVerifier(...), sslSocketFactory(...), maxConnTotal(...) and maxConnPerRoute(...). Apache HttpClientBuilder ignores its own copies of these once a connection manager is set. maxConnTotal(...) or maxConnPerRoute(...) makes the connection manager a pooling one, as pooled() does. Each build() creates a new connection manager, so a builder reused after build() gives every client its own, and closing one client does not break the next. A connection manager you supply with connectionManager(...) is used as is and keeps its own TLS and pool settings.

A custom sslSocketFactory(...), or a socket factory set with setSSLSocketFactory(...) on the HttpClientBuilder you pass to httpClientBuilder(...), is also used for policy-covered calls. The pin hands it the address it has already checked, and the factory must connect to exactly that address. A factory that ignores that address and connects by host name, resolving the name again, bypasses the pin.

The other settings Apache HttpClientBuilder applies only to a connection manager it builds itself reach both pools too: defaultSocketConfig(...), defaultConnectionConfig(...), connectionTimeToLive(...) (which also makes the connection manager a pooling one) and publicSuffixMatcher(...) (used by the default hostname verifier). So do TLS and pool settings made directly on the HttpClientBuilder you pass to httpClientBuilder(...), and the system properties useSystemProperties() reads: the JVM's default SSL socket factory (the javax.net.ssl.* trust and key stores), https.protocols, https.cipherSuites and, unless http.keepAlive is false, the http.maxConnections limit. A DNS resolver set on the HttpClientBuilder is used for ordinary calls only; policy-covered calls always resolve through the pin.

If httpClientBuilder(...) replaces the HttpClientBuilder after you have configured the client builder, these connection-manager settings made with the client builder's own setters carry over, and win over the same setting on the replacement: sslContext(...), sslSocketFactory(...), sslHostnameVerifier(...), publicSuffixMatcher(...) (for the default hostname verifier only; its use for cookies is lost with the replaced builder), maxConnTotal(...), maxConnPerRoute(...), pooled(), connectionTimeToLive(...), defaultSocketConfig(...), defaultConnectionConfig(...) and the TLS and http.maxConnections system properties of useSystemProperties(). Every other setting made before the replacement (interceptors, proxy, the rest of useSystemProperties(), and so on) was made on the replaced HttpClientBuilder and is lost. For these connection-manager settings, a value set with the client builder's own setter always wins over the same setting on the HttpClientBuilder, whichever was set last: unlike Apache's last-write-wins, a later httpClientBuilder().setDefaultSocketConfig(...) does not override an earlier defaultSocketConfig(...). As in Apache HttpClientBuilder, a socket factory wins over an SSL context wherever each was set, so a setSSLSocketFactory(...) on the replacement wins over an earlier sslContext(...).

close() shuts down the connection manager the client built for itself, even with connectionManagerShared(true): nothing else can reach it. "Shared" keeps open a connection manager you supply, and any connection manager an overridden createConnectionManager() returns, even one that calls super, because the override may hand the same instance to several clients.

The classic client refuses a policy-covered @Remote call ("Refusing to send a policy-covered @Remote request...") when it cannot guarantee the pin. It cannot pin a connection manager it did not build, or a call that goes out through an HttpClient other than the one it wired to its pinning connection manager. The guard travels on the request's HttpContext (each redirect hop gets a context marked for the guard), so it holds on whichever thread runs the request. A replaced call path may still send a copy of the request, or send it with another context, so the refusals below are what keep such a path from going out unpinned. That is the case when:

  • you supply a connection manager with connectionManager(...);
  • you override createConnectionManager() to return a different one;
  • you supply an HttpClient with httpClient(...);
  • you override createHttpClient() to return a different client, or a wrapper around the one super builds;
  • your createHttpClient() override, after calling super.createHttpClient(), sets an HttpProcessor on the HttpClientBuilder, adds a request interceptor with addInterceptorLast(...), or replaces its redirect strategy or request executor, and builds again. Make such changes before calling super. The builder is checked as it is once the client is built, so one of these changes made after the build is refused too; a change that the override undoes before it returns is not detected;
  • you pass httpClientBuilder(...) (or return from createHttpClientBuilder()) an HttpClientBuilder subclass that overrides build(), createMainExec(...), decorateMainExec(...) or decorateProtocolExec(...), or inherits such an override. Any of these can build the client around another connection manager or request executor, even when build() returns the client super built. This includes wrapping or decorating builders such as httpclient-cache's CachingHttpClientBuilder (it overrides decorateMainExec(...)). The check fails closed and has no escape hatch: under the guard, allowPrivateUrls(true) is the only way to use such a builder;
  • you set an HttpProcessor with httpProcessor(...). It replaces all of the client's request interceptors, among them the guard's own and the redirect credential guard. Add interceptors with addInterceptorFirst(...) or addInterceptorLast(...) on the HttpClientBuilder instead;
  • you set a custom RestCallHandler with callHandler(...), or a BasicRestCallHandler bound to a different client;
  • you use a RestClient subclass that overrides run(HttpHost,HttpRequest,HttpContext), execute(HttpHost,HttpRequest,HttpContext) or execute(HttpUriRequest,HttpContext);
  • createRequest(...) returns a RestRequest subclass that overrides run();
  • an override of createRequest(...) or request(...) returns a RestRequest bound to a different client. That request would go out through the other client's HttpClient.

A policy-covered request whose HttpContext lacks the guard marker, for example one a custom call path sent with a fresh context, is refused rather than sent unguarded. So is one that would open or reuse a connection of the ordinary pool. That can only happen when the guard's request interceptor did not run, for example a request with the guard's context sent directly through execute(...) on a client with an HttpProcessor. A refused request leaves nothing behind: the next ordinary request on the same thread is sent normally.

A policy-covered call on a proxy route is refused too, because the pin would check the proxy's address instead of the target's. This covers proxy(...), a routePlanner(...) that picks a proxy, and the proxy system properties read by useSystemProperties(). The refusal is reported as a RestCallException, with the connection manager's IllegalStateException as its cause.

Ordinary calls are unaffected. Set allowPrivateUrls(true) if the target is an intentional local or intranet service. MockRestClient supplies its own connection manager, so its @Remote calls need allowPrivateUrls(true) too.

Redirect re-validation​

For policy-covered calls, Juneau runs the redirect loop itself (PolicyEnforcedRedirects) and turns off the transport's own auto-redirect. For every Location hop:

  1. the target URL is re-checked against the URL policy;
  2. the hop is sent over a pinned connection;
  3. on the classic client, a hop to another origin (or an https-to-http downgrade) drops the caller-set credential headers: the RedirectSecurity set described below, and Proxy-Authorization in every case (a guarded call never goes through a proxy). They stay dropped for the rest of the chain, even if a later hop returns to the first origin. Every hop is compared with the first hop's origin, so the same headers are also dropped when the client's own interceptors or HttpClientBuilder default headers add them to a hop on another origin. With a context(...) setting, which all hops share, the target's authentication state is reset on a hop to another origin, as Apache HttpClient does on a change of host, so credentials that answered the first host's challenge are not sent unasked to the next.

Handled status codes are 301, 302, 303, 307, and 308. 301/302/303 rewrite the method to GET and drop the body (unless it was already GET/HEAD); 307/308 preserve the method and body. The call fails if:

  • the redirect has no Location;
  • more than 5 hops are followed (RemoteUrlPolicy.MAX_REDIRECT_HOPS);
  • the Location is invalid, or names a host that java.net.URI cannot parse (such as my_service), so the origins cannot be compared;
  • a 307/308 would have to replay a non-repeatable body (fail closed);
  • the JDK HttpClient was configured to follow redirects itself (Redirect.NEVER, the default, is required).

Each of these failures, like a refused pin, is a TransportException.

Credential stripping on redirects​

When a redirect changes origin, credentials must not follow it. RedirectSecurity defines the rule:

  • same origin compares scheme, host (case-insensitive), and port (80/443 normalized);
  • credentials are stripped when the origin differs or the redirect is an https-to-http downgrade;
  • the stripped set is RedactedHeaders.DEFAULT: Authorization, Cookie, Set-Cookie, Proxy-Authorization, and X-API-Key. The OkHttp transport keeps Proxy-Authorization only while the redirected request is still plain HTTP through the same HTTP proxy as the original request, and the original request was also plain HTTP (the header authenticates you to the proxy, so stripping it there makes the proxy answer 407 again). With no proxy, a different proxy, or an https target (reached through a CONNECT tunnel), it is stripped like the rest. The Apache HttpClient 5 transport, on a client you supply, likewise keeps it on a proxied route that is not tunnelled (see below);
  • if the two origins cannot be compared, the guard strips (fail safe).

Transports apply this rule through RedirectSecurity when a redirect is followed. The OkHttp and Apache HttpClient 4.5 transports, and the Apache HttpClient 5 transport on ordinary (not SSRF-guarded) requests, compare origins on the library's own host object, so a host name that java.net.URI cannot parse (for example the underscore name my_service) is compared like any other host. The transports apply the rule on the clients they create; the OkHttp, Apache HttpClient 4.5, Apache HttpClient 5 and Jetty transports also apply it to a client you supply (see below for Apache HttpClient 5). On OkHttp the origin is the first request after your application interceptors (so an interceptor that rewrites the URL sets the origin), and only the values you sent are stripped: cookies loaded from the client's CookieJar for the new origin, and credentials from its Authenticator or proxyAuthenticator, still go to the new origin.

The classic client strips headers only, too. Its redirect credential guard runs after the client's other request interceptors, so a Cookie header from its CookieStore is dropped on a hop to another origin like any other. But the client still answers an authentication challenge from the new origin with whatever its CredentialsProvider holds for that host. A credential registered for AuthScope.ANY (for example basicAuth(null, -1, user, pw): a null host and a -1 port are AuthScope.ANY_HOST and AuthScope.ANY_PORT) matches any host, so it is sent to the new origin when that origin asks for it. Scope credentials to the host and port they are meant for.

On a client you supply to the Apache HttpClient 4.5 or Jetty transport, only the values you sent are stripped on a cross-origin hop, as on OkHttp: cookies from the client's own cookie store, and credentials its CredentialsProvider (Apache HttpClient 4.5) or authentication store (Jetty) answers a challenge with, still go to the new origin. Same-origin redirects keep that library's own header handling (Jetty drops Authorization, Cookie and Proxy-Authorization on every redirect, including same-origin ones). The strip happens before the client's own request processing, so a header your client adds itself on every request (an Apache HttpClient 4.5 HttpRequestInterceptor, or a Jetty request listener, that sets a fixed Authorization or X-API-Key) is added again on the hop to the new origin; add credentials per request, or scope such code to the hosts it is meant for. On Jetty, the transport creates each request itself, so that its redirect copies are guarded, and does not call the client's newRequest(URI); an override of that method on a client you supply is not used.

On the client the Apache HttpClient 5 transport creates itself, a redirect to another origin is followed, as on the other transports, with the stripped set removed from that hop and from every later hop in the chain, including a hop back to the first origin; a same-origin redirect keeps your headers. It uses the same per-request context as a client you supply (below), with a redirect strategy that lets the guarded hop go ahead instead of refusing it. Otherwise HttpClient's own redirect handling applies: 301, 302, 303, 307 and 308 are followed, a POST redirected by 301/302, or any method other than GET or HEAD redirected by 303, becomes a new GET that carries none of your headers, at most 50 redirects are followed, and a circular redirect fails with a TransportException. Cookies that the transport's own cookie store holds for the new host are still sent, under HttpClient's cookie rules (cookies are not scoped to a port). An Authorization that a CredentialsProvider adds, and the resends after a retry (such as a 503 with Retry-After) or an authentication challenge, are stripped from a cross-origin hop the same way as on a client you supply. Unguarded requests on this client now connect to a cross-origin Location host (credentials stripped, other headers and the body of a 307/308 forwarded); only SSRF-guarded requests re-check each hop against the policy. They do not use this client; they follow redirects through the policy-checked loop described above.

On a client you supply to the Apache HttpClient 5 transport, the client's own redirect settings decide which redirects are followed: its RedirectStrategy, maxRedirects and circular-redirect settings, and disableRedirectHandling() or HttpClients.createMinimal() (no redirects) behave exactly as you configured them. The transport marks every header in the stripped set as sensitive, and DefaultRedirectStrategy (5.6) refuses, in isRedirectAllowed, a cross-origin redirect that replays sensitive headers: with it, a redirect that replays your headers is not followed to another origin while it carries any of them. You get the 3xx response instead, and the other origin is not contacted. A POST redirected by 301/302, or any method other than GET or HEAD redirected by 303, is followed as a new GET that carries none of your headers.

Whichever strategy follows a redirect to another origin (HttpClient's LaxRedirectStrategy, which allows every redirect, or a custom one), the transport passes each request a context that removes the caller-set credential headers from that hop and from every later hop in the chain, including a hop back to the first origin; same-origin hops before the first cross-origin hop keep them. On a proxied route that is not tunnelled (a plain http target through an HTTP proxy), Proxy-Authorization is kept on every hop, because it authenticates to the proxy that every hop goes through and removing it would make the proxy answer 407 again; on a direct route, or a tunnelled (https) one, it is removed with the rest. Other headers you set, and a 307/308 request body, still go to the new origin, as HttpClient sends them. The strip runs where HttpClient records each hop in the request context, before the client's own request interceptors run, so cookies from the client's cookie store for the new origin and headers its interceptors add still go to the new origin. HttpClient 5 adds a credential from the client's CredentialsProvider before that point, so on a cross-origin hop that Authorization is removed too: a client that relies on its CredentialsProvider to authenticate to a redirect target on another origin is not authenticated there. The guard is the per-request context the transport passes to execute; a client that replaces that context (for example by wrapping it as the parent of a context of its own) is not guarded.

Stale-connection retry​

Keep-alive connections that the server closed while idle fail on reuse. The JDK, Apache HttpClient 4.5, and Jetty transports detect that specific failure and replay the request once on a fresh connection. A failure counts as stale when it is an EOFException, ClosedChannelException, a connection-reset SocketException, or the JDK client's "received no bytes" message (TransportException.isStaleConnectionFailure).

The replay is only attempted when TransportRequest.isSafeToReplay() is true: an idempotent method (GET, HEAD, PUT, DELETE, OPTIONS, TRACE) and a body that is absent or repeatable. A POST is never replayed by this mechanism.

This is separate from the @Remote(retries=...) policy retries described in Next-Generation REST Client, which retry on application-visible failures according to your configuration.

See also​