From 7c0b886004abf232130cd0cb6127ad7143e2ca9a Mon Sep 17 00:00:00 2001 From: Abhay Singh Date: Thu, 27 Aug 2026 15:02:32 +0530 Subject: [PATCH] fix(vertex): validate location region to close a path-parameter SSRF (#3304) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Description Closes #3280. `vertex_target_for_location` interpolated the user-controlled `location` path segment straight into the upstream **hostname** with no validation: ```python return f"https://{location}-aiplatform.googleapis.com" ``` `location` comes from the Vertex route path `/{api_version}/projects/{project}/locations/{location}/publishers/...`, so it is fully attacker-controlled. A `location` of `169.254.169.254#` (decoded from a percent-encoded `%23` in the path) produces: ``` https://169.254.169.254#-aiplatform.googleapis.com ``` which an HTTP client parses as host `169.254.169.254` with the remainder treated as a URL fragment — a server-side request forgery (CWE-918) to the cloud metadata endpoint. A `host:port` payload (`127.0.0.1:44919#`) reaches an arbitrary internal port the same way. I confirmed the pre-fix formula against the reported PoC: ``` '169.254.169.254#' -> 'https://169.254.169.254#-aiplatform.googleapis.com' host='169.254.169.254' port=None '127.0.0.1:44919#' -> 'https://127.0.0.1:44919#-aiplatform.googleapis.com' host='127.0.0.1' port=44919 ``` The fix validates `location` against a strict GCP region shape (`^[a-z0-9]+(?:-[a-z0-9]+)*$`) before interpolation. Anything that is not a well-formed region — including port, path, userinfo, and fragment-delimiter payloads — falls back to the default public `aiplatform.googleapis.com` endpoint, which can never resolve to an attacker-chosen host. Real regions (`us-central1`, `europe-west4`, ...), `global`, empty, and an explicitly configured gateway target are all unaffected. Root-cause input validation on the pure routing formula fully closes the reported vector; it is the same place every Vertex route derives its target from, so there is one choke point rather than a per-route guard. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/providers/vertex/runtime.py`: added `_VERTEX_REGION_RE` (anchored `^[a-z0-9]+(?:-[a-z0-9]+)*$`) and `_VERTEX_GLOBAL_API_URL`. `vertex_target_for_location` now falls back to the public endpoint for any `location` that is empty, `global`, or not a well-formed region, and only interpolates a validated region into the hostname. - `tests/test_provider_vertex_runtime.py`: added a parametrized region-acceptance test, a parametrized SSRF-payload test (fragment/host:port/path/userinfo/underscore/uppercase/malformed-hyphen — asserts the fallback endpoint **and** that the parsed host is `aiplatform.googleapis.com` with no port), and a test that an explicit gateway target is still returned verbatim. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality ### Test Output ```text tests/test_provider_vertex_runtime.py tests/test_provider_proxy_targets.py tests/test_vertex_claude_compression.py -> 39 passed uvx ruff@0.16.2 check headroom/providers/vertex/runtime.py tests/test_provider_vertex_runtime.py -> All checks passed! uvx mypy@1.20.2 headroom/providers/vertex/runtime.py -> Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1, ruff 0.16.2 and mypy 1.20.2 via uvx. - Exact command / steps: reproduced the SSRF against the pre-fix formula (the two lines above, showing `urlsplit(...).hostname` = the injected `169.254.169.254` / `127.0.0.1:44919`); applied the fix and re-ran, confirming both fall back to `https://aiplatform.googleapis.com` with host `aiplatform.googleapis.com`. Red/green: with the fix reverted (`git stash`) the 11 SSRF-payload cases fail; restored, all 27 vertex-runtime tests pass. - Observed result: a malicious `location` can no longer place a host, port, path, or fragment delimiter into the upstream hostname; legitimate regions and configured gateways are unchanged. - Not tested: a live end-to-end request against a real metadata endpoint (would require a network SSRF target); the URL-construction root cause is covered by unit tests, including host/port parsing of the constructed URL. ## Runtime Rollout Safety - Rollout-managed feature(s): none — no feature flag or rollout channel involved. - Minimum rollout channel: N/A. - Stable/default behavior changed: only for malformed `location` values, which previously produced a broken/attacker-controlled host and now resolve to the public Vertex endpoint. Valid regions, `global`, empty, and configured gateways are byte-for-byte unchanged. - Kill switch / disable path: N/A (no config surface added). - Unsafe override required: no. - Qualification impact: none. - Rollback path: revert this commit; `location` goes back to being interpolated unvalidated (reintroducing the SSRF). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation (N/A: no user-facing surface change for valid input) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes The region allowlist is intentionally strict (lowercase alphanumeric groups joined by single hyphens), matching the shape of every GCP Vertex region; `global`/empty keep their existing public-endpoint behavior. If a defense-in-depth `is_safe_upstream_url` check at the route layer is also wanted (as the issue suggests), that can follow as a separate change — this PR fixes the root cause at the single point where the hostname is built. --- headroom/providers/vertex/runtime.py | 33 ++++++++++++++-- tests/test_provider_vertex_runtime.py | 55 +++++++++++++++++++++++++++ 2 files changed, 85 insertions(+), 3 deletions(-) diff --git a/headroom/providers/vertex/runtime.py b/headroom/providers/vertex/runtime.py index 5a1931ff7..4cf6194de 100644 --- a/headroom/providers/vertex/runtime.py +++ b/headroom/providers/vertex/runtime.py @@ -2,10 +2,22 @@ from __future__ import annotations +import re from dataclasses import dataclass from headroom.providers.registry import DEFAULT_VERTEX_API_URL +# The public (multi-region) Vertex endpoint, used for ``global`` and for any +# location that is not a well-formed region. +_VERTEX_GLOBAL_API_URL = "https://aiplatform.googleapis.com" + +# A GCP region label: lowercase alphanumeric groups joined by single hyphens +# (e.g. ``us-central1``, ``europe-west4``, ``asia-northeast1``). Anchored and +# deliberately strict — no dots, colons, slashes, ``#``, ``@``, uppercase, or +# empty groups — so a user-controlled ``location`` can never carry a host, +# port, path, or URL-fragment delimiter into the interpolated hostname. +_VERTEX_REGION_RE = re.compile(r"^[a-z0-9]+(?:-[a-z0-9]+)*$") + VERTEX_GOOGLE_PUBLISHER = "google" VERTEX_ANTHROPIC_PUBLISHER = "anthropic" VERTEX_GOOGLE_PROVIDER_NAME = "vertex:google" @@ -50,9 +62,24 @@ def vertex_anthropic_target(base_url: str, *, versionless_route: bool = False) - def vertex_target_for_location(configured_target: str, location: str) -> str: - """Return the Vertex upstream target for a request location.""" + """Return the Vertex upstream target for a request location. + + ``location`` is a user-controlled URL path segment that is interpolated into + the upstream hostname, so it must be validated against the GCP region shape + before use. Without that check a value such as ``169.254.169.254#`` (decoded + from a percent-encoded ``%23`` in the path) produces + ``https://169.254.169.254#-aiplatform.googleapis.com``, which an HTTP client + parses as host ``169.254.169.254`` with the remainder treated as a URL + fragment — a server-side request forgery to the cloud metadata endpoint + (CWE-918). Any ``location`` that is not a well-formed region (including port, + path, or fragment-delimiter payloads) falls back to the default public + endpoint, which can never resolve to an attacker-chosen host. + + An explicitly configured gateway target still wins outright; region + derivation only applies when running against the default Vertex endpoint. + """ if configured_target and configured_target != DEFAULT_VERTEX_API_URL: return configured_target - if not location or location == "global": - return "https://aiplatform.googleapis.com" + if not location or location == "global" or not _VERTEX_REGION_RE.match(location): + return _VERTEX_GLOBAL_API_URL return f"https://{location}-aiplatform.googleapis.com" diff --git a/tests/test_provider_vertex_runtime.py b/tests/test_provider_vertex_runtime.py index cf0bad667..8c3584054 100644 --- a/tests/test_provider_vertex_runtime.py +++ b/tests/test_provider_vertex_runtime.py @@ -1,5 +1,9 @@ from __future__ import annotations +from urllib.parse import urlsplit + +import pytest + from headroom.providers.registry import DEFAULT_VERTEX_API_URL from headroom.providers.vertex import ( VERTEX_ANTHROPIC_PROVIDER_NAME, @@ -71,3 +75,54 @@ def test_vertex_target_for_location_honors_explicit_gateway() -> None: assert vertex_target_for_location("https://vertex-gateway.internal", "europe-west1") == ( "https://vertex-gateway.internal" ) + + +_VERTEX_PUBLIC_ENDPOINT = "https://aiplatform.googleapis.com" + + +@pytest.mark.parametrize( + "region", + ["us-central1", "europe-west4", "asia-northeast1", "me-central1", "us-east5"], +) +def test_vertex_target_for_location_accepts_real_regions(region: str) -> None: + assert vertex_target_for_location(DEFAULT_VERTEX_API_URL, region) == ( + f"https://{region}-aiplatform.googleapis.com" + ) + + +@pytest.mark.parametrize( + "malicious", + [ + "169.254.169.254#", # fragment delimiter -> cloud metadata IP (the reported PoC) + "127.0.0.1:44919#", # host:port + fragment + "169.254.169.254/latest/meta-data/iam#", # path injection + "169.254.169.254:80", # port injection + "evil.example", # dotted host + "foo@evil.example", # userinfo delimiter + "us_central1", # underscore (not a region) + "US-CENTRAL1", # uppercase + "-leading", # leading hyphen + "trailing-", # trailing hyphen + "a--b", # empty hyphen group + ], +) +def test_vertex_target_for_location_rejects_ssrf_payloads(malicious: str) -> None: + """A non-region ``location`` must never carry an attacker-chosen host into + the interpolated Vertex hostname (CWE-918). It falls back to the public + endpoint, and the parsed host is always the legitimate Vertex host — never + a metadata IP, loopback, or injected authority. + """ + target = vertex_target_for_location(DEFAULT_VERTEX_API_URL, malicious) + assert target == _VERTEX_PUBLIC_ENDPOINT + parsed = urlsplit(target) + assert parsed.hostname == "aiplatform.googleapis.com" + assert parsed.port is None + + +def test_vertex_target_for_location_ssrf_fallback_only_on_default_target() -> None: + """A validated non-region value still cannot override an explicitly + configured gateway (that path returns the operator's target verbatim and + is not user-derived).""" + assert vertex_target_for_location("https://gw.internal", "169.254.169.254#") == ( + "https://gw.internal" + )