mirror of
https://github.com/maziggy/bambuddy.git
synced 2026-08-11 00:30:12 -04:00
Builds on the recent uvicorn-access-log-into-bambuddy.log change.
Until now the access line told us who called an endpoint, but there
was no way to tie that line to the application records emitted on the
server side while handling that request. The rogue stop_print mystery
on 2026-04-26 left exactly that gap: even with access logs piped in,
correlating "this POST landed" with "this MQTT publish went out 6 ms
later" required eyeball-matching timestamps across different loggers.
A new ContextVar + middleware + logging filter wire a trace ID through
every record:
* trace_id_middleware mints an 8-char hex ID per request (or honours
a sane inbound X-Trace-Id for cross-system correlation), stores it
in trace_id_var (ContextVar), echoes it on the response as
X-Trace-Id, and resets the var in finally.
* TraceIDFilter, attached to root + uvicorn.access, copies the
current trace_id_var value onto every LogRecord so the format
string [%(trace_id)s] resolves to the right ID per record.
* Records emitted outside any request scope (startup, MQTT
callbacks, scheduler) get a stable "-" placeholder so the column
stays visually aligned and grep stays simple.
ContextVars are the right plumbing because asyncio copies the current
context into every asyncio.create_task, so background work spawned
from inside a request inherits the same ID without explicit threading.
request.state can't make that hop. The logging filter also has no
access to the FastAPI request object — it runs synchronously inside
the stdlib logging machinery — and the ContextVar is the only
mechanism that bridges async request scope to sync log emission.
Inbound X-Trace-Id is hard-validated against [A-Za-z0-9_-]+ (max 64
chars) before being honoured — a hostile/buggy caller cannot smuggle
log-injection payloads (newlines, control chars, megabyte blobs) into
bambuddy.log via the trace ID column; values that fail the gate
silently trigger a freshly minted server-side ID rather than failing
the request.
Middleware is decorated AFTER auth_middleware on purpose: Starlette
stacks @app.middleware decorators LIFO so the last-decorated runs
first inbound, making trace stamp the OUTERMOST layer — auth log
lines and every record emitted on the way down to and back from the
route handler all carry the same ID.
Output now correlates as:
2026-04-26 09:51:39,152 INFO [uvicorn.access] [a4f3b1e7] - "POST
/api/v1/printers/1/print/stop HTTP/1.1" 200
2026-04-26 09:51:39,158 INFO [bambu_mqtt] [a4f3b1e7] [SERIAL] Sent
stop print command
One grep a4f3b1e7 returns the full causality chain.
30 new tests: 22 unit (ContextVar placeholder, filter copies value,
asyncio task propagation, concurrent-request isolation, hex generator
uniqueness, hostile-payload validator, max-length boundary, all four
write verbs survive, GET/HEAD/OPTIONS dropped, URL-substring false-
match guards, edge cases) and 8 integration (X-Trace-Id round-trips,
body matches header, hostile inbound replaced, overlong inbound
replaced, ContextVar resets after request, generator format stable,
each request gets unique ID).
118 lines
5.1 KiB
Python
118 lines
5.1 KiB
Python
"""Per-request trace ID plumbing.
|
|
|
|
Each HTTP request gets a short hex ID set in a ``ContextVar``; downstream
|
|
log records (application *and* uvicorn access) read the same context and
|
|
include the ID in their output. The result is that one ``grep <trace_id>``
|
|
on ``bambuddy.log`` returns the access line + every line emitted on the
|
|
server side while that request was being handled — closing the loop
|
|
opened by piping uvicorn access into the file: the access line tells you
|
|
*who* called the endpoint, the trace ID tells you *what else happened*
|
|
on the server because of it.
|
|
|
|
Why a ContextVar instead of e.g. ``request.state``:
|
|
|
|
* asyncio copies the current context into every ``asyncio.create_task``,
|
|
so background work spawned from within a request inherits the same
|
|
trace ID without having to be passed it explicitly. ``request.state``
|
|
doesn't survive that hop.
|
|
* The logging filter has no access to the FastAPI request object — it
|
|
runs synchronously inside the stdlib logging machinery — and the
|
|
ContextVar is the only mechanism that bridges async request scope to
|
|
sync log emission.
|
|
|
|
Why no fancy structured-logging schema: this is a small project. The
|
|
existing log format is a single line per record; we add a single
|
|
bracketed token for the trace ID and stop there. If structured logging
|
|
is wanted later, it can layer on top — the ContextVar carries an opaque
|
|
string regardless of what consumes it downstream.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
import re
|
|
import secrets
|
|
from contextvars import ContextVar
|
|
|
|
# Default ``"-"`` (instead of None or empty string) so the format string
|
|
# always produces a stable visual width; a bare empty bracket pair would
|
|
# read as "no trace ID at all" which is hard to grep for. ``-`` reads as
|
|
# "no value in this column" the way it does in HTTP access logs already.
|
|
TRACE_ID_PLACEHOLDER = "-"
|
|
|
|
trace_id_var: ContextVar[str] = ContextVar("trace_id", default=TRACE_ID_PLACEHOLDER)
|
|
|
|
# Length of a freshly minted trace ID in hex chars. 8 chars = 32 bits of
|
|
# entropy = ~4 billion possibilities; collisions are astronomically
|
|
# unlikely within a single rotation window of bambuddy.log and grep stays
|
|
# easy at this length. Increase later if it proves too short for a busy
|
|
# install — the filter and format don't care about width.
|
|
_GENERATED_LENGTH = 8
|
|
|
|
# Bound on how long an *inbound* trace ID can be when echoed from the
|
|
# X-Trace-Id request header. Without a cap a malicious / buggy client
|
|
# could push 1 MB of garbage into every log line for a request. 64 chars
|
|
# comfortably accommodates UUIDs (32 hex), Datadog-style 64-bit IDs,
|
|
# OpenTelemetry's 32-hex spans — anything longer is almost certainly
|
|
# wrong and we'd rather mint our own than honour it.
|
|
_MAX_INBOUND_LENGTH = 64
|
|
|
|
# Whitelist of characters allowed in an inbound trace ID. Restricted to
|
|
# the alphanumerics + a small set of separators that real-world
|
|
# correlation IDs use, so newlines / quotes / control chars cannot be
|
|
# smuggled into log lines via the X-Trace-Id header. A request with an
|
|
# unacceptable header just gets a freshly minted server-side ID — we
|
|
# never reject the request for it.
|
|
_VALID_INBOUND = re.compile(r"^[A-Za-z0-9_\-]+$")
|
|
|
|
|
|
def get_trace_id() -> str:
|
|
"""Return the current trace ID, or the placeholder if none is set."""
|
|
return trace_id_var.get()
|
|
|
|
|
|
def generate_trace_id() -> str:
|
|
"""Mint a fresh server-side trace ID."""
|
|
return secrets.token_hex(_GENERATED_LENGTH // 2)
|
|
|
|
|
|
def normalise_inbound_trace_id(raw: str | None) -> str | None:
|
|
"""Validate and return a caller-supplied trace ID, or ``None`` to mint fresh.
|
|
|
|
Accepts only short alphanumeric + ``_-`` strings so a hostile or buggy
|
|
client can't smuggle log-injection payloads through the X-Trace-Id
|
|
header. Returns ``None`` for any input that fails the gate, signalling
|
|
to the middleware that it should generate one instead.
|
|
"""
|
|
if raw is None:
|
|
return None
|
|
if not raw or len(raw) > _MAX_INBOUND_LENGTH:
|
|
return None
|
|
if not _VALID_INBOUND.match(raw):
|
|
return None
|
|
return raw
|
|
|
|
|
|
class TraceIDFilter(logging.Filter):
|
|
"""Inject the current ``trace_id_var`` value into every LogRecord.
|
|
|
|
Attach to the file handler (or any handler whose format string
|
|
references ``%(trace_id)s``) so that every line written through that
|
|
handler carries the request scope it was generated under. The filter
|
|
always returns ``True`` — it never drops records, only annotates
|
|
them.
|
|
|
|
Records emitted outside any HTTP request (startup, MQTT callbacks,
|
|
scheduled tasks not chained from a request) get the placeholder
|
|
string, so the format column stays aligned and absent values are
|
|
obviously visible as ``[-]`` rather than blanks.
|
|
"""
|
|
|
|
def filter(self, record: logging.LogRecord) -> bool: # noqa: A003 — stdlib API name
|
|
# Use direct attribute set (not setdefault-style) so the value is
|
|
# always taken from the *current* context — a record formatted on
|
|
# a different task than where it was created (rare but possible
|
|
# via QueueHandler or async-throttled handlers) still picks up
|
|
# the right ID.
|
|
record.trace_id = trace_id_var.get()
|
|
return True
|