Send instance name as x-api-key to the OTLP relay#316
Open
archandatta wants to merge 6 commits into
Open
Conversation
Adds an optional in-VM OTLP/HTTP export sink that converts browser telemetry events into OTLP log records and forwards them to a configured endpoint, alongside the existing S2 sink. Gated on OTLP_RELAY_ENDPOINT; no behavior change when unset. - Converter maps each telemetry envelope to an OTLP log record: event type to EventName (also mirrored to kernel.event.type for backends that drop it), structured JSON body, derived severity, and kernel.* attributes. Network and console fields are promoted to semantic-convention attributes. Screenshot and monitor categories are excluded from export. - Sink mirrors S2StorageWriter: an independent ring reader feeds records through the OTel log SDK (otlploghttp + batch processor), running independently of the S2 sink. Export failures surface via a logging exporter wrapper. - Config and startup wiring, plus a dev collector config for local testing.
Rename export env vars from OTLP_RELAY_* to BTEL_OTLP_* so they don't collide with the standard OTEL_* vars that configure the API server's own telemetry, and drop the metro-api-specific "relay" naming (dev points at a collector). Rename the platform-identity config fields to InstanceJWT / InstanceName / MetroName since they aren't OTLP-specific. Default export path is now the standard /v1/logs. Make sink startup non-fatal: an optional best-effort exporter failing to start must not take down the browser, so log and continue instead of os.Exit(1). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The OTel log SDK reports records dropped under sustained backpressure via its global logger at logr V(1). That logger is a no-op until wired, and even once wired it renders just below slog Info, so a default Info handler would still swallow it. Route the SDK global logger to a handler that admits that level when export is enabled, so drops surface instead of being silently discarded (which looked identical to a healthy idle sink). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
TestLoad's expected structs didn't account for the OTLPPath and OTLPServiceName defaults that Load applies, so it failed once those defaults were in place. Assert the applied defaults. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The batch processor caps records per export by count (200), not bytes. With the 1MB per-envelope publish ceiling, a full batch of large records could reach ~200MB and be rejected by the collector/relay, and the SDK drops rejected records without retry. Split each export in the loggingExporter into sub-requests under a byte budget (4MiB) so a batch of large records can't exceed the target's HTTP body limit; a single record (<=1MB) always fits in one sub-request. Adds a chunk-splitting unit test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The metro-api OTLP relay authenticates a VM by its instance name checked against active sessions (mirroring the capmonster/hcaptcha relays), so send INST_NAME as x-api-key instead of the instance JWT as a bearer token. Drop the now-unused KERNEL_INSTANCE_JWT exporter config (the env still exists for the envoy wrapper; the exporter just no longer reads it). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Aligns the in-VM OTLP exporter to the metro-api relay's auth contract. The relay authenticates a VM by its instance name checked against active sessions (the same model as the capmonster/hcaptcha relays), so the exporter now sends
INST_NAMEas thex-api-keyheader instead of the instance JWT as a bearer token. The now-unusedKERNEL_INSTANCE_JWTexporter config is dropped (the env still exists for the envoy wrapper — the exporter just no longer reads it).Companion to the metro-api relay PR (kernel/kernel#2763). Wire contract:
x-api-key: <instance_name>+ POST toBTEL_OTLP_PATH(set to/otlp-relay/v1/logsby provisioning when pointing a VM at the relay).Why
The relay decides which org's destination + credentials to use from the instance identity. Verifying an instance JWT signature would require plumbing
XDS_JWT_SECRETinto metro-api (which has no such secret today and verifies no instance-JWT signatures) and would make the relay the odd one out among sibling relays. Instance names are unguessable cuid2 pairs gated on an active session, sox-api-keyis the consistent, sufficient credential.Changes
server/cmd/api/main.go:Authorization: Bearer <InstanceJWT>→x-api-key: <InstanceName>.server/cmd/config/config.go: remove theInstanceJWTfield and itsLogValueredaction.Test plan
go build ./...andgo test ./cmd/config ./lib/eventsgreen.Notes
archand/kernel-1213/otel-export, Add in-VM OTLP telemetry export sink #308) since that isn't merged yet; retarget tomainonce Add in-VM OTLP telemetry export sink #308 lands.401/403/404from the relay as a normal steady state. A later change can quiet those specific statuses inloggingExporter(needs confirming the status is cleanly extractable from theotlploghttperror) so they don't inflate the failure counter/logs.Note
Medium Risk
Changes telemetry export credentials and must deploy with the relay PR; misconfiguration or ordering can break OTLP until both sides match.
Overview
Switches in-VM OTLP export to the metro-api relay’s auth model:
INST_NAMEis sent asx-api-keyinstead ofKERNEL_INSTANCE_JWTas anAuthorization: Bearerheader, matching capmonster/hcaptcha relays (active-session instance name lookup).server/cmd/config: dropsInstanceJWT/KERNEL_INSTANCE_JWTfrom exporter config and startup logging redaction;InstanceNameandMetroNameremain for the OTLP resource and the new header.KERNEL_INSTANCE_JWTis unchanged for other components (e.g. envoy wrapper).Reviewed by Cursor Bugbot for commit 8f83406. Bugbot is set up for automated code reviews on this repo. Configure here.