Skip to content

fix: harden metric scrape edge cases#2297

Open
zeitlinger wants to merge 3 commits into
mainfrom
codex/fix-sec-report-bugs
Open

fix: harden metric scrape edge cases#2297
zeitlinger wants to merge 3 commits into
mainfrom
codex/fix-sec-report-bugs

Conversation

@zeitlinger

Copy link
Copy Markdown
Member

Summary

This addresses the bug reports from #2282, #2283, #2284, #2285, #2286, and #2287 in one PR.

  • Fix Buffer stripe indexing so large thread IDs cannot produce negative stripe indexes.
  • Bound the Buffer spin wait during collection and replay buffered observations before failing the scrape on timeout.
  • Return a generic HTTPServer error response instead of exposing stack traces to clients.
  • Avoid blocking the HTTPServer dispatcher on executor saturation by using a bounded queue with the default rejection policy.
  • Close rejected authenticated request bodies instead of draining them without a size limit.
  • Bound default query parsing by query length and parameter count, and return HTTP 400 for invalid query parameters.
  • De-duplicate exact metric-name filters for faster lookup.
  • Redact invalid configuration values from PrometheusPropertiesException messages.

Fixes #2282
Fixes #2283
Fixes #2284
Fixes #2285
Fixes #2286
Fixes #2287

Validation

  • mise run lint:fix
  • ./mvnw -pl prometheus-metrics-core,prometheus-metrics-config,prometheus-metrics-exporter-common,prometheus-metrics-exporter-httpserver,prometheus-metrics-model test -Dcoverage.skip=true -Dcheckstyle.skip=true
  • mise run build

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

⚠️ API changes detected — maintainer review required

This PR modifies the published API diff for the following module(s):

  • prometheus-metrics-exporter-common
  • prometheus-metrics-exporter-httpserver

Please review the changes in docs/apidiffs/current_vs_latest/ carefully before approving.

@zeitlinger zeitlinger force-pushed the codex/fix-sec-report-bugs branch from 6cc0f6c to 9a7bf86 Compare July 9, 2026 14:03
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger zeitlinger force-pushed the codex/fix-sec-report-bugs branch from 9a7bf86 to 0020301 Compare July 10, 2026 17:08
@zeitlinger zeitlinger marked this pull request as ready for review July 10, 2026 17:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens the metrics scrape path across core buffering, query parsing/filtering, and the standalone HTTPServer exporter to address multiple reported scrape-triggered crashes/DoS vectors and information disclosure.

Changes:

  • Make Buffer thread striping overflow-safe and bound spin-wait during collection (replaying buffered observations before failing).
  • Stop exposing stack traces to HTTP clients; return a generic 500 error body while logging server-side.
  • Bound query parsing (length + parameter count) and de-duplicate metric-name filters for more predictable scrape cost.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
prometheus-metrics-model/src/main/java/io/prometheus/metrics/model/registry/MetricNameFilter.java De-duplicates filter inputs and adds fast-path contains() checks.
prometheus-metrics-exporter-httpserver/src/test/java/io/prometheus/metrics/exporter/httpserver/HTTPServerTest.java Updates tests for new default executor + generic error responses.
prometheus-metrics-exporter-httpserver/src/test/java/io/prometheus/metrics/exporter/httpserver/HttpExchangeAdapterTest.java Verifies generic 500 body and absence of stack traces.
prometheus-metrics-exporter-httpserver/src/main/java/io/prometheus/metrics/exporter/httpserver/HTTPServer.java Switches default executor to bounded queue and changes unauthorized-body handling.
prometheus-metrics-exporter-httpserver/src/main/java/io/prometheus/metrics/exporter/httpserver/HttpExchangeAdapter.java Replaces stack-trace responses with a generic error message + server-side logging.
prometheus-metrics-exporter-httpserver/src/main/java/io/prometheus/metrics/exporter/httpserver/BlockingRejectedExecutionHandler.java Removes the blocking rejection handler to avoid dispatcher thread stalls.
prometheus-metrics-exporter-common/src/test/java/io/prometheus/metrics/exporter/common/PrometheusScrapeHandlerTest.java Adds tests for rejecting overly-long / overly-many query parameters.
prometheus-metrics-exporter-common/src/main/java/io/prometheus/metrics/exporter/common/PrometheusScrapeHandler.java Catches invalid query parameter parsing and returns HTTP 400.
prometheus-metrics-exporter-common/src/main/java/io/prometheus/metrics/exporter/common/PrometheusHttpRequest.java Implements bounded query parsing for getParameterValues().
prometheus-metrics-exporter-common/src/main/java/io/prometheus/metrics/exporter/common/InvalidQueryParameterException.java Introduces an internal exception for invalid query parsing.
prometheus-metrics-core/src/test/java/io/prometheus/metrics/core/metrics/BufferTest.java Adds regression tests for stripe indexing and timeout/replay behavior.
prometheus-metrics-core/src/main/java/io/prometheus/metrics/core/metrics/Buffer.java Uses floorMod for striping; adds spin-wait timeout + replay-on-timeout.
prometheus-metrics-config/src/test/java/io/prometheus/metrics/config/UtilTest.java Updates tests for redacted/escaped invalid-value error messages.
prometheus-metrics-config/src/test/java/io/prometheus/metrics/config/OpenMetrics2PropertiesTest.java Updates expected messages to quote/escape invalid values.
prometheus-metrics-config/src/test/java/io/prometheus/metrics/config/ExporterPushgatewayPropertiesTest.java Updates expected messages to quote/escape invalid values.
prometheus-metrics-config/src/test/java/io/prometheus/metrics/config/ExporterPropertiesTest.java Updates expected messages to quote/escape invalid values.
prometheus-metrics-config/src/main/java/io/prometheus/metrics/config/Util.java Centralizes invalid-value message formatting with escaping + truncation.
prometheus-metrics-config/src/main/java/io/prometheus/metrics/config/ExporterPushgatewayProperties.java Uses Util.invalidValueMessage(...) to avoid leaking raw config values.
integration-tests/it-exporter/it-exporter-test/src/test/java/io/prometheus/metrics/it/exporter/test/HttpServerIT.java Validates generic HTTPServer error body in integration tests.
integration-tests/it-exporter/it-exporter-test/src/test/java/io/prometheus/metrics/it/exporter/test/ExporterIT.java Factors error-body assertions behind an overridable hook.
docs/apidiffs/current_vs_latest/prometheus-metrics-exporter-httpserver.txt Records API-diff output for exporter-httpserver changes.
docs/apidiffs/current_vs_latest/prometheus-metrics-exporter-common.txt Records API-diff output for exporter-common changes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 79 to 82
int idx = pair.indexOf("=");
if (idx != -1 && URLDecoder.decode(pair.substring(0, idx), "UTF-8").equals(name)) {
result.add(URLDecoder.decode(pair.substring(idx + 1), "UTF-8"));
}
Comment on lines +41 to +43
private static final int DEFAULT_MIN_THREADS = 1;
private static final int DEFAULT_MAX_THREADS = 10;
private static final int DEFAULT_QUEUE_SIZE = 100;
Comment on lines 158 to 160
} else {
drainInputAndClose(exchange);
exchange.getRequestBody().close();
exchange.sendResponseHeaders(403, -1);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment