Skip to content

feat(pii): regex-only block-output redaction + drop GLiNER/GPU image#5697

Merged
TheodoreSpeaks merged 4 commits into
stagingfrom
feat/pii-regex-block-outputs
Jul 16, 2026
Merged

feat(pii): regex-only block-output redaction + drop GLiNER/GPU image#5697
TheodoreSpeaks merged 4 commits into
stagingfrom
feat/pii-regex-block-outputs

Conversation

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator

Summary

  • Restrict the block-output PII stage to regex/checksum entities only — drop the 4 spaCy-NER types (PERSON/LOCATION/NRP/DATE_TIME). Enforced at the UI grid, the API contract (migration-safe strip, never reject → existing rules stay saveable), and the resolver (so un-migrated rules can't feed NER into execution).
  • Add a Presidio skip-spaCy fast path: regex-only requests reuse a blank NlpArtifacts to bypass the spaCy NER pass entirely — the per-leaf cost that made bulk block-output redaction slow. redact_batch logs nlp=skip|full.
  • Remove the GLiNER/GPU image: strip torch/CUDA/GLiNER from docker/pii.Dockerfile, delete engines.py/requirements-gliner/bench+tests, restore server.py to a self-contained CPU spaCy service (keeps the NER-only pipe-disable ~2x win + the logging-emit fix), and drop the Helm pii.engine/pii.device wiring.

Type of Change

  • New feature + cleanup

Testing

  • 43 unit tests pass (regex-only grid + strip, contract transform, resolver strip); type-check, lint, check:api-validation:strict clean; py_compile server.py; helm lint + helm unittest (10/10) pass.
  • ⚠️ The server skip-spaCy path is logic/syntax-verified but not runtime-tested locally (Presidio isn't installable here). Validate on staging: a regex-only block-output redaction should log nlp=skip and run dramatically faster than a NER (nlp=full) request.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Jul 15, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 15, 2026 11:49pm

Request Review

@cursor

cursor Bot commented Jul 15, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Block-output masking no longer covers NER entity types (behavior change for in-flight redaction), and the skip-spaCy fast path is not covered by removed integration tests—staging validation of nlp=skip is advised. GLiNER deployments must move to spaCy-only images.

Overview
Block-output PII redaction is now regex/checksum only: spaCy-NER types (PERSON, LOCATION, NRP, DATE_TIME, plus ORGANIZATION) are stripped from that stage in the UI picker, API schema transform, retention resolver, and save/hydration paths. Existing rules that only selected NER for block outputs are migration-safe (entities removed, stage disabled rather than rejected).

The Presidio service adds a regex-only analyze fast path that skips the spaCy NLP pass when the request lists only non-NER entities and has no positive score_threshold, using prebuilt blank NlpArtifacts; /redact_batch logs nlp=skip|full. Analyzer setup is inlined in server.py (large spaCy models, VIN + locale recognizers, disabled non-NER spaCy pipes, fixed request timing logs).

GLiNER/GPU support is removed: engines.py, GLiNER deps/bench/tests, torch/CUDA/small spaCy models from the PII image, and Helm pii.engine / pii.device wiring.

Reviewed by Cursor Bugbot for commit 47ca743. Bugbot is set up for automated code reviews on this repo. Configure here.

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile review

@greptile-apps

greptile-apps Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR makes block-output PII redaction regex-only and simplifies the PII service. The main changes are:

  • Filters spaCy NER entities from block-output policies in the UI, API contract, and runtime resolver.
  • Adds a spaCy-skipping path for eligible regex-only requests.
  • Removes GLiNER, Torch, CUDA, and related image configuration.
  • Updates the Presidio service and Helm deployment for CPU-only spaCy.

Confidence Score: 5/5

This looks safe to merge.

  • The latest changes cover spaCy-backed entities such as ORGANIZATION through the live recognizer registry and an explicit fallback set.
  • Requests with a positive score threshold use the full NLP path, preserving context-aware analysis.
  • No blocking issue was found in the updated code.

Important Files Changed

Filename Overview
apps/pii/server.py Adds the regex-only fast path, derives spaCy-backed entities from the recognizer registry, and preserves full analysis for positive score thresholds.
apps/sim/lib/guardrails/pii-entities.ts Defines and filters the spaCy NER entity set used by block-output policies.
apps/sim/lib/api/contracts/primitives.ts Normalizes block-output policies by stripping NER entities and disabling empty stages.
apps/sim/lib/billing/retention.ts Applies the regex-only restriction while resolving stored PII policies.
docker/pii.Dockerfile Removes the GLiNER and GPU layers and packages the standalone spaCy service.

Reviews (4): Last reviewed commit: "fix(pii): include ORGANIZATION in app-si..." | Re-trigger Greptile

Comment thread apps/pii/server.py Outdated
@greptile-apps

greptile-apps Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR makes block-output PII redaction regex-only and removes the GLiNER deployment path. The main changes are:

  • Filters spaCy NER entities from block-output policies in the UI, API contract, and runtime resolver.
  • Adds a blank-artifacts fast path for regex-only Presidio requests.
  • Consolidates the PII service into a CPU spaCy implementation.
  • Removes GLiNER dependencies, image layers, tests, and Helm settings.

Confidence Score: 4/5

Thresholded Presidio requests can miss context-sensitive PII and need fixes before merging.

  • Empty artifacts bypass the tokens and lemmas used for context scoring.
  • Disabled spaCy pipes weaken context scoring on every analysis path.
  • The UI, contract, resolver, image, and Helm changes otherwise remain consistent.

apps/pii/server.py

Security Review

The changed Presidio paths can expose PII in thresholded requests. Empty artifacts and globally disabled language pipes prevent context enhancement, allowing context-sensitive pattern matches to fall below the requested threshold.

Important Files Changed

Filename Overview
apps/pii/server.py Adds regex-only analysis helpers and globally disables spaCy pipes, creating false-negative paths for thresholded requests.
apps/sim/lib/guardrails/pii-entities.ts Defines the shared NER entity set and filters block-output choices and hydrated policies.
apps/sim/lib/api/contracts/primitives.ts Strips NER entities from block-output policies and disables stages left empty.
apps/sim/lib/billing/retention.ts Enforces regex-only block-output policies during runtime resolution.
apps/sim/ee/data-retention/components/data-retention-settings.tsx Restricts block-output selections and sanitizes policies before saving.
docker/pii.Dockerfile Removes GLiNER, Torch, CUDA, development-test dependencies, and deleted source files.
helm/sim/templates/deployment-pii.yaml Removes obsolete engine and device wiring while retaining generic environment variables.

Reviews (2): Last reviewed commit: "feat(pii): restrict block-output redacti..." | Re-trigger Greptile

Comment thread apps/pii/server.py
Comment thread apps/pii/server.py
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/lib/guardrails/pii-entities.ts Outdated
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 47ca743. Configure here.

@TheodoreSpeaks TheodoreSpeaks merged commit 4d6301c into staging Jul 16, 2026
18 checks passed
@TheodoreSpeaks TheodoreSpeaks deleted the feat/pii-regex-block-outputs branch July 16, 2026 00:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant