feat(lib): capture client-attested build provenance

[max-parke-scale]

Adds agentex.lib.utils.build_provenance — the shared capture util for client-attested build provenance: git coordinates (repo/commit/ref/subpath), a deterministic working_tree_hash over the build inputs (not the tarball), a dirty flag (Go vcs.modified / Nix dirtyRev shape), and normalize_remote. Capture is best-effort and never raises into a build. Also makes the build archive’s member order deterministic via a sorted enumeration shared with the hash.

First of three surfaces for AGX1-418 (Phase 1, client-attested). Provenance is delivered via the build-record sink — source_* columns on POST /v5/builds (Surface C, scaleapi) consumed by the sgpctl + CI uploaders (Surface B, scaleapi/sgp). This PR lands the util + archive determinism where agentex.lib lives; the uploaders/columns follow.

Scope notes

• No build-info.json / runtime sink. An earlier revision wrote build-info.json into the build context for the register_agent() → registration_metadata path. Greptile (T-Rex) correctly flagged it as dead-on-arrival (written to the archive root, which the templates’ Dockerfiles don’t COPY and locate_build_info_path() doesn’t read). It’s also redundant: AgentexCloudDeploy.build_id is an FK to AgentexCloudBuild, so a deployment’s source provenance derives from the build record over that join — the same Build→Deploy edge lineage already traverses. Dropped; can be revived (correctly placed) if a real consumer for deployment-history provenance ever appears.

Identity model

working_tree_hash is always computed (content identity); commit/ref/repo anchor it to source when in a git work tree; dirty records uncommitted changes (None outside git).

Tests

20 provenance unit tests (clean/dirty/untracked/detached-HEAD/no-remote/non-git/monorepo-subpath, hash determinism + one-byte/added/exec-bit/symlink sensitivity, and a never-raises-on-hash-failure guard). ruff/pyright clean; full lib suite green.

🧑‍💻🤖 — posted via Claude Code

Greptile Summary

This PR introduces agentex.lib.utils.build_provenance, a best-effort utility that captures client-attested build provenance (git coordinates, a deterministic working_tree_hash over build inputs, and a dirty flag) and makes the build archive's member order deterministic by sharing the same sorted file enumeration between the hash and the tarball.

• build_provenance.py: capture_build_provenance probes git and computes a content hash; all failure modes degrade gracefully (_safe_working_tree_hash wraps the hash, _git swallows subprocess errors, missing fields fall back to None). normalize_remote strips credentials and scheme from remote URLs. source_fields() exposes only the six fields destined for POST /v5/builds. The build_info() method and author_name/author_email fields are vestigial from the dropped runtime sink and have no current callers, but the author explicitly notes they can be revived correctly if a deployment-history consumer appears.
• agent_manifest.py: zipped() now delegates enumeration to iter_context_files, which sorts entries by POSIX relative path and also includes dangling symlinks (previously excluded by the is_file() guard). This keeps the archive contents and the hash in lock-step.
• tests/lib/test_build_provenance.py: 20 unit tests covering clean, dirty, untracked, detached-HEAD, no-remote, non-git, and monorepo-subpath cases, plus hash sensitivity to one-byte changes, added files, exec-bit flips, and symlink target changes, and a monkeypatch guard confirming no exception escapes capture_build_provenance.

Confidence Score: 5/5

Safe to merge; the change is additive, all provenance capture is best-effort and isolated from the build path, and the archive ordering fix is a pure determinism improvement.

The new module is well-structured with proper fallback handling at every failure point. The test suite is thorough and covers the edge cases that matter. The one behavioral change to the archive (dangling symlinks now included) is intentional and consistent with the hash semantics. No incorrect data flows, missing guards, or broken contracts were found.

No files require special attention.

Important Files Changed

Filename	Overview
src/agentex/lib/utils/build_provenance.py	New module implementing client-attested build provenance capture; correctly wraps all hash and git operations in best-effort guards so nothing raises into a build. build_info() is vestigial from the dropped runtime sink (acknowledged in the PR description) and author_name/author_email are collected but not surfaced via source_fields().
src/agentex/lib/sdk/config/agent_manifest.py	Refactored zipped() to use iter_context_files for deterministic archive ordering; now also includes dangling symlinks in the tarball (previously is_file() excluded them) — intentional and consistent with the content hash semantics.
tests/lib/test_build_provenance.py	Comprehensive 20-test suite covering clean/dirty/untracked/detached-HEAD/no-remote/non-git/monorepo-subpath scenarios plus hash determinism, one-byte/exec-bit/symlink sensitivity, and the never-raises-on-failure guard.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[capture_build_provenance\nrepo_path, context_root] --> B[_safe_working_tree_hash\nhash_root]
    B --> C{success?}
    C -->|yes| D[tree_hash = sha256 of sorted inputs]
    C -->|no, log warning| E[tree_hash = None]
    D --> F[_git rev-parse --show-toplevel]
    E --> F
    F --> G{git repo?}
    G -->|no| H[BuildProvenance\nworking_tree_hash only]
    G -->|yes| I[collect git coords\ncommit, ref, remote,\nauthor, subpath]
    I --> J[git status --porcelain\n-- subpath if monorepo]
    J --> K{output non-empty?}
    K -->|yes| L[dirty = True]
    K -->|no / fail| M[dirty = False]
    L --> N[BuildProvenance\nall fields populated]
    M --> N

    subgraph working_tree_hash
    P[iter_context_files\nrglob sorted by POSIX relpath] --> Q[for each file/symlink:\ncompute mode + content sha256]
    Q --> R[sha256 of joined lines]
    end

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[capture_build_provenance\nrepo_path, context_root] --> B[_safe_working_tree_hash\nhash_root]
    B --> C{success?}
    C -->|yes| D[tree_hash = sha256 of sorted inputs]
    C -->|no, log warning| E[tree_hash = None]
    D --> F[_git rev-parse --show-toplevel]
    E --> F
    F --> G{git repo?}
    G -->|no| H[BuildProvenance\nworking_tree_hash only]
    G -->|yes| I[collect git coords\ncommit, ref, remote,\nauthor, subpath]
    I --> J[git status --porcelain\n-- subpath if monorepo]
    J --> K{output non-empty?}
    K -->|yes| L[dirty = True]
    K -->|no / fail| M[dirty = False]
    L --> N[BuildProvenance\nall fields populated]
    M --> N

    subgraph working_tree_hash
    P[iter_context_files\nrglob sorted by POSIX relpath] --> Q[for each file/symlink:\ncompute mode + content sha256]
    Q --> R[sha256 of joined lines]
    end

Loading

Comments Outside Diff (1)

1. General comment

   Clean committed git provenance still emits working_tree_hash

   • Bug
     • The stated identity rule requires clean committed repositories to key identity on the clean commit and omit/null working_tree_hash. Runtime evidence from the head checkout shows clean_git_committed_tree returns commit 9d5b3e883d0470449c554424c322277f5b0ddaf4 with dirty=false, but also returns working_tree_hash 06ae68b1d1ef47dbe60829f2fd3bff3367e51853bea8781c5fb647f222cf85a0 in BuildProvenance, source_fields, and build_info.
   • Cause
     • capture_build_provenance computes tree_hash before checking git state and always passes working_tree_hash=tree_hash into BuildProvenance for git captures. The changed return path is anchored at src/agentex/lib/utils/build_provenance.py:224-230, specifically working_tree_hash=tree_hash on line 229; the docstring/comments also say it is always computed.
   • Fix
     • Only compute/assign working_tree_hash when there is no clean commit identity: non-git, unborn/no HEAD, or dirty work tree. For clean committed git captures, return working_tree_hash=None so source_fields/build_info omit it while keeping commit/ref/repo and dirty=false.

   _{Ran code and verified through T-Rex}

_{Reviews (6): Last reviewed commit: "Scope provenance dirtiness to build cont..." | Re-trigger Greptile}

[max-parke-scale]

Addressed both Greptile findings in cf9994d:

1. Ignored files lose hashing — fixed by removing the "skip hash on clean commit" path entirely: working_tree_hash is now always computed over the staged context, so a .gitignored-but-not-.dockerignored file is captured by the content hash regardless of git status. (Identity = the always-present hash; commit anchors it to source. Dedupe is unaffected since the hash is deterministic.)
2. Hash failures abort builds — wrapped the computation in _safe_working_tree_hash, which degrades to None and logs on any error, honoring the “capture never raises into a build” contract.

Also, per design discussion: dirtiness is now a first-class dirty flag (surfaced as source_dirty / dirty) rather than implied by hash-presence — matching Go’s vcs.modified and Nix’s dirtyRev; None outside a git work tree.

🧑‍💻🤖 — posted via Claude Code