Skip to content

Add audit-logs download command#195

Open
yummybomb wants to merge 17 commits into
mainfrom
hypeship/audit-logs-download
Open

Add audit-logs download command#195
yummybomb wants to merge 17 commits into
mainfrom
hypeship/audit-logs-download

Conversation

@yummybomb

@yummybomb yummybomb commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds kernel audit-logs download for verified chunked audit-log exports.

  • Requires an explicit, exclusive [start, end) range of at most 30 days.
  • Downloads gzip-compressed JSONL with no alternate format modes.
  • Walks the API cursor sequentially and verifies every source chunk checksum before processing it.
  • Matches audit-logs search filter semantics, including stacking the default GET exclusion with another --exclude-method.
  • Writes to an adjacent .partial file and publishes the final .jsonl.gz only after all chunks complete.
  • Removes incomplete partial output after transfer failures and preserves a completed partial if final publication fails.
  • Refuses to replace an existing output unless --force is passed; forced replacement happens only during the final rename.

Rollout

Before merge, replace the temporary client-side stacked-exclusion filter with the native multi-value API after kernel/kernel#2706 is deployed and the corresponding Go SDK release is available.

Test plan

  • go test ./...
  • go vet ./...
  • Race-enabled audit-log tests
  • Windows amd64 test-binary cross-compile
  • Stacked method exclusions match search output semantics
  • Finalization failures preserve completed partial and existing output files
  • Live smoke after simplifying failure cleanup

yummybomb and others added 10 commits July 9, 2026 14:26
Wraps GET /audit-logs/export/chunk (kernel-go-sdk v0.75.0). Chunks are
verified against X-Content-Sha256 before being appended to the output
file, and a sidecar state file persists the cursor and committed byte
offset so an interrupted download resumes without duplicating rows.
Ranges over the API's 30-day cap fail fast with ready-to-run per-window
commands.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
--include-get lifts the exclusion. Because chunks are appended verbatim,
a second exclusion can't stack client-side the way search filters, so
--exclude-method with the GET default active is rejected with guidance
rather than silently replacing it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Bind resume state to a credential identity (hashed API key or OAuth
  org ID) so switching credentials can't mix organizations in one archive
- Validate the output file against recorded progress on resume; a missing
  or shortened file now fails instead of being zero-extended and reported
  complete
- Fail closed on missing X-Has-More and X-Content-Sha256 headers, and
  validate pagination headers before any bytes are written
- Commit initial state before the first fetch so an early failure retries
  without --force; --force removes stale state immediately
- Write state via os.CreateTemp and use 0600/0700 modes for outputs
- Cap buffered chunk responses at 256 MiB
- Fix stale -o references in errors; document that resume needs explicit
  --start/--end

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@yummybomb yummybomb marked this pull request as ready for review July 10, 2026 15:12
yummybomb and others added 2 commits July 10, 2026 15:14
The server controls chunk sizing and every chunk is checksum-verified,
so the client-side byte ceiling only added a failure mode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The params fingerprint already invalidates stale state files, and the
state is too short-lived to need format versioning.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment thread cmd/audit_logs_download.go
Comment thread cmd/audit_logs_download.go
Comment thread cmd/audit_logs_download.go Outdated
@yummybomb yummybomb changed the title Add audit-logs download command with resumable chunked export Add audit-logs download command Jul 10, 2026
Comment thread cmd/audit_logs_download.go

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ee8ccd4. Configure here.

Comment thread cmd/audit_logs_download.go Outdated
if err := os.Rename(partialPath, outPath); err != nil {
return fmt.Errorf("finalize %s: %w", outPath, err)
}
return nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Force overwrite fails Windows

Medium Severity

With --force, a completed download is finalized via os.Rename onto an existing output path. On Windows, rename cannot replace an existing file, so overwrite fails even though the command accepted --force at the start and wrote a full .partial file.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit ee8ccd4. Configure here.

if got := hex.EncodeToString(sum[:]); got != want {
return nil, nil, fmt.Errorf("chunk checksum mismatch (got %s, want %s)", got, want)
}
return body, res.Header, nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chunk fetch ignores HTTP status

Medium Severity

fetchAuditLogsChunk treats a nil SDK error as success and always reads the body and verifies X-Content-Sha256, without checking res.StatusCode. A non-2xx response that still returns a response object could be checksummed and appended into the export.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit ee8ccd4. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant