Skip to content

FIX: Prune emptied atomic attacks on scenario resume#2210

Merged
romanlutz merged 1 commit into
microsoft:mainfrom
romanlutz:romanlutz-scenario-resume-prune
Jul 17, 2026
Merged

FIX: Prune emptied atomic attacks on scenario resume#2210
romanlutz merged 1 commit into
microsoft:mainfrom
romanlutz:romanlutz-scenario-resume-prune

Conversation

@romanlutz

Copy link
Copy Markdown
Contributor

Description

On resume, Scenario._apply_persisted_objectives restricts each atomic attack's seed groups to the persisted objective subset via keep_seed_groups_with_hashes. For scenarios that build one atomic attack per objective (for example AdaptiveScenario / TextAdaptive, where every seed group becomes its own AtomicAttack), any atomic attack whose single objective falls outside the persisted subset gets pruned down to zero seed groups. Before this fix those emptied atomic attacks stayed in self._atomic_attacks and would be scheduled with nothing to run.

This change collects only the atomic attacks that still have seed groups after pruning into a retained_attacks list and reassigns self._atomic_attacks, so emptied attacks no longer linger. Bulk scenarios (a single atomic attack holding all objectives) are unaffected, since their one attack never empties as long as at least one persisted objective survives, which is why the bug stayed latent until the per-objective (adaptive) shape hit resume.

Tests and Documentation

  • Added test_resume_discards_per_objective_attacks_outside_persisted_subset in tests/unit/scenario/core/test_scenario.py. It uses a per-objective scenario stand-in to assert that emptied atomics are dropped on resume and that every retained atomic still carries its seed groups.
  • tests/unit/scenario/core/test_scenario.py passes in full (57 passed).
  • JupyText: attempted doc/code/scenarios/3_adaptive_scenarios.py. It currently fails at cell In[3] due to a pre-existing main regression that is unrelated to this change: the flip technique sets a system-prompt seed at sequence 0 which collides with the objective's user prompt at sequence 0, raising ValidationError: Inconsistent roles found for sequence 0: {'system', 'user'}. That failure is on the fresh-run build path, not the resume-prune path touched here.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 52d2518d-ebe5-4888-8189-6146b6503448
@romanlutz
romanlutz added this pull request to the merge queue Jul 17, 2026
Merged via the queue into microsoft:main with commit f5659ce Jul 17, 2026
53 checks passed
@romanlutz
romanlutz deleted the romanlutz-scenario-resume-prune branch July 17, 2026 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants