Skip to content

External-source idempotency conflicts storm for ~40 s after a scoped observation reset #1130

Description

@ScriptedAlchemy

Observed

Surfaced while root-causing observation_reset_recovery_test on #707 (tip 15ee56ae5). After the scoped observation-authority reset and daemon reopen, the daemon log shows one

observation repository provenance collision (authority_write_failed)

followed by

registered external-source commit failed error=external source idempotency key conflicts with a prior command (external_source_commit_failed, retryable)

every ~1.5 s for ~40 s before the retained temporal authority converges. With the test's 24-rollout fixture the first describe after reopen takes 30–42 s; a 240-rollout trial took ~6 min. The retry storm dominates recovery time.

Suspected cause

reset.rs clears the authority/projection tables, anchors and cursors (same class as 73072ebe5 / c4c025fd5) but the external-source receipt journal survives the scoped reset, so the replaced-identity transcript conflicts with prior idempotency keys on re-admission and is retried on a fixed cadence until the journal window passes.

Expected

Re-admission after a scoped reset should either scope the idempotency journal to the reset (clear it with the authority it describes) or recognise the replaced identity as a supersession rather than a conflict. Fix the cost, not the retry cadence or the test fixture (AGENTS.md: limits are symptoms).

Repro

cargo nextest run -p tracedecay-cli --test core_cli_suite -E 'test(observation_authority_reset_recovers_the_retained_temporal_authority)' with daemon logs captured; the test passes but its ~60 s runtime is this storm.

Related: converging-state mapping converging_projection_unavailable (e4760ea22) is no longer exercised through the CLI since 40c78417d rides out after-delay refusals; a deterministic session-runtime-level test is the right home for it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions