Repository navigation
Description
Symptoms
After running amplifier update, two of four configured providers (openai, gemini) silently failed to load. The only visible signal was:
Partial provider failure: 2/4 loaded. Missing: {'openai': 1, 'gemini': 1}.
Loaded: ['anthropic', 'github-copilot']. Session continuing with available providers
(self-healing NOT triggered for partial failure).
The actual exception (ModuleNotFoundError: No module named 'amplifier_module_provider_openai') is swallowed before reaching the user-visible log. Self-healing did not trigger because the failure was classified as "partial."
The same amplifier update run also produced this clone error at the top, which appears to be correlated:
✗ Bundle: joi: Failed to clone ssh://git@github.com/Joi/amplifier-bundle-joi@main:
fatal: could not open '/Users/joi/.amplifier/cache/amplifier-bundle-joi-<hash>/.git/objects/pack/tmp_pack_dnAj5l' for reading: No such file or directory
fatal: fetch-pack: invalid index-pack output
amplifier update ran git clone into a populated cache dir instead of git pull, failed mid-pack, but left the bundle in its previously healthy state.
Root cause
amplifier update re-clones modules into content-hash-suffixed cache directories when upstream changes. For openai+gemini on the affected run, the re-clone produced new hash dirs, but the editable install (uv pip install -e) was not re-run against the new paths. The old hash dirs were subsequently removed by the updater's GC step, leaving the editable install pointers dangling.
Observed state on the affected machine (uv tool install-managed amplifier venv at ~/.local/share/uv/tools/amplifier/):
| Module | .pth and .dist-info/direct_url.json pointed to |
Cache currently contained |
|---|---|---|
| openai | cache/amplifier-module-provider-openai-e2c4b8c10222ed8c (deleted) |
…-f62df101b02f7b3e |
| gemini | cache/amplifier-module-provider-gemini-06d1437d03d6b064 (deleted) |
…-055b52ecbb4ac482 |
| anthropic (control, working) | …-5181591dcf06d076 |
…-5181591dcf06d076 ✓ |
import amplifier_module_provider_openai → ModuleNotFoundError because the .pth pointed to a non-existent directory. Anthropic and github-copilot worked because their content hashes hadn't changed and their install pointers were still consistent.
The "tmp_pack_dnAj5l" clone error at the top of the update run is likely related: the bundle clone failure may have caused the updater to abort its post-clone editable-install consistency check, leaving modules whose hash DID change in an inconsistent state.
Surgical fix that worked
Without touching the cache directories:
AMP_PYTHON=~/.local/share/uv/tools/amplifier/bin/python
uv pip install --python "$AMP_PYTHON" -e ~/.amplifier/cache/amplifier-module-provider-openai-<current-hash>
uv pip install --python "$AMP_PYTHON" -e ~/.amplifier/cache/amplifier-module-provider-gemini-<current-hash>uv pip install cleanly reported the swap from the dead path to the live one:
- amplifier-module-provider-openai==1.0.0 (from file:///…/openai-e2c4b8c10222ed8c)
+ amplifier-module-provider-openai==1.0.0 (from file:///…/openai-f62df101b02f7b3e)
After this, Partial provider failure disappeared and all 4 providers loaded.
The heavier alternative amplifier reset --remove cache -y && amplifier update would also have worked but re-clones everything.
Diagnostic indicators (for other users hit by this)
amplifiersession start printsPartial provider failure: …with{'<provider>': N, ...}<venv>/lib/python3.*/site-packages/_editable_impl_amplifier_module_provider_*.pthpoints at a~/.amplifier/cache/...directory that no longer exists on disk- The matching
*.dist-info/direct_url.jsonconfirms the stale URL
Suggested fixes
- Surface the actual exception in the partial-failure message. Today it just shows a count (
{'openai': 1, 'gemini': 1}) — include the underlyingImportError/ModuleNotFoundErrorso users can diagnose without dropping into the venv. - Treat editable-install consistency as a post-update invariant. After cloning modules to new hash dirs, verify (via
direct_url.jsonor equivalent) that every editable install points at a path that still exists; reinstall against the current path if not. Sub-millisecond check. - Prefer
git pullovergit clonewhen the cache dir already contains a valid git repo with a matching origin URL. Thetmp_pack_dnAj5lerror would not have happened, and the partial pipeline failure would not have left consistency gaps downstream. - Don't treat "partial provider failure" as silently OK. It's the worst kind of failure for a system that routes by role — a session configured with
model_role: critical-opsagainstopenaiwill silently downgrade to anthropic with no signal to the calling agent. Consider non-zero exit and/or hard-fail by default with an explicit--allow-partial-providersflag if quiet downgrade is desired.
Environment
- macOS 26.5 arm64 (M-series)
amplifierinstalled viauv tool install amplifier- Python 3.12 (uv-managed)
- Bundle: a private bundle composed against
microsoft/amplifier-foundation+microsoft/amplifier-bundle-routing-matrix - All 4 provider modules sourced from
git+https://github.com/microsoft/amplifier-module-provider-*@main
Happy to attach pip list, the affected .pth contents, or the cache directory listing if useful.
Activity
kenotron-ms commentedon May 27, 2026
Action Plan: Verification Infrastructure + Fix
What We're Proposing
Before landing the ~10 line fix to
update_module(), we're building verification infrastructure that proves the install/update pipeline works across all scenarios. The fix is straightforward -- addModuleActivator._install_dependencies()afterresolver.resolve()inmodule_cache.py, mirroring whatupdate_bundle()already does correctly. The harder problem is: how do we prove it works and gate future changes?The approach: extend amplifier-tester's existing validator with a new
install-updatecheck type and create 3 DTU profiles that exercise the full install/update lifecycle.Scenarios Covered
Normal User (install -> use -> update -> use):
amplifier reset --remove cache-> clean slate)Developer (source overrides + update):
amplifier source add provider-X ~/dev/Xamplifier source add provider-X git+https://fork@branchamplifier update --forceinside container)Regression checks (scenarios that work today, must not break):
Newly verified (works today but never tested):
amplifier --version, never starts a session)Assertion Layer
Extending the amplifier-tester validator with baseline checks that run after every lifecycle phase:
PTH_INTEGRITY: Every .pth in site-packages pointing at ~/.amplifier/cache/ references a directory that existsIMPORT_SMOKE:python -c "import amplifier_module_provider_<X>"succeeds for every configured providerSTATE_CONSISTENT: Every entry in install-state.json has a path key that exists on diskSESSION_VIABLE:amplifier run "Say exactly: install-verify-ok"completes successfullyPlus phase-specific assertions for post-install, post-update, and post-override phases.
DTU Profiles
Three profiles, one per persona track:
amplifier-install-update-lifecycle.yaml(Normal User) -- Fresh install, first-run setup, then simulated hash changes via cache dir deletion to reproduce the bug deterministically. Covers U1-U7.amplifier-source-override-lifecycle.yaml(Module Developer) -- Install + configure, then exercise both local path and git fork overrides through update cycles. Covers D1-D5.amplifier-dev-ecosystem-lifecycle.yaml(Ecosystem Developer) -- Install via amplifier-dev bundle with Gitea URL rewrites, then update and verify session viability (the gap in the current self-validation profile). Covers D6.Proposed Sequence
amplifier-app-cli/utils/module_cache.py-- U4, U5, D5 flip greenThe Fix (for reference)
After
resolver.resolve(uri)returns the new path inupdate_module()(~line 514):Mirrors
update_bundle()atamplifier-foundation/updates/__init__.py:260-268. Closes the gap for all module types in one change.Joi -- does this cover what you'd expect? Interested in your read on whether the three persona tracks capture all the scenarios you care about.
Joi commentedon Jul 5, 2026
Still present on amplifier-app-cli main @ ba1957266b8dc72c16b194eb46727caf5b7cbc0d.
amplifier updateroutes modules throughexecute_updates→execute_selective_module_update→update_module(amplifier_app_cli/utils/module_cache.py:468-521), which callsclear_module_cache()at :502 andSimpleSourceResolver.resolve()at :514 and returns — the editable install is never refreshed, so the venv.pthkeeps pointing at the deleted hash dir. The bundle path already does this correctly:update_bundle()re-runsModuleActivator._install_dependencies()on every updated module (amplifier-foundationamplifier_foundation/updates/__init__.py:261-268@ 9ec8dee9). The fix described in the 2026-05-27 action plan has not landed. I closed #298 as a duplicate of this issue.Joi commentedon Jul 29, 2026
Update. Two PRs merged on amplifier-app-cli yesterday (2026-07-28) change the picture, but do not close this.
microsoft/amplifier-app-cli#238 (ad69c69) aligned install-time provider source resolution with runtime resolution. microsoft/amplifier-app-cli#239 (145ba2ef) made the first-run repair path repair all configured provider modules rather than only the active one. #239's problem statement is exactly the symptom reported here — "After
amplifier updateoramplifier reset, only one provider module ends up installed. Every other configured provider silently fails to mount." So the user-visible failure should now self-heal on the next boot viacheck_first_run().The root cause in this issue is unchanged. On current main,
update_module()inamplifier_app_cli/utils/module_cache.pystill ends:There is no
_install_dependencies()call or other editable-install refresh after resolve, and no reference toModuleActivatoranywhere in that file. Soamplifier updatestill leaves the venv.pthpointing at the deleted hash dir — we now depend on a repair pass in the next session rather than the update itself leaving a consistent state.update_bundle()in amplifier-foundation still handles this correctly inline.Suggest keeping this open scoped to the update-path fix from the 2026-05-27 action plan (the ~10 line
_install_dependencies()call after resolve), since the new repair path is a safety net rather than the fix.