[BUG] apm pack omits per-plugin version for INTERNAL/private github.com repos (raw.githubusercontent.com 404, no REST fallback)

[rgarciapariente]

Describe the bug
apm pack does not emit the per-plugin version field in the generated Claude marketplace.json when the marketplace packages are remote git-subdir references to a GitHub.com repository with INTERNAL (or private) visibility.

Root cause: the per-plugin version is sourced from a best-effort "metadata enrichment" fetch of each resolved package's remote apm.yml(marketplace/builder.py::_fetch_remote_metadata). For hosts classified as github.com, APM fetches it from https://raw.githubusercontent.com/.... For INTERNAL/enterprise repos, raw.githubusercontent.com returns HTTP 404 even with a valid token, while the GitHubREST /contents API serves the file correctly. APM only falls back to the REST API for GHES/GHE-Cloud hosts, never for github.com. Because the fetch fails, version is silently omitted.

This is not a permissions problem: APM resolves and sends a valid token (source=gh-auth-token), and the same 404 happens both locally and in CI.

Secondary factor: the root apm.yml marketplace entries pin a semver range (version: "^1.0.0"), which _is_display_version() deliberately rejects, so version depends exclusively on the remote fetch that is failing.

To Reproduce
Steps to reproduce the behavior:

1. Have a marketplace repo on github.com with INTERNAL (or private) visibility, whose root apm.yml declares marketplace.packages[] as remote git-subdir entries (source: <owner>/<repo> + subdir: + version: "^1.0.0"), with versioning.strategy: tag_pattern.2. Ensure the referenced packages are tagged ({name}-v{version}) so resolution succeeds, and that you are authenticated (gh auth login/ a valid token).
2. Run command: apm pack -m claude --marketplace-path claude=build/marketplace.json
3. Inspect the output: every entry under .plugins[] has name, description, and source, but no version field.
4. Run with APM_LOG_LEVEL=DEBUG and observe, for every entry:

   DEBUG apm_cli.marketplace.builder Resolved GitHub token for metadata fetch (source=gh-auth-token)
   DEBUG apm_cli.marketplace.builder Could not fetch remote metadata for <name>
       urllib.error.HTTPError: HTTP Error 404: Not Found   (builder.py:975)
   

5. Confirm the endpoint mismatch directly (replace <sha>):

   # raw CDN — fails even WITH a token
   curl -s -m 5 -o /dev/null -w "%{http_code}\n" \
     -H "Authorization: token $(gh auth token)" \
     https://raw.githubusercontent.com/<owner>/<repo>/<sha>/<subdir>/apm.yml
   # -> 404
   
   # REST contents API — works
   gh api repos/<owner>/<repo>/contents/<subdir>/apm.yml?ref=<sha> --jq .name
   # -> apm.yml

Expected behavior
apm pack should populate the per-plugin version for remote entries on INTERNAL/private github.com repositories — e.g. by falling back to the GitHub REST /contents API (which works with the token) when raw.githubusercontent.com returns 404, instead of silently droppingthe version field. Metadata enrichment should not depend on a CDN that does not serve INTERNAL-visibility repos.

Environment (please complete the following information):

• OS: Windows (MINGW64_NT-10.0)
• Python Version: 3.12.10
• APM Version: 0.20.0
• VSCode Version (if relevant): N/A

Logs

DEBUG apm_cli.marketplace.builder Resolved GitHub token for metadata fetch (source=gh-auth-token)
DEBUG apm_cli.marketplace.builder Could not fetch remote metadata for foundations-core
  File ".../apm_cli/marketplace/builder.py", line 975, in _fetch_remote_metadata
urllib.error.HTTPError: HTTP Error 404: Not Found

(identical 404 for all 12 marketplace entries)

Reproduced endpoint behavior:

• raw.githubusercontent.com/.../apm.yml (with Authorization: token): HTTP 404
• gh api repos/.../contents/.../apm.yml?ref=<sha>: returns the file (200)

Additional context

• Relevant code: marketplace/builder.py::_fetch_remote_metadata (host branch: github.com → raw CDN; else → REST API) and marketplace/output_mappers.py (_is_display_version rejects semver ranges, so version relies solely on the remote-metadata fetch).
• Affects both local runs and CI; independent of token/permissions.
• Workaround on the consumer side: post-process the generated marketplace.json to inject each entry's version read from the local leafapm.yml (the single source of truth), avoiding the remote fetch entirely.

[github-actions]

Triage decision

accept

Proposed labels

theme/portability
area/marketplace
area/distribution
type/bug
status/accepted

Milestone

null

Suggested next action

Add a REST API fallback in _fetch_remote_metadata for github.com hosts when raw.githubusercontent.com returns 404, mirroring the existing GHES/GHE-Cloud fallback path already in the code.

Suggested issue comment

Thanks for this thorough and well-evidenced report, @rgarciapariente. You have correctly identified the root cause: `_fetch_remote_metadata` in `marketplace/builder.py` uses `raw.githubusercontent.com` for github.com hosts, but that CDN returns HTTP 404 for INTERNAL and private repositories even with a valid token. The REST `/contents` API fallback is already implemented for GHES/GHE-Cloud hosts but not for github.com -- this is the gap.

This is accepted as a `type/bug` against `area/marketplace` (the pack workflow) and `area/distribution` (marketplace.json generation). The fix is to add a REST API fallback for github.com hosts in `_fetch_remote_metadata` when the raw CDN returns 404, reusing the token already resolved by `source=gh-auth-token`. This aligns with how the non-github.life-white.uk path already works.

Your endpoint mismatch confirmation (curl showing 404 from raw CDN, REST API returning 200) is exactly the evidence needed to land this fix. If you would like to contribute, the entry point is `marketplace/builder.py::_fetch_remote_metadata` -- extend the `github.com` branch to attempt the REST API when the raw CDN fails.

Thank you for including the DEBUG log output, the curl reproduction steps, and the OS/version details -- this made the root cause immediately clear.

Per-lens notes (collapsed)

DevX UX Expert -- User-Need Reviewer

Concrete failure in a real user workflow: apm pack for Claude marketplace generation silently drops version for INTERNAL/private github.com packages. The user correctly maps the failure: raw.githubusercontent.com does not serve private content, but the REST API does. The gap is that the code only falls back to REST for non-github.life-white.uk hosts. High quality report with exact log output and endpoint verification.

Supply Chain Security Expert -- Risk-Surface Reviewer

Auth surface touched: _fetch_remote_metadata resolves GitHub tokens via source=gh-auth-token. The proposed fix reuses the already-resolved token against a different endpoint (REST API instead of raw CDN); no new trust boundary is introduced. Touches area/marketplace (marketplace.json generation) and area/distribution (pack workflow). Theme is theme/portability -- accessing INTERNAL-visibility repos via the correct API channel. No theme/security or theme/governance trigger.

OSS Growth Hacker -- Contributor-Tone Reviewer

Author rgarciapariente has author_association: NONE -- first-time contributor signal. The report quality is exceptional: DEBUG log evidence, curl endpoint verification, OS/version info, and a clear workaround. Reply tone should be warm, explicitly acknowledge the depth of investigation, and point to the specific code location for a potential contribution. The author has done most of the diagnostic work already.

Python Architect -- Architecture Reviewer

Not activated -- the fix is a targeted conditional in a single method (_fetch_remote_metadata): try raw CDN, fall back to REST API on 404 for github.com hosts. No new interface or data model design required. The pattern already exists for other hosts in the same method.

Doc Writer -- Documentation Reviewer

Not activated -- this bug fix changes no user-facing command surface; at most a changelog entry noting that INTERNAL/private github.com repos are now correctly handled by apm pack.

APM CEO -- Triage Arbiter

Clear and well-documented bug in the marketplace pack pipeline. The fix mirrors existing code for other host types. Accept as type/bug under theme/portability (private/INTERNAL github.com repo access), area/marketplace + area/distribution. No active current-cycle milestone (stale 0.14.0 only; latest release is v0.20.0). Milestone set to null. Tone: warm and appreciative of the thorough investigation.

{
  "decision": "accept",
  "decision_detail": "",
  "theme": "theme/portability",
  "areas": ["area/marketplace", "area/distribution"],
  "type": "type/bug",
  "status": "status/accepted",
  "priority": null,
  "preserved_labels": [],
  "milestone": null,
  "next_action": "Add a REST API fallback in _fetch_remote_metadata for github.com hosts when raw.githubusercontent.com returns 404, mirroring the existing GHES/GHE-Cloud fallback path.",
  "comment_markdown": "Thanks for this thorough and well-evidenced report, @rgarciapariente. You have correctly identified the root cause: _fetch_remote_metadata uses raw.githubusercontent.com for github.com hosts, but that CDN returns HTTP 404 for INTERNAL and private repositories even with a valid token. The REST /contents API fallback is already implemented for GHES/GHE-Cloud hosts but not for github.com -- this is the gap.\n\nThis is accepted as a type/bug against area/marketplace and area/distribution. The fix is to add a REST API fallback for github.com hosts when the raw CDN returns 404, reusing the token already resolved. This aligns with how the non-github.life-white.uk path already works.\n\nIf you would like to contribute, the entry point is marketplace/builder.py::_fetch_remote_metadata -- extend the github.com branch to attempt the REST API when the raw CDN fails.\n\nThank you for including the DEBUG log output, the curl reproduction steps, and the OS/version details."
}

---

Triage status: agentic proposal pending human ratification.
Silence is approval. Maintainers can:

• Override any label or milestone above by editing it directly --
  human edits are authoritative and will not be reverted on
  subsequent runs.
• Re-trigger triage by applying the status/needs-triage label, or
  by removing status/triaged to enroll the issue in the next
  daily sweep.

Posted by the Triage Panel workflow. See .apm/skills/apm-triage-panel for the panel skill.

Generated by Triage Panel · sonnet46 3.5M · ◷