Skip to content

Census 2026-11-04 - #1002

Merged
bong-water-water-bong merged 2 commits into
mainfrom
census/2026-11-04
Nov 4, 2026
Merged

bong-water-water-bong merged 2 commits into
mainfrom
census/2026-11-04

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

433638 text-generation models on HF, 343006 with an architecture (2990 architectures).

models share
mapped (a backend's code accepts the architecture) 323929 94.44%
checked (passed serve_e2e on Strix Halo) 218517 63.71%
backend mapped checked
hrx 94.44% 63.71%
npu 53.61% 9.42%

Unmapped architectures with the most models:

  • Step1MoEForCausalLM: 2882
  • ParlerTTSForConditionalGeneration: 1585
  • ChessForCausalLM: 970
  • LlavaLlamaForCausalLM: 453
  • DynamicAlibiForCausalLM: 245
  • OpenLMForCausalLM: 245
  • DynamicForgettingForCausalLM: 240
  • BertLMHeadModel: 212
  • MobileLLMForCausalLM: 176
  • BartForConditionalGeneration: 174

…rchitectures): mapped 323929 (94.44%), checked 218517 (63.71%)
@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit 876f0c4)

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Data Consistency

The census data includes new model counts and architecture mappings that may not be fully validated against actual hardware performance or backend compatibility. The increase in mapped models and checked models for both hrx and npu backends should be verified against real-world testing on Strix Halo and iGPU to ensure claims of 94.44% mapped and 63.71% checked accuracy are accurate.

{
 "date": "2026-11-04",
 "coverage": {
  "models": 433638,
  "with_architecture": 343006,
  "architectures_seen": 2990,
  "mapped_models": 323929,
  "mapped_pct": 94.44,
  "checked_models": 218517,
  "checked_pct": 63.71,
  "per_backend": {
   "hrx": {
    "mapped": 323929,
    "checked": 218517,
    "mapped_pct": 94.44,
    "checked_pct": 63.71
   },
   "npu": {
    "mapped": 183889,
    "checked": 32322,
    "mapped_pct": 53.61,
    "checked_pct": 9.42
   }
  },
  "top_unmapped": [
Architecture Count Discrepancy

The number of architectures seen increased from 2980 to 2990, but the top unmapped architectures list shows some architectures with higher model counts than previously reported. This could indicate inconsistencies in how architectures are counted or mapped, particularly for architectures like Step1MoEForCausalLM which has 2882 models.

   "architecture": "MobileLLMForCausalLM",
   "models": 176
  },
  {
   "architecture": "BartForConditionalGeneration",
   "models": 174
  },
  {
   "architecture": "MixFormerSequentialForCausalLM",
   "models": 156
  },
  {
   "architecture": "K2HorizonForCausalLM",
   "models": 145
  },
  {
   "architecture": "RobertaForCausalLM",
   "models": 139
  },
  {
   "architecture": "Spark2_5ForCausalLM",
   "models": 135
  },
  {
   "architecture": "NanochatGPTForCausalLM",
   "models": 126
  },
  {
   "architecture": "OpensciForCausalLM",
   "models": 120
  },
  {
   "architecture": "Glm5NextForConditionalGeneration",
   "models": 112
  },
  {
   "architecture": "MBartForConditionalGeneration",
   "models": 112
  },
  {
   "architecture": "BartForCausalLM",
   "models": 100
  },
  {
   "architecture": "LlavaQwen2ForCausalLM",
   "models": 97
  },
  {
   "architecture": "Lfm3ForCausalLM",
   "models": 89
  },
  {
   "architecture": "BioGptForCausalLM",
   "models": 86
  },
  {
   "architecture": "RwkvForCausalLM",
   "models": 86
  },
  {
   "architecture": "GPT2LMHeadCustomModel",
   "models": 84
  },
  {
   "architecture": "BailingMoeV3ForCausalLM",
   "models": 80
  }
 ]
},

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY

Data-only census bump; I re-derived every number in it from the file itself rather than from the body.

registry/census.json at b929d6109 parses, and:

  • models - raw.no_arch == with_architecture: 433638 - 90632 = 343006.
  • sum(raw.counts) == 343006, so the per-architecture table accounts for every model with an architecture — no rows
    silently dropped.
  • architectures_seen 2990 == len(raw.counts) 2990.
  • every stored percentage recomputes: mapped 323929/343006 = 94.44, checked 218517/343006 = 63.71,
    npu mapped 183889/343006 = 53.61, npu checked 32322/343006 = 9.42 — all four match to the digit.
  • raw.counts is sorted descending throughout (the Lfm3ForCausalLM move 82 -> 89 is a real reorder, not a
    misplacement).
  • the body's unmapped table matches raw.counts name for name: Step1MoE 2882, ParlerTTS 1585, Chess 970,
    LlavaLlama 453, DynamicAlibi 245, OpenLM 245, DynamicForgetting 240, BertLMHeadModel 212, MobileLLM 176, Bart 174.

tests/census_watch_test.py passes with this file in place (it reads registry/census.json): all checks passed,
including the two checks that assert the census still records the engine#619 class and that the granite switch is not
a one-off — so nothing here moves the case study the census exists for.

CI on the head: build and the four Analyze jobs green. Day-over-day deltas are all small and in the same direction
as the rest (Llama 101445 -> 101572, Qwen2 47626 -> 47661), consistent with 481 new models.

@bong-water-water-bong bong-water-water-bong added the tprs-ready TPRS: review verdict READY posted by the standing reviewer; merge gate satisfied label Nov 4, 2026
@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Persistent review updated to latest commit 876f0c4

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY (re-verdicted at 876f0c4e6; the census file is unchanged)

git diff b929d6109 876f0c4e6 -- registry/census.json is empty — the head moved by main merges only (tests/,
third_party/llama.cpp), so every number I re-derived at b929d6109 stands: sum(raw.counts) = 343006 = with_architecture, architectures_seen = len(counts) = 2990, and all four percentages exact.

@bong-water-water-bong
bong-water-water-bong merged commit a7e5e81 into main Nov 4, 2026
9 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the census/2026-11-04 branch November 4, 2026 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Review effort 2/5 tprs-ready TPRS: review verdict READY posted by the standing reviewer; merge gate satisfied

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant