Skip to content

[Frontend] Replace production frontend with transactional ACIR for #361 - #368

Closed
hmljy2020 wants to merge 46 commits into
mainfrom
issue361
Closed

hmljy2020 wants to merge 46 commits into
mainfrom
issue361

Conversation

@hmljy2020

@hmljy2020 hmljy2020 commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

本 PR 将生产前端替换为 issue #361 的事务级 ACIR:Python capture → Source → Linked → 公共事务分析 → Routed → 物理存储 lowering → C++/Verilog。Module 管结构、状态和组合计算,Rule 管原子事务,Method 提供同拍调用接口;普通 Rule 显式注册,状态读取遵循 old Q。

当前提交:e3355f02367f14d61bdf97ccc4fe289e6bcb7067。替换尚未完成,完整 API 验收仍有失败,PR 保持开放,不声明可合并。

本次相对远程原 head c36a973a94222c7c4cc64fdebf95b31b2fbc2741 新增一个提交,80 文件,+2,494/-1,826 行,净增 668 行;其中包含物理 lowering 搬移、文档与迁移清单。ROB 按用户要求暂缓进一步修复,保留源码、完整历史和独立 oracle,不计入通过集合。

Change type

  • Bug fix
  • Feature / behavior change
  • Performance
  • Refactoring / cleanup
  • Tests / CI
  • Documentation / configuration

Related issues

Related to #361、#319,不关闭这些 issue。语义依据为 issue #361、已批准的 Struct/Bits 部分更新契约和仓库 transactional replacement plan。#363–#367 是已合入 merge base 的集成输入。#322 主线数值能力的完整迁移仍待完成。

Changes

  • Capture / common IR:此前已实现 typed pure helpers、Struct 默认构造、局部字段/Bits 静态切片更新、事务 State 部分写、静态 for、固定组件 Table。保持独立源码编译、接口/实现闭包、实例身份与原子性。本批修复局部字面量 SSA 约束、纯 Struct/Enum 条件选择和 checked Table 索引证明,保留名义类型与四态 payload。
  • 公共分析与 Link:缓存实际绑定上下文的 dependency paths,利用真实 SSA use-list 索引重放二元操作;保留检查、真实费用计量和原预算,不按示例特判。保存完整 Linked 的 bytecode provenance v2,拒绝旧版本并要求重编译。artifact parser 使用稳定源标识,保留显式属性和 location,修复独立 CLI 链接的非确定性。
  • 物理 lowering / 旧路径退出:提取 HardwareStateLowering.cpp/.h,保留独立物理存储一致性验证;公共 router 拒绝旧源事务,删除旧 transaction-state pass 及两个旧分析 selector。旧 private materializer、Method-summary/shared verifier 仍在库中,属于明确未完成的清理工作,不能声称旧实现全部退出。
  • Examples / gates:catalog 独立记录迁移状态和执行 tier,31 个原设计均有去向;当前 13 项 gate、另 2 项 nightly,16 项待迁移或暂缓。CMake 直接消费 catalog,不因目录存在就宣称支持。sibling_atomic/aliases 完成迁移;transitive、StructTransform、filter、picker、ROB 保留部分迁移及完整独立历史,不冒充完整验收。multi_outputs 原设计保留。
  • 测试清理:保留底层 refinement 和共享 verifier。删除两项旧候选/IR 形状 lit、八项已取消语义 native cases,以及 work-partition 专属旧源生成器;work-partition 用 verified physical IR 保留原三项独立 oracle。删除项不计为通过,覆盖变化记录在 main-capability-migration.md。
  • Package / docs:安装依赖 relocation 尊重配置的 lib/lib64 路径;更新保存产物契约、能力清单、执行 tier 与各未迁移设计阻塞说明。两个物理后端继续消费同一验证后的 Final。本轮不扩展 t-gfsim。

Validation

工具链从本 checkout 构建,pyc6 / Clang 22 / LLVM-MLIR 22.1.8。以下严格区分当前提交、较早内容候选和未执行项。

内容限制:功能证据来自 c36a973a 加本次任务内容的共享工作区;两处原有修改 PythonImportExpressions.inc、transactional-table-data.py 逐字节保留且未提交,另有未登记的 examples/ppt_pipeline/ 未提交。因此本地结果不能冒充远程 e3355f02 的隔离干净 checkout 验收。远程 CI 必须验证实际 head。

状态 实际执行与结果 候选范围 / 限制
PASS / exit 0 干净目录构建并安装 compiler/runtime;安装目录再次更新 dependency/replay cache 本 checkout;不是另一个 worktree 的工具链
PASS / exit 0 bash tools/run_examples.sh --tier nightly,15/15,79.36s 确定性 loader 修复后的内容,早于最后 dependency/replay cache 优化;包含 gate 13 项,未把 pending 项当 skip/pass
PASS / exit 0 物理 refinement 5/5;work-partition 3/3;dependency native 6/6;ACIRTransactionalGfsimTests 1/1 聚焦执行,不是全量 API PASS
PASS / exit 0 table、structured-packet、constraints、linked、state-interfaces 等受影响 lit;C++ workers 1/2、RTL;四态选择明确使用 Icarus 同一个 Final 驱动两个后端,独立预期;各组按其修复后的内容执行
PASS / exit 0 saved-anchor escaped-owner 与 Runtime-only/CompilerDev 安装消费测试,3/3,17.02s loader 修复后的安装;Runtime-only 不需要 LLVM
PASS / exit 0 full picker 原 344 周期加 fallback 345 行,Linked → verified Final,9.02s dependency cache 修复后;不代表完整后端执行
PASS / exit 0 python -m pytest tests/unit/test_example_catalog.py -q,20/20 最终提交内容,含 ROB deferred 状态
PASS / exit 0 全部 75 个新增/修改现存文件逐项 SKIP=pyc-api-hygiene BLACK_NUM_WORKERS=1 pre-commit run --files <file>;mkdocs build --strict;git diff --cached --check 最终提交内容;保留 CI fixture 排除规则,API gate 单独报告
FAIL / exit 8 bash tools/run_api_tests.sh --tier gate,26 个 CTest targets 中 18 PASS、8 FAIL 中间集成候选;后续局部修复不等于重跑全量通过
FAIL 原完整 ROB System 已清除事务 routing 预算阻塞,但 physical storage correspondence proof 超过独立 1 MiB 预算 用户于 2026-11-01 暂缓继续修复;284 行 System 与 569 Work oracle 均保留,无预算增加或历史缩减
FAIL / 部分 PASS digital_filter 冷构建超时;预编译后的 C++ workers 1/2 和 RTL 输出一致,但当时验收受旧 provenance 非确定性阻断 不将 warm 运行提升为最终完整验收
NOT RUN 最后 cache 优化后完整 example/API gate、picker 完整 C++/RTL、ROB 执行、全量 API nightly、长 oracle/mutation/coverage、完整跨平台和发布矩阵 仍需候选绑定验收;ROB 明确暂缓

API gate 的 8 个失败 target:ACIRAnchorArtifactTests、ACIRSourceInterfacePassTests、ACIRRecordProjectionSimplificationTests、ACIRQueueContractsTests、ACIRTransactionEffectsTests、ACIRTableDependencySummaryTests、ACIREnumSourceUnitAuthorityTests、ACIRLitGateTests。混合旧测试需逐项迁移,不能整文件删除或继续缩小成功集合来掩盖失败。此前 transactional-constraints 的 payload 约束失败已有 owning compiler 修复及聚焦 PASS,但未宣称全部上述 target 已通过。

原始日志、JUnit 和独立审查在本地忽略目录 docs/gates/logs/20261101-example-migration/、20261101-example-migration-accepted/、dependency-cache-review/、replay-binary-index-review/ 等。未上传远程 artifact;本说明提供可远程阅读的结果摘要。更早 c36a973a 的精简/执行计数仅属于历史候选,不沿用为当前 head PASS。

  • Executed checks and outcomes are recorded above
  • Skipped, deferred or unavailable checks and reasons are recorded above

Review checklist

  • Scope and affected contracts are identified; new proposals are distinguished from approved behavior
  • Both backends preserve common-IR semantics in the tested bounded scope
  • Active documentation and remaining callers are updated where needed
  • No generated build output, temporary evidence logs or secrets are committed
  • Tests cover all required migrated capabilities
  • Required CI and independent reviews are complete for the full replacement candidate

Reviewer notes

review 5702393068 的具体 P1(ordinary typed helper 被唯一 importer 拒绝)已通过 owning importer、authenticated closure、共同 preparation 和跨文件双后端测试修复;要求兼容能力和完整 gate 完成前保持 PR 开放的总体意见仍 OPEN,不声明整个 review 已解决。

本轮源码迁移与 authoring 独立审查由独立 gpt-6-astra 实例执行;compiler、独立测试和实现交叉审查由独立 Codex 实例分工。物理拆分、dependency cache、replay index、名义类型选择、bytecode/parser 保存属性修复及 libdir 修复有有界独立复核;不是最终整个 PR 的 clear。先前曾尝试的有损 SCF text 重解析已撤回,最终 loader 保留显式属性/location。

剩余工作包括旧 private APIs 与混合 native tests 清理、Memory、旧 Queue 同拍容量借用、多时钟、原物理 root/same-Work 导出,以及其他未完成设计。源资产和独立 oracle 保留,known semantics 的实现缺口不宣称为已取消能力。原先提前集中删除旧实现的实施偏差仍记录在能力清单;本轮局部通过不消除该义务。

本次只快进更新现有 PR 的代码和描述,不合并、不关闭 issue、不改变 assignment/project 或评审线程状态。

Capture PR-07 resource interfaces, SSA and Method constraints, PR-08 Queue execution, and PR-09 composition examples with their tests and design records. Include native transactional CI coverage and the current common-analysis refactor.
Persist formal read identity and authored payload evaluation through source
publication and branch joins. Resolve reached-path consumption in common
analysis, remove cached reads, and require source contract 4 artifacts.

Share callable return normalization and representation constructors, use a
common read-only constant proof, and replace repeated dependency scans with
stable ready processing. Extend independent resource and return histories.

Validation: affected C++ workers 1/2, Verilog, four-state and supported t-gfsim
regressions pass; independent review, changed-file pre-commit and strict docs
build pass. Full API/examples gates retain baseline-reproduced failures;
nightly not run. Controlled routing measurement showed no speedup.
Integrate main 3d37651 without restoring deleted importers or superseded v2 boundary fixtures. Retain evolved normalized contracts, Linked continuation, common aggregate availability, Queue proposals and Bits width checks. Static-parameter Method checks remain in retained common legacy consumers, not a frontend support claim.

Validation: rebuilt current compiler; baseline 13 transactional/display cases and native runtime passed. Changed-file pre-commit and strict docs passed. Initial integrated run identifies void direct-call reference gap, addressed next. Full API and examples gates fail on retained migration inputs; no nightly claim. Local integration only.
Void Module calls now bind a non-payload reference to their actual allocation, preserving aliases and distinct children without dummy ports. Migrate void coverage to interface-only source publication, saved Linked continuation and independent old-Q histories on C++ workers 1/2, RTL and t-gfsim. Replace redundant legacy IR-shape and rejection matrices with four public zero-payload export cases.

Validation: 14 transactional/display lit tests pass; four pure void export cases pass; changed-file Ruff/Black/pre-commit pass. Pure roots retain compiler-owned clock/reset inputs; the previous no-domain expectation is not claimed. Broader legacy gates and static/Table migrations remain open.
Build generated-System examples through their emitted CMake project instead of inventing a translation unit per Python source. Preserve declaration-only and Rule-only units. Document migrated void behavior, retained common fixes, removed coverage and pending static-parameter/Table capabilities.

Validation: 10 current System example targets build and execute successfully; final 14 transactional/display lit tests, runtime/hardware checks, focused aggregate checks, pre-commit and strict docs pass. Complete API/example gates still FAIL on retained legacy migration inputs; five static Method cases and full Table pool remain FAIL at admission. Full nightly NOT RUN. Independent implementation review clear; evidence docs/gates/logs/20261029-issue361-main-integration/.
…ations

Check transactional body assertions on every Work evaluation with ordinary reset masking in physical backends and t-gfsim. Preserve independent low-epoch failure and rollback histories; add normalized resource anchor and interface publication checks and a named main-capability migration inventory. Retire only the obsolete legacy summary-forgery matrix.

Validation: 14 transactional/display lit PASS; 15 current saved-anchor checks PASS; two normalized native anchor checks PASS. Changed-file pre-commit and strict MkDocs PASS. Full API and examples gates retain documented migration failures; remaining legacy native failures and original nominal/report oracle remain open. Related to #361.
Publish symbolic Module/System parameters and defer payload/Method constraints until Link binds actual instances using shared static evaluation. Preserve independent state, dependent defaults, root overrides, static operands and Queue depth through saved Linked continuation; validate unused/default-bound bodies and retain source provenance.

Validation: final12 parameter/Method cases PASS across C++ workers1/2 and RTL;14 transactional/display lit PASS;15 saved-anchor regressions PASS; full resource_composition example PASS. Native Routed/runtime PASS; retained native Anchor15/interface148 failures and full gate migration failures remain explicit. Ruff/Black and strict docs PASS. Independent compiler and Astra authoring/oracle review clear. Related to #361; depends on9b398d0d.

@zhoubot zhoubot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The staged transactional path has useful passing evidence, but this candidate is not ready to replace main. Please restore compatible capabilities before merge; the requested acceptance boundary is to keep this PR open until that work is complete.

P1: compiler/lib/Compiler/PythonImportRecords.cpp:723–728 rejects ordinary pure helpers on the sole production frontend. A minimal typed def twice(value: ac.u8) -> ac.u8: return value + value, called from an @ac.module, compiles with PR #322/main's frontend but fails here with transactional source functions require module/system/rule decoration. This also blocks the helper-based numerical providers in #322. Ordinary helpers are retained migration obligations in main-capability-migration.md batch 3, not cancelled semantics. Restore them through the owning importer/common analysis with source-unit/header authority and independent native/RTL evidence; do not add a fallback compiler or remove coverage.

Fresh local build from this exact head with LLVM/MLIR 22.1.8: native transaction Runtime CTest 1/1 PASS; all 14 focused transactional/display lit tests PASS; independent Python boundary checks 105 PASS. Separate physical/runtime review found no additional actionable issue in its inspected scope. However bash tools/run_api_tests.sh --tier gate exits 1 while compiling retained module_loop, and bash tools/run_examples.sh --tier gate exits 1 at multi_outputs (Module payload outputs require a Rule result). These failures preserve the original registry. Full nightly and cross-platform acceptance were NOT RUN. Hosted G0/G1 success does not resolve these full-gate failures.

Independent review reports were produced by separate gpt-6.1-sol instances for core and physical/runtime paths; local raw commands/results are under docs/gates/logs/pr-review-20261030/. No whole-migration or merge-ready claim.

Capture static range loops as Source SCF and expand occurrence-private bodies after parameter binding. Preserve local values, resource connections, instance identities and transaction boundaries through common analysis.

Add dual-backend loop, reset, failure and saved-artifact coverage; migrate queue_rule_pipeline and document the bounded profile.

Validation: 11 focused lit regressions, queue pipeline execution, changed-file pre-commit, strict MkDocs and independent review pass. Aggregate API/examples gates and retained legacy-contract tests still fail; full acceptance is not claimed.
Separate packed Array payloads from component Table references and bump the source contract to 5. Add Table capture, member linking, selected transaction analysis, static traversal, independent histories and a dispatch example.

Validation: 18 transactional lit tests, 12 bounded parameter/Method tests, 271 unit tests, dual backend Table scenarios and example, pinned changed-file hooks and strict docs passed. Retained legacy module_loop aggregate build and two legacy dependency tests remain failing; full nightly not run.
@hmljy2020
hmljy2020 requested a review from zhoubot November 1, 2026 09:24
…wering

Related: #361.

Make catalog tiers distinguish verified executable designs from pending work,
migrate sibling histories and retain partial example migrations and oracles.
Defer further ROB repairs at the user's request.

Separate physical storage lowering from the old transaction materializer,
remove legacy executable routing and obsolete analysis selectors, and retire
selected superseded tests while retaining shared validation and private APIs
still needed by mixed native tests.

Fix payload constraints, bounded dependency/replay traversal, lossless saved
provenance and deterministic artifact loading, and configured-libdir packaging.

Validation: focused native/lit regressions and independent reviews recorded
under docs/gates/logs/; 15 example nightly cases passed before the final proof
cache optimizations; catalog 20/20, installed artifact/runtime tests 3/3,
changed-file pre-commit and strict docs checks passed. Full API acceptance is
not green (18/26 targets on the recorded candidate); ROB execution is deferred,
full picker backend acceptance and remaining legacy cleanup are unfinished.

Other agents' two pre-existing edits and ppt_pipeline are excluded. Evidence
is from the shared worktree, not a claim of isolated clean-commit acceptance.
@hmljy2020 hmljy2020 closed this Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants