Skip to content

[AUTOMATED] Address-based variable matching for TypeMatch - #48

Draft
mahaloz wants to merge 1 commit into
mainfrom
experiment/local-variable-edit-distance
Draft

mahaloz wants to merge 1 commit into
mainfrom
experiment/local-variable-edit-distance

Conversation

@mahaloz

@mahaloz mahaloz commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

[AUTOMATED]

Adds TypeMatch variable correspondence using ABI argument position, calibrated stack offsets, and validated instruction-address overlap.

  • Native provenance is implemented only for angr, Binary Ninja, dewolf, Ghidra, IDA, Kuna, and r2dec.
  • Other and unknown producers remain evaluable through the legacy name-based fallback; their Type results are marked as fallback evidence.
  • Reevaluation, result persistence, and the report carry that evidence classification.
  • Usage-only, fused/stacked modes, prototype tooling, and non-top-seven producer integrations are intentionally excluded.

This PR remains a draft and must not be merged yet.

@mahaloz mahaloz added documentation Improvements or additions to documentation enhancement New feature or request labels Jul 27, 2026
Comment thread scripts/calibrate_local_variable_matcher.py Fixed
@mahaloz mahaloz changed the title [EXPERIMENTAL] Evaluate local-variable correspondence on Coreutils O2 [Experimental] Improved local variable grading Jul 27, 2026
@mahaloz mahaloz changed the title [Experimental] Improved local variable grading [Experimental] Local-variable matching by address and usage Aug 11, 2026
@mahaloz mahaloz changed the title [Experimental] Local-variable matching by address and usage Type-blind variable matching with native line provenance Aug 17, 2026
Comment thread decbench/rendering/assets/app.js Fixed
Comment thread decbench/rendering/assets/app.js Fixed
Comment thread decbench/rendering/assets/app.js Fixed
@mahaloz mahaloz changed the title Type-blind variable matching with native line provenance [AUTOMATED] Type-blind variable matching with native line provenance Aug 25, 2026
mahaloz added a commit that referenced this pull request Aug 27, 2026
[AUTOMATED] Generated by Codex for PR #48.
mahaloz added a commit that referenced this pull request Aug 27, 2026
…r docs

PR #48 carried ~27k lines of calibration and prototype scaffolding around a
~4k-line metric core. The scaffolding is removed so the shipped change is
reviewable; everything it documented about behavior that actually ships is
migrated into docs/metrics.md first.

Removed: decbench/experimental/ (6,782 lines), six scripts/ calibration and demo
drivers (4,704), the four tests that only exercised them (2,201), and
docs/experiments/ (13,616, of which a generated evidence.json is 12,640).

Of decbench/experimental/, two files were one-line `import *` compat shims over
production decbench/metrics/{variable_match,variable_features}.py -- which is why
three test files imported "experimental" while really testing shipped code. Those
are kept, with imports repointed and two renamed to match what they test. The
other two experimental modules were substantial: local_variable_checkpoint.py
(2,391 lines) and local_variable_semantic_audit.py (4,379). Nothing outside the
deleted scripts and tests imported either.

Migrated into docs/metrics.md, each value re-verified against the current code
rather than copied from the old prose: the production policy constants and their
validators; the tree-sitter parsing contract and why tree-sitter is a runtime
dependency; the candidate-discovery rules; the sanitization applied to candidates
recovered from rendered C, which is what keeps address-mode numbers honest for
text-only backends; the feature inventory and the type-leakage exclusions; and
the similarity and fusion math. Result tables, frozen replay anchors and
"Reproducing" sections were not migrated -- they measure an unpublished local
tree through now-deleted scripts.

An adversarial review of the first draft of this migration caught three real
defects, fixed here: it claimed overriding a matcher threshold needs a
cache_version bump (it does not -- the policy is part of the per-function cache
key, so an override re-keys itself); it stated the extractor abstains only on
multiple mismatched definitions (any function_definition present with no matching
name abstains, and no fragment wrapping is attempted); and it described the
degree term as within-function (it is counted over the union of the source and
decompiled variables being matched). It had also dropped the inferred_from_code
sanitization rule and the candidate-discovery contract entirely, and inverted the
threshold provenance -- the address pair was a pre-existing frozen baseline, only
the usage and combined values were selected on the tuning partition.

docs/metrics.md is also updated for cache_version 12, which the Thumb fix earlier
on this branch introduced.

Deleting the four scaffolding test files cost shipped code some coverage;
test_variable_match_features.py regains the literal-recursion case
(variable_features.py:309-312), which was their only reach into production code
that no remaining test touched.

Suite: 6 failed, 1116 passed, 29 skipped -- the same 6 failures that fail on main.
ruff and black show no new debt (18 errors and 6 files before and after, identical
lists). Residual-reference sweep for every deleted module, script and directory
returns nothing.

[AUTOMATED] change produced by Claude Code under maintainer direction.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016P13XTwaZ1HtpNU3F47nMg
@mahaloz
mahaloz force-pushed the experiment/local-variable-edit-distance branch from c2871f8 to 4808a62 Compare September 10, 2026 19:12
@mahaloz mahaloz changed the title [AUTOMATED] Type-blind variable matching with native line provenance [AUTOMATED] Address-based variable matching for TypeMatch Sep 10, 2026
[AUTOMATED]

Limit native variable correspondence to the seven full-dataset decompilers and retain a labeled legacy fallback for unsupported producers.
@mahaloz
mahaloz force-pushed the experiment/local-variable-edit-distance branch from 4808a62 to eb2d6d7 Compare September 11, 2026 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants