fix(vocabulary): skip fenced code blocks when harvesting headings (#2142)

[akshatmalik-bruh]

Summary

• Strips fenced code blocks (both ``` and ~~~) before harvesting headings in vocabulary.py so that shell/script comments (`# Install dependencies`) inside code examples are not mined as project terminology.
• Replaces code block content lines with empty strings rather than deleting them, preserving character count and byte offsets so _definition_after_heading remains accurate.
• Applies the same stripping in _is_release_notes() to prevent docs containing code examples with version comments (e.g. # v1.0.0) from being misclassified as release notes.

Related Issues

Fixes #2142

Test Plan

• Tests pass (uv run pytest tests/unit/generation/test_vocabulary_house_terms.py -v)
• Output fixture test test_extract_terms_output_is_byte_identical passes
• Added 5 regression unit tests for [Bug] The heading harvest never skips fenced code blocks, so shell comments are mined as house vocabulary #2142 covering backtick fences, tilde fences, real headings after code blocks, inline code spans, and version comments in code blocks.

Checklist

• My code follows the project's code style
• I have added tests for new functionality
• All existing tests still pass
• I have updated documentation if needed

[akshatmalik-bruh]

@RaghavChamadiya check the PR once !

[RaghavChamadiya]

Thanks @akshatmalik-bruh. Two things you got right that I want to name before the problem: covering _is_release_notes as well as _harvest (a code block full of # v1.2.0 lines misclassifying a whole doc is the nastier half of #2142, and it is silent), and handling tilde fences, which most patches for this skip.

There is one blocker, in the offsets. _strip_fenced_blocks says it replaces fence content with "blank lines of the same length" and the inline comment says "an empty string of the same byte length", but the code appends "", which is shorter than the line it replaces. So the string shrinks, and _harvest then feeds an offset taken in the shrunk string to a function that slices the original:

entries = [
    (m.group(1), _definition_after_heading(text, m.end()))
    for m in _HEADING.finditer(prose)
]

_definition_after_heading is text[offset:].split("\n")[1:...] (vocabulary.py:514), so every heading that sits after a fenced block reads its definition from the wrong place, shifted earlier by however many characters the block contained. On this document:

# Ledger

```bash
# install the dependencies for the ledger service
npm install
```

## Blast radius

Blast radius is the set of files a change can reach.

len(text) is 155 and len(prose) is 95, and for ## Blast radius the slice you hand the definition scanner starts here:

text[m.end():] -> 'cies for the ledger service\nnpm install\n```\n\n## Blast radius'

It lands mid-code-block. So the patch trades mining code comments as terms for reading definitions out of code blocks, on any README with a fence above a heading, which is most of them. The existing tests do not catch it because they all assert on the term list and never on the definition attached to a term after a fence.

The fix is small, either way round:

• pass prose instead of text to _definition_after_heading, since the offsets are already offsets into prose and the real prose lines are untouched by the strip, or
• make the replacement genuinely length-preserving with " " * len(line), which also makes the docstring true. _HEADING needs a # so a run of spaces cannot match.

I would take the first. Worth a test that asserts the definition, not just the term, for a heading below a fence.

Non-blocking: once you pick one, the docstring and the inline comment both need to stop claiming byte-length preservation if it is not what the code does.

Ping me when it is pushed and I will merge it.

[akshatmalik-bruh]

@RaghavChamadiya check it out now and let me know any changes !