Repository navigation
docs: bare #NNN citations do not resolve in Sphinx — add an extlinks role instead of auto-linking #989
Description
Activity
besides
:issue:, we need:pr:and perhaps other useful roles that can save some duplicated work in the docs.Taken — a role set rather than one role. And measuring which roles beats guessing at them, because the
biggest source of duplicated work in these docs is not GitHub at all.URL counts across
pcapkit/,tests/,util/anddocs/:www.iana.org 471 pypi.org 37 github.com 164 www.ietf.org 36 en.wikipedia.org 84IANA outnumbers GitHub nearly three to one, and its URLs are the most literally duplicative thing in the
tree. 373 of them followassignments/<registry>/<registry>.xhtml#<anchor>— and in all 373 the registry
name is repeated verbatim within the same URL, across 19 distinct registries (mobility-parameters148,
hip-parameters72,protocol-numbers19,sctp-parameters16,tcp-parameters16). So the single
highest-value role is an IANA one:extlinks = { 'issue': ('https://github.com/JarryShaw/PyPCAPKit/issues/%s', '#%s'), 'pr': ('https://github.com/JarryShaw/PyPCAPKit/pull/%s', 'PR #%s'), 'iana': ('https://www.iana.org/assignments/%s/%s.xhtml', None), # see note 'wikipedia':('https://en.wikipedia.org/wiki/%s', None), }
One caveat on
:iana:that decides whether it needs a custom role rather thanextlinks.extlinks
substitutes a single%s, so it cannot fill the registry name twice. Either the role takes the doubled form
(:iana:mobility-parameters/mobility-parameters.xhtml#mobility-parameters-10``, which saves the host but not the
duplication), or this is a small custom role inconf.pythat expands one registry name into the full URL
— about fifteen lines, and the thing that actually removes the 373 repetitions. I would write the custom role.:pr:as a separate role earns its place for a reason beyond tidiness. GitHub redirects between the two
forms — I verified/issues/986→ 302 →/pull/986and/pull/985→ 302 →/issues/985— so a single
:issue:would resolve either way. But #719's whole ruling turns on the issue-versus-pull-request distinction,
andtests/is exempt precisely so it can keep citing pull requests. A single role would flatten that
distinction in the source even while the link worked.And a trap that makes
:issue:actively wrong in two places.#106and#251are GitHub Discussions —
the issues API returns404for both, so:issue:106`` would render a link to a page that does not exist.
Either a third:discussion:role (`/discussions/%s`), or those two sites keep an explicit URL. There may be
more than two; the discussions API needs GraphQL to enumerate, which I have not done.:rfc:needs nothing — it is a Sphinx built-in and already in use, including the section-anchor form
:rfc:`8684#3.1`.Sequencing unchanged: this lands after #987, so the 1,858 citation sites are not touched twice.
Labelled
needs: decisionandblocked. That should have happened when I asked the questions, and its
absence had a concrete cost: my decision watch keys on theneeds: decisionlabel, so for the last ~15
minutes it reported nothing waiting while two real asks sat in this thread. An unlabelled ask is an invisible
ask.needs: decision— two questions, both open since 16:06Z:- Does
:iana:get a custom role?extlinkssubstitutes a single%s, so it cannot fill the registry
name twice — and the duplication is the prize: 373 URLs follow
assignments/<registry>/<registry>.xhtmland in all 373 the registry name repeats inside the same URL,
across 19 registries (mobility-parameters148,hip-parameters72). Anextlinksentry saves the host
only; a ~15-line custom role inconf.pyremoves all 373 repetitions. My recommendation is the custom role. - Should the roles land before docs(tests,pcapkit): paraphrase the remaining quoted rulings instead of quoting them (#719) #987 rather than after? As sequenced, docs(tests,pcapkit): paraphrase the remaining quoted rulings instead of quoting them (#719) #987 rewrites the prose at these sites
and then this issue revisits the same ~1,858 citations to add markup — two passes over one set. Roles first
means docs(tests,pcapkit): paraphrase the remaining quoted rulings instead of quoting them (#719) #987's conversions are written as:issue:/:pr:from the start. The cost of switching is that docs(tests,pcapkit): paraphrase the remaining quoted rulings instead of quoting them (#719) #987's
first group is already in flight, so a handful of sites would still need the second pass.
blocked— the checkable condition. Even once both are decided, this waits on #987 finishing, for the
same file-collision reason that gated #987 behind #982: two changes editing the same docstrings on different
branches clobber silently, with the later push winning and nothing announcing it. Unblock when
gh issue view 987 --json statereturnsCLOSED.Unchanged and still measured, so the decision has a baseline: bare
#NNNresolves to nothing —conf.py
carries noextlinks, nosphinx-issues, no issue role. 1,858 bare refs in docstrings across 222 files, 492 in
docs/**.rst, 627 in plain comments that need no change because GitHub's code view linkifies them. A
digit-keyed regex is the wrong instrument in both directions: widening#\d{3}to#\d+catches 46
one-digit RFC packet-diagram labels (DH GROUP ID #1-#4) that are not citations, and numbering crosses
#1000shortly. And#106/#251are Discussions — the issues API 404s on both, so:issue:would link
to a page that does not exist.- Does
- if
:ianacan work (Sphinx can correctly render), then i think it buys simplicity for the docs. - either way, go with your recommendations.
- if
Both taken, and the condition on (1) is met — I built the role and rendered it rather than assuming. A
15-line custom role inconf.py, run through a real Sphinx build on the repo venv (Sphinx 9.1.0), emits exactly
what it should from a single registry name::iana:`mobility-parameters` -> href="https://www.iana.org/assignments/mobility-parameters/mobility-parameters.xhtml" :iana:`hip-parameters` -> href="https://www.iana.org/assignments/hip-parameters/hip-parameters.xhtml"Clean build, no warnings. So
:iana:works and it does buy the simplicity — one registry name in, the doubled
path out, which is what removes all 373 repetitions.needs: decisionis off;blockedstays.On (2) I am keeping the current order — #989 after #987 — but I have to correct the reasoning I gave you,
because the figure I used overstated the case. I said the two issues would mean "two passes over the same
~1,858 citations". That is wrong: #987's scope is 86 owner-attributed quotation sites, not 1,858. The
1,858-site markup pass happens exactly once whichever order is chosen, and the genuine overlap is only those 86.
So ordering barely matters for cost, and I should not have presented it as a saving.What does decide it is collision, the same thing that gated #987 behind #982. #987 has six conversion groups
still queued acrosspcapkit/andtests/; running the markup pass concurrently would put two changes into the
same docstrings on different branches, where the later push silently wins. Sequencing after #987 removes that
risk entirely, and the cost is a mechanical second visit to 86 sites rather than to 1,858.One thing to decide when this unblocks, since the measurement is already in hand: the markup pass cannot be
a blind regex in either direction.#\d{3}misses the four-digit numbers arriving shortly;#\d+catches the
46 one-digit RFC packet-diagram labels (DH GROUP ID #1-#4), which are not citations. An opt-in role is
immune to both, but it means the 1,858 sites are converted deliberately rather than swept — so this issue is a
large body of mechanical work, not a config change plus a sed.Your third point came through empty, so if there was a (3) it did not arrive.
- added a commit that references this issue
on Oct 2, 2026 One decision before the next tranche lands, and it sets a precedent for 243 more sites.
The
pcapkit/corekit/tranche is prepared and verified against068192625.pcapkit/corekit/holds 119 citation occurrences, not the 66 I first measured — my figure was a line count and several lines carry two. They split three ways by rendering context:context sites treatment docstrings 89 converted to :issue:#:autodoc doc-comments11 converted plain #comments19 left alone — this is the question The case for leaving them: a plain
#comment is never parsed as reStructuredText, so:issue:`431`there produces role syntax in a place nothing will ever render. The reader loses a readable#431and gains no link. This issue exists for link resolution in the built documentation, and a plain comment is not in the built documentation.The case against:
conventions/documentation.rst:203-213says the exempt set is the changelog andtests/and "nothing else is". On a literal reading that leaves plain comments in scope. I read that passage as governing the issue-versus-pull-request naming rule rather than role conversion, but it is a reading.Tree-wide this decides 243 plain-comment citations under
pcapkit/, so it is worth settling once rather than per tranche. My own recommendation is to leave them, on the rendering argument. The 19 here are infields/collections.py,fields/field.py,fields/misc.py,fields/numbers.py,infoclass.pyandmodule.py— converting them would also makemodule.pyandcollections.pythe only corekit files the tranche touches at all.Two things that need no decision, for the record. All 38 distinct numbers in the tranche are issues — zero pull requests, zero of the five Discussions — so #719's pull-request-called-an-issue half had nothing to fix here, and
:pr:/:discussion:are unused. And there are zero inline-literal conversions in this tranche, so unlike the conventions pages there is no monospace-to-link rendering change to declare.We can keep the original
#NNNformats for plain comments. So that GitHub code tree view will be able to properly render them.12 remaining items
- added 5 commits that reference this issue
on Oct 3, 2026 The
pcapkit/conversion is complete on79fce29c5. Five tranches landed: #1000 (corekit), #1002 (foundation/utilities/toolkit/dumpkit), #1003 (protocols/internet+protocol.py), #1004 (the rest ofprotocols/) and #1005 (vendor+const).Measured on current
main, counting bytokenizetoken kind and includingFSTRING_MIDDLE— 28 bare three-or-more-digit citations remain in convertible positions, and every one is a deliberate exclusion:- 25 sit inside the
vendorgenerators'LINEtemplates on lines that begin with#, so they emit plain#comments in the generatedconstfiles, which the ruling here keeps bare. - 3 are runtime exception message strings, where a role would print as raw markup in a user's traceback:
pcapkit/protocols/schema/schema.py:338(SchemaError, Schema subclasses share one ABCMeta cache on Python <=3.10, so isinstance/issubclass return whichever answer was asked first #439),pcapkit/vendor/ngap/procedure_code.py:148andpcapkit/vendor/ngap/protocol_ie.py:120(bothImportError, fix: six .get implementations mint unknown keys at a shared -1, so the second aliases to the first #880). Two tests pin that last pair —tests/vendor/test_vendor_ngap_unit.py:169,180assert'#880'appears in the raised text.
Citations in plain
#comments throughoutpcapkit/are untouched, per the ruling, andtests/is outside this issue's scope by the same carve-out.What remains is the ~475 changelog sites, which belong to #657 — including the only two bare Discussion citations in the tree, #106 and #251. I have dropped
wipsince no open PR covers this issue any more.Decision needed: close this now and let #657 carry the changelog conversion, or hold it open until that lands? Labelling
needs: decisionand leaving it alone until you say.- 25 sit inside the
Close it with notes and update the other issue with the remaining tasks.
Closing on your ruling, with the notes recorded here and the leftovers moved to #719 — #657 is a closed pull request, so #719 is the other open issue.
Done: the
pcapkit/conversion, across five tranches — #1000 (corekit), #1002 (foundation,utilities,toolkit,dumpkit), #1003 (protocols/internet+protocol.py), #1004 (the rest ofprotocols/), #1005 (vendor+const). #998 and #999 defined the roles; #1001 added:wikipedia:and:iana:.The rules this issue settled, which the remaining work inherits:
- Convert docstrings and
#:autodoc doc-comments only. Plain#comments keep the bare#NNNform, because GitHub's code-tree view hyperlinks them and a role would render as literal text there. - Never convert a citation in a runtime string — a role prints as raw markup in a user's traceback. That is why
pcapkit/protocols/schema/schema.py:338and the twoImportErrormessages inpcapkit/vendor/ngap/stay bare;tests/vendor/test_vendor_ngap_unit.py:169,180pin the last pair. pcapkit/const/**is generated frompcapkit/vendor/**, so a const-side edit alone is reverted at the next regeneration; the conversion goes in the generator and is mirrored by hand, without running the crawlers.- Resolve each number rather than assuming:
:issue:/:pr:/:discussion:.GH-nnnis deliberate and untouched.tests/is carved out entirely.
Verified complete on
711e3a96f: 28 bare three-or-more-digit citations remain in convertible positions acrosspcapkit/, and every one is a deliberate exclusion — 25 insidevendortemplates on lines that emit plain#comments, plus the 3 runtime exception strings above.Remaining and now on #719: 889 bare citations over 274 distinct numbers in 38 changelog files, including the tree's only two bare discussion citations,
#106and#251.- Convert docstrings and
A bare
#NNNin a docstring or an.rstpage renders as literal text — Sphinx has nothing configured toresolve it.
docs/source/conf.py'sextensionslist has nosphinx.ext.extlinks, nosphinx-issues, and nocustom issue role; it is
viewcode,intersphinx,autodoc,napoleon,todo,sphinx_autodoc_typehints,sphinxext.opengraph,sphinx_copybutton,sphinxcontrib.mermaid. So every citation added by the #719 work isunclickable in the published docs, while the same text in a plain
#comment is fine — GitHub's code viewlinkifies it there, and those need no change.
Measured scale (AST over docstrings, so
#:attribute comments that autodoc renders are included; plaincomments excluded):
A blind regex transform is the wrong instrument, and the digit range is why. Counting
#\d+rather than#\d{3}turns up 46 one-digit matches that are not citations at all — every one is an RFC packet-diagramlabel,
DH GROUP ID #1through#4, in HIP and MH docstrings. Today there are 0 two-digit and 0 four-digitmatches, so a
#\d{3}pattern happens to be clean — but that is luck, and it expires twice: numbering crosses#1000shortly, and the diagram labels stay one-digit forever. A transform keyed on digit count will eithermiss the new four-digit citations or linkify the packet diagrams.
Two further forms already excluded from the counts above, both of which a naive pattern would also catch: hex
format specifiers like
{spi:#010x}, and RFC section anchors like:rfc:`8684#3.1`.Recommended: an explicit opt-in role via
sphinx.ext.extlinks, not an automatic rule.Then a citation is written
:issue:`719`and renders as#719, linked. Being opt-in per site is the wholepoint:
DH GROUP ID #1is left alone because nobody marks it up, so the packet diagrams cannot be corrupted bya future numbering change.
One role covers both issues and pull requests, verified rather than assumed — GitHub redirects between the
two forms, so the URL need not know which it is:
That matters here because #719's ruling turns on the issue-versus-pull-request distinction, and a single
:issue:role would quietly flatten it in the markup even though the link still resolves. Worth decidingwhether to add a separate
:pr:role purely to keep the distinction visible in source — the rendered text is#NNNeither way.Sequencing. This should land after #987's paraphrase sweep rather than before: #987 is already rewriting
the prose around most of these citations, and doing the markup conversion first would mean touching the same
1,858 sites twice. Also note
tests/project/test_conventions_doc_claims.pyalready checks that a displayednumber matches the number in its own URL for the hyperlinked cases — that test is the natural place to extend
coverage once a role exists.