feat: integrate native AcroForm widgets as typed fields

[cau-git]

What

Extracts native interactive PDF widgets (AcroForm) from docling-parse parsed pages and materializes them as keyless, format-neutral fillable field values in the DoclingDocument.

This supersedes and replaces #4055. That PR extracted fields late from pypdfium2 and attached legacy FormItem + GraphData objects after document assembly; this change deletes that path (docling/utils/form_utils.py) entirely. #4055 should be closed once this lands.

How

A small PdfFormFieldModel stage runs after OCR and before layout postprocessing — the only point in the pipeline where both inputs exist:

• raw layout FORM cluster bboxes have not yet been tightened, and
• parsed-page widgets (SegmentedPdfPage.widgets) are still available.

The stage:

1. normalizes each widget into a format-neutral FieldValuePrediction (text / orig / bbox) — checkable controls resolve from widget-local /AS first, then /V; pushbuttons are excluded;
2. binds each widget to a raw FORM cluster by strong coverage (deterministic tie-break on coverage → smallest container → id);
3. puts unmatched widgets into one explicit page-local fallback region;
4. never mutates the layout prediction and never infers labels.

Page assembly replaces matched FORM container elements with FieldRegionElements (reusing the raw region bbox and the cluster's visible children) and emits fallback regions with collision-free synthetic ids. Reading order places each region by bbox and materializes it as FieldRegionItem → FieldItem → FieldValueItem(kind="fillable") in native widget order, with no FieldKeyItem.

Boundaries

• PdfWidget and raw AcroForm state (/Ff, /AS, field names) stay on SegmentedPdfPage.widgets; nothing PDF-specific enters DoclingDocument, FieldValueItem, or add_field_value().
• No PdfWidget survives past the form-field stage.
• Document schema is unchanged (1.10).
• extract_form_fields=False and documents without widgets are exact no-ops (layout prediction identical either way).

Dependencies

Pins the AcroForm branches of the companion PRs (draft until they release):

• docling-core: raw widget contract on the parsed page — feat: expose raw AcroForm widget state docling-core#733 (branch feat/acroform-widget-extraction)
• docling-parse: widget extraction (/Ff, /AS, field name/type) — feat: expose complete AcroForm widget state docling-parse#334 (branch feat/acroform-widget-extraction)

pyproject.toml uses git-branch sources for these; they must be repointed to published versions before merge.

Tests

tests/test_form_extraction.py:

• docling-parse exposes the complete widget contract in native order;
• field mapping / assembly / materialization preserve regions and native order across matched + fallback + overlapping-FORM cases, with no FIELD_KEY;
• full pipeline produces keyless format-neutral fields, and disabling the option is a no-op with an identical layout prediction.

What this does NOT solve (yet)

Deliberately out of scope for this PR — the fields are extracted and typed, but not yet semantically labeled or optimally ordered:

• Field name / key association. Widgets are materialized as keyless FieldValueItems. Raw AcroForm names (/T) and descriptions (/TU) are available on SegmentedPdfPage.widgets but are not associated with the value widgets as FieldKeyItems. No key is inferred from geometry, proximity, or /T//TU (an earlier reconciliation experiment that did this produced wrong keys and leaked machine names like topmostSubform[0].Page1[0].f1_08[0]). Authoritative key↔value association is future work.
• Form reading order. Field values are emitted in native widget order within each region, and regions are placed by bbox. This is not a true form-aware reading order — values are not interleaved with their visible printed labels, and multi-column / tab-order layouts are not reconstructed. Improving intra-region reading order is future work.

[github-actions]

❌ DCO Check Failed

Hi @cau-git, your pull request has failed the Developer Certificate of Origin (DCO) check.

This repository supports remediation commits, so you can fix this without rewriting history — but you must follow the required message format.

---

🛠 Quick Fix: Add a remediation commit

Run this command:

git commit --allow-empty -s -m "DCO Remediation Commit for Christoph Auer <cau@zurich.ibm.com>

I, Christoph Auer <cau@zurich.ibm.com>, hereby add my Signed-off-by to this commit: 06790b294d6bbcc692a257d2a0c6fad6afc6a916
I, Christoph Auer <cau@zurich.ibm.com>, hereby add my Signed-off-by to this commit: ac110b9a584a20d1a7a2f2da2bec3262536fa02c"
git push

---

🔧 Advanced: Sign off each commit directly

For the latest commit:

git commit --amend --signoff
git push --force-with-lease

For multiple commits:

git rebase --signoff origin/main
git push --force-with-lease

More info: DCO check report

[mergify]

Merge Protections

🔴 1 of 2 protections blocking · waiting on 👀 reviews

	Protection	Waiting on
🔴	Require two reviewer for test updates	👀 reviews
🟢	Enforce conventional commit	—

🔴 Require two reviewer for test updates

Waiting for

• #approved-reviews-by >= 2

This rule is failing.

When test data is updated, we require two reviewers

• #approved-reviews-by >= 2

---

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[cau-git]

Before / After

Before

After

[codecov]

Codecov Report

❌ Patch coverage is 41.55844% with 90 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...cling/models/stages/form_field/form_field_model.py	33.96%	69 Missing and 1 partial ⚠️
...models/stages/page_assemble/page_assemble_model.py	35.29%	9 Missing and 2 partials ⚠️
.../models/stages/reading_order/readingorder_model.py	0.00%	8 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

[wittjeff]

Review notes on #4140, read against Well-Tagged PDF 1.0 (WTPDF), the basis of PDF/UA-2. The overall shape is right and I am building on it; these are the things I would change or decide before it lands.

1. Push buttons are not excluded. The description says they are, but _normalize_widget only uses the pushbutton flag to skip the checkbox branch, so a push button becomes an empty fillable value. Signature fields take the same path because /V is a dictionary and docling-parse renders it as empty text. WTPDF 8.10.3.2 says a button's meaning is its CA caption; until docling-parse exposes /MK /CA, skipping push buttons matches the description. (Done in feat(pdf): key form fields by their /TU description and skip artifact widgets #4144.)

2. Zero-size widgets are artifacts (WTPDF 8.9.2.4.13) and should be skipped. Timestamp-signature workflows produce these. (Done in feat(pdf): key form fields by their /TU description and skip artifact widgets #4144.)

3. /TU as key. WTPDF 8.10.2.1 lists TU as a context source and says T is not one, which matches your finding that /T leaks machine names. Using /TU for FieldKeyItem needs no inference. (feat(pdf): key form fields by their /TU description and skip artifact widgets #4144.)

4. Checkbox representation diverges from the HTML backend. This PR emits a FieldValueItem with text checked; html_backend.py emits CHECKBOX_SELECTED / CHECKBOX_UNSELECTED items inside the FieldItem. Both are valid docling-core shapes, but the serializers and downstream consumers will see two different trees for the same form. Worth picking one before this lands.

5. Multi-select list boxes lose their value. docling-parse feat: added excel backend #334 only accepts string or name /V, so an array becomes an empty string. WTPDF 8.10.3.4 also says the Opt display text, not the export value, carries the intent, so choice fields should use the display text once /Opt is exposed.

6. Regions matched against raw FORM clusters that postprocessing later drops end up as fallback regions with the dead cluster's bbox, overlapping the surviving FORM_AREA group. Not wrong, but noisy; it may be worth matching against the surviving clusters instead, or dropping matched regions whose source cluster did not survive.

7. Coverage. The matched-form path is exercised only through a mocked parsed page; the real pipeline test hits the fallback branch. A fixture whose layout actually yields a FORM cluster would close that.

Longer-term direction (structure-tree labels, PrintField for non-interactive forms, ordering by structure rather than /Tabs, role and state slots in docling-core) is in #4143.