Skip to content

fix(jats): preserve structured abstract sections and fix nested list parent - #4172

Open
ceberam wants to merge 2 commits into
mainfrom
fix/jats-abstract
Open

fix(jats): preserve structured abstract sections and fix nested list parent#4172
ceberam wants to merge 2 commits into
mainfrom
fix/jats-abstract

Conversation

@ceberam

@ceberam ceberam commented Sep 4, 2026

Copy link
Copy Markdown
Member

Summary

This PR addresses two gaps in the JATS backend. In particular, it addressed a gap not fully covered by #3584 and documented on #3584 (comment)

Structured abstracts: JATS articles commonly use a structured abstract — an <abstract> element whose content is split into named <sec> children (e.g. Background, Results, Conclusion). The previous implementation collected all section text into a single TextItem, prepending each section title inline as "Title: text…". This lost the document structure that JATS explicitly encodes. Each <sec> inside an <abstract> is now emitted as a dedicated section heading (DocItemLabel.SECTION_HEADER) with its paragraph(s) as separate TextItem children beneath it. Plain (un-sectioned) abstracts are unaffected.

Nested list parent: Nested <list> elements inside a <list-item> were walked with the ListItem itself as the docling parent, causing add_list_item() to receive a non-list-group parent. This triggered a DeprecationWarning from docling-core and a silent on-the-fly group creation. The list group is now created explicitly before recursing into the nested list.

Changes

docling/backend/xml/jats_backend.py

  • Adds a new AbstractSection TypedDict (title, paragraphs) to represent a parsed abstract sub-section.
  • Extends the Abstract TypedDict with a sections: list[AbstractSection] field alongside the existing content field (which is retained for plain paragraph-only abstracts).
  • Rewrites _parse_abstract_section() to return an AbstractSection instead of a flat string.
  • Rewrites _parse_abstract() to separate <sec> children (→ sections) from bare <p> children (→ content).
  • Rewrites _add_abstract() to emit each section as doc.add_heading(level=hlevel+2) with individual doc.add_text() calls underneath, falling back to the original single-TextItem path for plain abstracts.
  • Fixes nested list handling: explicitly creates a GroupLabel.LIST group under the ListItem before walking a nested <list>, so that child list-item elements always receive a proper list-group parent.

tests/test_backend_jats.py

  • Updates test_jats_structured_abstract_sections_are_preserved to assert the new structure (sub-headings present, flat "Title: text" format absent) and to verify the DoclingDocument item labels directly.

tests/data/jats/sources/pmc2231364.nxml

  • Adds a minimal JATS fixture derived from the BMC Microbiology article PMC2231364 (CC BY 2.0), which has the three-section Background/Results/Conclusion abstract structure that originally exposed the gap.

tests/data/jats/groundtruth/pmc2231364.nxml.*

  • Groundtruth files generated from the new fixture. The markdown output now matches the structure visible on the PubMed page:
## Abstract

### Background

Environmental modulation of gene expression…

### Results

To provide us with a comprehensive view…

### Conclusion

The comparative transcriptomics analysis…

Behaviour before and after

Structured abstract

Before — all sections collapsed into one TextItem:

## Abstract

Background: Environmental modulation… Results: To provide… Conclusion: The comparative…

After — each section is a separate heading with its own paragraph:

## Abstract

### Background

Environmental modulation…

### Results

To provide…

### Conclusion

The comparative…

Nested lists

BeforeListItem passed directly as parent to nested list walk, triggering:

DeprecationWarning: ListItem parent must be a list group, creating one on the fly.

After — a GroupLabel.LIST group is created explicitly under the ListItem before recursing, matching the required parent contract.

Testing

All 40 JATS tests pass with -W error::DeprecationWarning. The new fixture is picked up automatically by test_e2e_jats_conversions.

Checklist:

  • Documentation has been updated, if necessary.
  • Examples have been added, if necessary.
  • Tests have been added, if necessary.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
Nested <list> elements inside a <list-item> were walked with the
ListItem itself as the docling parent, causing add_list_item() to
receive a non-list-group parent and emit a DeprecationWarning while
silently creating a list group on the fly. Fix by explicitly creating
the GroupLabel.LIST group before recursing into the nested list.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
@ceberam ceberam added bug Something isn't working xml issue related to supported schema-specific XML formats labels Sep 4, 2026
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

DCO Check Passed

Thanks @ceberam, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🔴 1 of 2 protections blocking · waiting on 👀 reviews

Protection Waiting on
🔴 Require two reviewer for test updates 👀 reviews
🟢 Enforce conventional commit

🔴 Require two reviewer for test updates

Waiting for

  • #approved-reviews-by >= 2
This rule is failing.

When test data is updated, we require two reviewers

  • #approved-reviews-by >= 2

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@ceberam ceberam changed the title Fix/jats abstract fix(jats): preserve structured abstract sections and fix nested list parent Sep 4, 2026
@ceberam ceberam mentioned this pull request Sep 4, 2026
8 tasks
@codecov

codecov Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 86.48649% with 5 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
docling/backend/xml/jats_backend.py 86.48% 1 Missing and 4 partials ⚠️

📢 Thoughts on this report? Let us know!

@PeterStaar-IBM PeterStaar-IBM left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working xml issue related to supported schema-specific XML formats

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants