Skip to content

[API Feedback]: ExtractionDocument traversal: the obvious loop silently drops content, and the result depends on the provider #7962

Open

Description

@luisquintanilla

Summary

While writing the introductory blog post and samples for DocumentExtractionClient, I hit a usability problem with the shared document model in Microsoft.Extensions.Documents:

  1. The obvious loop (foreach section in document.Sections → foreach element in section.Elements) compiles, runs, and returns a result that looks valid, but it silently skips content whenever a provider nests sections.
  2. Whether content is nested depends on the provider. On the same page, PdfPig and Mistral return flat documents, while Content Understanding and Document Intelligence return nested sections. Code tested against one provider can lose most of its output after a provider swap, which is the scenario the abstraction is meant to make safe.
  3. Even the recommended EnumerateContent() does not reach table cell text. Every table cell's Content is an ExtractionDocumentSection, and there is no built-in way to walk a section. In our samples, the table held more text than a typical consumer can reach.

The tree model itself is right, and I'm not proposing to flatten it. I am proposing:

  • Make the safe path the obvious path: clearer docs, EnumerateContent() on sections, and a built-in text projection.
  • Tighten one enumerator: Cells currently yields null entries.

All of this is cheap to do while the API is in preview.

Tested with Microsoft.Extensions.Documents.Abstractions / Microsoft.Extensions.DocumentExtraction 11.0.0-preview.1.26560.3 and the .NET AI Community Toolkit providers.

What I observed

The input is one page with headings, paragraphs, a 6×5 table with merged cells, and a figure (tokens-explained-hero.pdf). I sent the same bytes to each provider and measured four things:

  • Two-level loop: the obvious nested foreach.
  • EnumerateContent(): the elements it returns, counting characters from each element's own TextContent.
  • Table cell text: characters reachable only through table.Cells → cell section → Elements.
  • Section depth: how deeply the provider nested sections.
Provider Root / total sections Max depth Two-level loop EnumerateContent() Table cell text
PdfPig 1 / 1 1 35 elements, 1,035 chars 35 elements, 1,035 chars n/a (no tables)
Mistral OCR 1 / 1 1 3 elements, 1,285 chars 3 elements, 1,285 chars n/a (no tables)
Azure Content Understanding 1 / 2 2 4 elements, 88 chars 13 elements, 666 chars 447 chars, 26 cells
Azure Document Intelligence 1 / 4 3 4 elements, 137 chars 13 elements, 666 chars 447 chars, 26 cells

Findings:

  • The flat providers hide the bug. A two-level loop is correct for PdfPig and Mistral, so a developer who starts locally with PdfPig (the zero-setup path in our samples) writes this loop, tests it, and ships it.
  • After a swap to a cloud provider, the same loop loses most of the page. It reaches 4 of 13 elements and 13–21% of the text that EnumerateContent() reaches. Nothing throws or warns, and the result is plausible-looking text.
  • EnumerateContent() doesn't include tables either. It returns the ExtractionDocumentTableElement, whose own Content is empty on both Azure providers. All 26 cells hold an ExtractionDocumentSection, and each section contains one text element. A consumer that reads Content from each enumerated element gets 666 characters. The 447 characters in the table, often the most valuable data on the page, are reachable only through a custom recursive walk.
  • table.Cells yields null for covered positions. Enumerating the 6×5 grid produced 30 items: 26 anchor cells and 4 nulls. A natural foreach (var cell in table.Cells) cell.RowIndex... throws a NullReferenceException. (The indexer remarks document null for covered positions, but the enumerator behavior surprised me.)

Minimal repro

ExtractionDocument document = await client.ExtractAsync(stream, "page.pdf", "application/pdf");

// 1. The obvious loop: correct for flat providers, silently incomplete for nested ones.
foreach (var section in document.Sections)
{
    foreach (var element in section.Elements)
    {
        if (element is ExtractionDocumentTextElement { Content: TextContent text })
        {
            Console.WriteLine(text.Text);
        }
        // A nested ExtractionDocumentSection lands here and is skipped.
    }
}

// 2. The recommended walk: reaches every content element, but not table cell text.
foreach (var element in document.EnumerateContent())
{
    if (element is ExtractionDocumentTableElement table)
    {
        foreach (var cell in table.Cells)
        {
            // cell can be null (covered position).
            // cell.Content is an ExtractionDocumentSection: there is no built-in way to get its text.
        }
    }
}

The full probe that produced the table is in the appendix.

Why this matters

  • It breaks the core promise. The abstraction exists so you can swap providers without rewriting extraction code. Today, provider-dependent shapes plus a loop that silently drops content mean a provider swap can quietly degrade RAG quality, summaries, or data extraction, with no error to catch.
  • The failure is silent and plausible. You get some text, headings included, so it looks right in a quick test. In RAG, the missing content shows up later as worse answers, and nobody traces it back to the loop.
  • Everyone writes the same helper. Our samples needed a recursive Leaves(element) walker, a TextOf(element) helper that joins section text (because "table cells hold a section too"), and a null check in every cell loop. The Data Ingestion bridge I wrote also walks nested sections and cell sections manually. If every consumer writes these helpers, some will get them wrong, and the library is the right place to provide them.
  • Tables are where users get value. The capability that sets layout-aware providers apart (tables with merged cells and headers) is the part that is hardest to read through the API.

What's right today (keep it)

  • The tree model. Nesting preserves heading → subheading structure, which chunkers, Markdown rendering, citations, and navigation need. Flattening loses information that can't be recovered.
  • Content is local, not aggregated. This avoids duplicated text and keeps mutation predictable. Any aggregation should be computed, not stored.
  • EnumerateContent() as a named method instead of IEnumerable. This was the right call in [API Proposal]: DataIngestion: Document representation #6893, for discoverability and to document that sections aren't returned.
  • Consistency with IngestionDocument. With Integrate shared Documents while retaining IngestionDocument #7904, MEDI forwards Sections and EnumerateContent() to the shared model, so any change here lands in both places. That's good, and it raises the stakes for getting it right before GA.

Options considered

A. Documentation only

Change the Sections summary from "Gets the ordered root sections of the document." to something like "Gets the root sections. Sections can nest, and providers differ in how deeply. To visit all content, use EnumerateContent()." Do the same for ExtractionDocumentSection.Elements, and add a README section titled "Reading a document" that covers the tree, the provider differences, and tables.

  • Pros: Not breaking, nearly free, and it helps right away in IntelliSense.
  • Cons: It doesn't fix table cells, and it doesn't remove the need for every consumer to write a recursive helper. Docs alone rarely prevent a loop that looks obviously correct.

B. Add EnumerateContent() to ExtractionDocumentSection

The #6893 review concluded this "isn't needed at this time." The table-cell shape is new evidence: every cell's content is a section, so reading a cell requires walking a section.

  • Pros: Small, additive, and symmetric with the document method. It makes cell.Content usable as ((ExtractionDocumentSection)cell.Content).EnumerateContent() without custom code, and it lets a chunker process one subtree.
  • Cons: Adds a little surface. Cell content is typed ExtractionDocumentElement, so callers still need a type check. A helper on the cell (option D) would hide that.

C. Built-in text projection (computed, not stored)

Add a method that renders the plain text of an element and everything below it, including table cells. For example, GetText() on ExtractionDocument and ExtractionDocumentElement, or extension methods in the same namespace.

  • Pros: Covers the most common need (text for a prompt, an embedding, or a search index) in one call that can't silently drop content. It's consistent with "content is local", because the aggregate is computed on demand. It mirrors the GetMarkdown() aggregation that IngestionDocumentElement had in [API Proposal]: DataIngestion: Document representation #6893, which the shared model doesn't carry over.
  • Cons: It requires policy decisions: separators between elements and cells, whether to include image OCR text or alt text, and whether to include page headers and footers. A Markdown renderer is even more opinionated (merged cells, heading levels), so it might belong in Data Ingestion rather than in the abstractions.

D. Cell enumeration fixes

  • Make ExtractionDocumentTableCellCollection's enumerator yield only anchor cells, never null. Covered positions remain reachable through the indexer, GetCoveringCell, and IsCovered.
  • Optionally add a cell-level text helper, cell.GetText(), as part of C.
  • Pros: Removes a NullReferenceException trap. Anchor-only enumeration matches how people think about cells ("each merged cell appears once").
  • Cons: A behavior change. Anyone who relies on positional enumeration would need to switch to the indexer, but that is unlikely in a preview.

E. Rename Sections to RootSections

  • Pros: The name itself explains the shape.
  • Cons: It's breaking. It also diverges from IngestionDocument.Sections, so it would need to be renamed in both places, coordinated with Integrate shared Documents while retaining IngestionDocument #7904. On its own, a rename still leaves consumers writing a recursive helper. The benefit is small next to A–D.

F. Make the shape consistent across providers (for example, always flat, or always nested)

  • Pros: Code written against one provider would behave the same on all of them.
  • Cons: Nesting reflects real capability differences: Document Intelligence returns deeper hierarchy than Content Understanding, and PdfPig has none. Forcing one shape either discards structure or invents it. The abstraction should make provider differences safe to consume, not hide them.

G. Analyzer for nested Sections/Elements loops

  • Pros: Catches the bug at compile time.
  • Cons: It's a heuristic that is hard to make precise (legitimate code inspects one level on purpose) and costs more to maintain. It's worth reconsidering only if A–D don't move the needle.

H. Flatten the model

Not recommended. It loses hierarchy, conflicts with "content is local", and would undo #6893 and #7904.

Recommendation

In priority order, all before GA:

Priority Change Breaking?
P0 A. Docs on Sections, Elements, and Cells, plus a README "Reading a document" section that shows the provider differences and the right loops No
P0 B. ExtractionDocumentSection.EnumerateContent() No (additive)
P0 D. Cells enumerator yields anchor cells only Behavior change (preview)
P1 C. Computed plain-text projection for the document, elements, and cells; leave Markdown rendering to Data Ingestion or a separate helper No (additive)
P1 Conformance: give provider authors a test fixture that has nested sections and section-valued cells, so consumer guidance and samples are tested against both shapes No
Skip for now E (rename), F (normalize shape), G (analyzer), H (flatten) n/a

Sketch of the additive API (names are placeholders):

namespace Microsoft.Extensions.Documents;

public sealed class ExtractionDocumentSection : ExtractionDocumentElement
{
    public List<ExtractionDocumentElement> Elements { get; }

    // New: depth-first, non-section content of this subtree, same semantics as ExtractionDocument.EnumerateContent().
    public IEnumerable<ExtractionDocumentElement> EnumerateContent();
}

public static class ExtractionDocumentTextExtensions
{
    // New: computed (never stored) plain text of the node and all descendants, including table cells
    // in row-major order, each merged cell once. Policy for separators and image text documented.
    public static string GetText(this ExtractionDocument document);
    public static string GetText(this ExtractionDocumentElement element);
    public static string GetText(this ExtractionDocumentTableCell cell);
}

public sealed class ExtractionDocumentTableCellCollection : IEnumerable<ExtractionDocumentTableCell>
{
    // Changed: enumerates anchor cells only (no nulls). Positional access stays on the indexer.
    public IEnumerator<ExtractionDocumentTableCell> GetEnumerator();
}

With these changes, the code in the repro becomes:

string text = document.GetText();               // everything, including table cells

foreach (var element in document.EnumerateContent())
{
    if (element is ExtractionDocumentTableElement table)
    {
        foreach (var cell in table.Cells)       // never null
        {
            Console.WriteLine($"[{cell.RowIndex},{cell.ColumnIndex}] {cell.GetText()}");
        }
    }
}

Open questions

  1. Should GetText() live on the abstractions (for every consumer) or in a helpers package? I lean toward the abstractions, because a correct traversal is part of the contract.
  2. What should the default text policy be for images (OCR Text versus AlternativeText) and for pageHeader/pageFooter roles? I suggest including OCR text, excluding alt text, and including headers and footers, with an options overload later if needed.
  3. Should EnumerateContent() take an option to descend into table cells? I lean toward no: keep it structural and let GetText() and cell helpers handle cell content.
  4. Does Integrate shared Documents while retaining IngestionDocument #7904 change any of this? It forwards Sections and EnumerateContent(), so the additions would benefit MEDI automatically. The Cells behavior change should land in the same preview.

Appendix: probe

Probe source (C# file-based app) and output
#!/usr/bin/env dotnet
// Run from the root of luisquintanilla/dotnet-document-extraction-samples.
#:include common/Providers.cs
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Documents;

foreach (var p in args)
{
    using var extractor = Providers.Create(p, out _);
    var doc = await Samples.ExtractFileAsync(extractor!, Path.Combine("books", "pages", "tokens-explained-hero.pdf"));

    static int Chars(ExtractionDocumentElement e) => e.Content is TextContent t ? t.Text.Trim().Length : 0;
    static int Deep(ExtractionDocumentElement e) => Chars(e) + (e is ExtractionDocumentSection s ? s.Elements.Sum(Deep) : 0);

    var naive = doc.Sections.SelectMany(s => s.Elements).Where(e => e is not ExtractionDocumentSection).ToList();
    var all = doc.EnumerateContent().ToList();
    var tables = all.OfType<ExtractionDocumentTableElement>().ToList();

    Console.WriteLine($"{p}: two-level {naive.Count} elements/{naive.Sum(Chars)} chars; " +
        $"EnumerateContent {all.Count}/{all.Sum(Chars)}; " +
        $"cell text {tables.Sum(t => t.Cells.Sum(c => c?.Content is null ? 0 : Deep(c.Content)))} chars; " +
        $"null cells {tables.Sum(t => t.Cells.Count(c => c is null))}");
}
pdfpig:   rootSections=1 totalSections=1 maxDepth=1
          two-level 35 / 1035 chars; EnumerateContent 35 / 1035 chars (Text:35)
cu:       rootSections=1 totalSections=2 maxDepth=2
          two-level 4 / 88 chars;    EnumerateContent 13 / 666 chars (Text:11, Table:1, Image:1)
          table 6x5: 26 cells, 4 nulls from enumerator, every cell.Content is a section; cell text 447 chars
docintel: rootSections=1 totalSections=4 maxDepth=3
          two-level 4 / 137 chars;   EnumerateContent 13 / 666 chars (Text:11, Table:1, Image:1)
          table 6x5: 26 cells, 4 nulls from enumerator, every cell.Content is a section; cell text 447 chars
mistral:  rootSections=1 totalSections=1 maxDepth=1
          two-level 3 / 1285 chars;  EnumerateContent 3 / 1285 chars (Text:2, Image:1)

Related: #6893 (original document representation and the EnumerateContent decision), #7587 (DocumentExtractionClient proposal), #7904 (shared Documents model in MEDI), #7961 (partial-extraction errors).

Activity

  1. luisquintanilla commented on Nov 19, 2026

    @luisquintanilla
    ContributorAuthor

    A few refinements after reviewing this against the goal of keeping Microsoft.Extensions.Documents a set of flexible exchange types rather than an implementation contract.

    None of the recommendations constrain providers.

    • Docs, ExtractionDocumentSection.EnumerateContent(), and anchor-only Cells enumeration are pure functions of, or ergonomics fixes to, the exchange types.
    • The only option that would dictate producer behavior (F, normalizing shape across providers) is the one I recommend against.

    The burden lands on consumers. Every freedom the contract gives producers is a case every consumer has to handle:

    • arbitrary nesting
    • section-valued cells
    • non-text payloads
    • local payloads on the document and sections

    This is the same pattern as AIContent in Microsoft.Extensions.AI. There, ChatMessage.Text and ChatResponse.Text give consumers a shape-agnostic way to be correct without constraining producers. That's the precedent for the additions proposed here.

    With that lens, three corrections to the original post:

    1. Keep GetText() policy minimal. Open question 2 suggested including OCR text and page headers and footers, and excluding alt text. That's more opinion than an exchange type should carry. Mirror ChatMessage.Text instead: concatenate TextContent in document order, including table cells (each anchor cell once), with a simple documented separator. Leave richer rendering (Markdown, image text policy, header/footer filtering) to consumers or Data Ingestion.
    2. "Every cell's content is a section" is provider behavior, not the contract. ExtractionDocumentTableCell.Content is typed as any ExtractionDocumentElement. The Content Understanding and Document Intelligence providers in the .NET AI Community Toolkit choose to wrap cell text in a section. That's valid, because a cell can hold several paragraphs. The helper is still needed, because consumers have to handle any legal shape, but the shape itself comes from the providers. If provider-side changes turn out to be worthwhile, that's a separate conversation in CommunityToolkit/AI.
    3. The original post missed local payloads. ExtractionDocument.Content and ExtractionDocumentSection.Content can carry text, and EnumerateContent() intentionally skips them (per its remarks). None of the providers I tested populate them on this page, but the contract allows it. So GetText() should include document and section local payloads in order, or the "can't silently drop content" goal doesn't hold.
  2. luisquintanilla commented on Nov 19, 2026

    @luisquintanilla
    ContributorAuthor

    I prototyped the proposal and measured it against live providers.

    Setup. Ten hard single-page PDFs from ParseBench-2: charts, complex table headers, dense and financial tables, a form, headings, layout and CJK text. Each page was extracted with the five providers in the samples repo, and each result was read five ways. The number is the share of the page's ground-truth words that each way returns.

    Provider A: two-level loop B: EnumerateContent + is TextElement C: recursive walker, no image text E: document.GetText()
    Azure Content Understanding 0.05 0.48 0.84 0.92
    Azure Document Intelligence 0.06 0.50 0.77 0.85
    Mistral 0.76 0.76 0.76 0.76
    Ollama (qwen3.5:4b) 0.83 0.83 0.83 0.83
    PdfPig 0.81 0.81 0.81 0.81

    Compare within a row, not across rows; this isn't a provider ranking.

    • Same code, same PDFs, different answer. The loop reads 76–83% on flat providers and 5–6% on the Azure providers, which nest content. On a 10-K page it reads 0% on both, while GetText() reads 99%.
    • Recursion alone isn't enough. B handles nesting but still drops table cells (timetable: 0.02 vs 1.00).
    • Cells alone aren't enough either. C, the walker in the CommunityToolkit README, drops chart labels stored in image text (charts page: 0.29 vs 1.00).
    • Nothing regresses. On flat providers every strategy agrees.

    The prototype is in #7963. The harness, pinned corpus and per-page numbers are in luisquintanilla/dotnet-document-extraction-samples validation/parsebench (luisquintanilla/dotnet-document-extraction-samples#3). Running it uncovered a separate bug, CommunityToolkit/AI#95: Document Intelligence fails on one page whose reported table cells overlap.

  3. duguankui commented on Nov 20, 2026

    @duguankui

    Thanks for the prototype. Feedback from Vault, a consumer with its own Markdown renderer, router and chunker, checked against #7963.

    GetText() vs HasText. Yes, it covers the gap. Our HasText reads a table cell only when the cell's content is a text element, so a section-valued cell reads as empty. We haven't hit it because the only engine behind that check (PdfPig) writes text-element cells, but a Document Intelligence-shaped tree would. Two differences: HasText also counts AlternativeText, and it tests for a letter or digit, because OCR engines do return whitespace-only blocks. The PR says "non-empty blocks"; is that IsNullOrEmpty? A whitespace-only block would add a blank line.

    Separator. Please use "\n". Vault stores the rendered text and keeps offsets into it (source spans, chunk locations, a hash) and runs on Windows and Linux. Environment.NewLine makes the same document produce different text per OS. Our renderer already joins with "\n".

    Headers and footers. Including them is fine for us as long as per-element filtering stays easy. For search and embeddings we exclude them: our Markdown has no furniture (Docling's export_to_markdown/export_to_text also default to the body layer only, and Document Intelligence writes them as comments in its Markdown). One thing for the README: EnumerateContent() + element.GetText() can't see the document's or a section's local payload, so document.GetText() is not the join of the per-element texts when a producer sets one. Our renderer treats a section's payload as a paragraph, though none of our engines set one.

    AlternativeText. In Vault it is the printed caption: the PDF engine binds a caption line to the figure, and we index it. A printed caption is document text; a model-generated description (what ImageAlternativeTextEnricher writes) isn't, and the contract can't tell them apart. Excluding by default is defensible, but the AlternativeText docs should say which one it means, and an overload for consumers who want captions would be welcome later.

    Anchor-only Cells. No impact. We read cells by indexer, and our two loops over Cells already tolerate null. Fine as a preview break; no need for a separate Anchors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-aiMicrosoft.Extensions.AI librariesuntriaged

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions