Repository navigation
Description
Summary
While writing the introductory blog post and samples for DocumentExtractionClient, I hit a usability problem with the shared document model in Microsoft.Extensions.Documents:
- The obvious loop (
foreach section in document.Sections→foreach element in section.Elements) compiles, runs, and returns a result that looks valid, but it silently skips content whenever a provider nests sections. - Whether content is nested depends on the provider. On the same page, PdfPig and Mistral return flat documents, while Content Understanding and Document Intelligence return nested sections. Code tested against one provider can lose most of its output after a provider swap, which is the scenario the abstraction is meant to make safe.
- Even the recommended
EnumerateContent()does not reach table cell text. Every table cell'sContentis anExtractionDocumentSection, and there is no built-in way to walk a section. In our samples, the table held more text than a typical consumer can reach.
The tree model itself is right, and I'm not proposing to flatten it. I am proposing:
- Make the safe path the obvious path: clearer docs,
EnumerateContent()on sections, and a built-in text projection. - Tighten one enumerator:
Cellscurrently yieldsnullentries.
All of this is cheap to do while the API is in preview.
Tested with Microsoft.Extensions.Documents.Abstractions / Microsoft.Extensions.DocumentExtraction 11.0.0-preview.1.26560.3 and the .NET AI Community Toolkit providers.
What I observed
The input is one page with headings, paragraphs, a 6×5 table with merged cells, and a figure (tokens-explained-hero.pdf). I sent the same bytes to each provider and measured four things:
- Two-level loop: the obvious nested
foreach. EnumerateContent(): the elements it returns, counting characters from each element's ownTextContent.- Table cell text: characters reachable only through
table.Cells→ cell section →Elements. - Section depth: how deeply the provider nested sections.
| Provider | Root / total sections | Max depth | Two-level loop | EnumerateContent() |
Table cell text |
|---|---|---|---|---|---|
| PdfPig | 1 / 1 | 1 | 35 elements, 1,035 chars | 35 elements, 1,035 chars | n/a (no tables) |
| Mistral OCR | 1 / 1 | 1 | 3 elements, 1,285 chars | 3 elements, 1,285 chars | n/a (no tables) |
| Azure Content Understanding | 1 / 2 | 2 | 4 elements, 88 chars | 13 elements, 666 chars | 447 chars, 26 cells |
| Azure Document Intelligence | 1 / 4 | 3 | 4 elements, 137 chars | 13 elements, 666 chars | 447 chars, 26 cells |
Findings:
- The flat providers hide the bug. A two-level loop is correct for PdfPig and Mistral, so a developer who starts locally with PdfPig (the zero-setup path in our samples) writes this loop, tests it, and ships it.
- After a swap to a cloud provider, the same loop loses most of the page. It reaches 4 of 13 elements and 13–21% of the text that
EnumerateContent()reaches. Nothing throws or warns, and the result is plausible-looking text. EnumerateContent()doesn't include tables either. It returns theExtractionDocumentTableElement, whose ownContentis empty on both Azure providers. All 26 cells hold anExtractionDocumentSection, and each section contains one text element. A consumer that readsContentfrom each enumerated element gets 666 characters. The 447 characters in the table, often the most valuable data on the page, are reachable only through a custom recursive walk.table.Cellsyieldsnullfor covered positions. Enumerating the 6×5 grid produced 30 items: 26 anchor cells and 4nulls. A naturalforeach (var cell in table.Cells) cell.RowIndex...throws aNullReferenceException. (The indexer remarks documentnullfor covered positions, but the enumerator behavior surprised me.)
Minimal repro
ExtractionDocument document = await client.ExtractAsync(stream, "page.pdf", "application/pdf");
// 1. The obvious loop: correct for flat providers, silently incomplete for nested ones.
foreach (var section in document.Sections)
{
foreach (var element in section.Elements)
{
if (element is ExtractionDocumentTextElement { Content: TextContent text })
{
Console.WriteLine(text.Text);
}
// A nested ExtractionDocumentSection lands here and is skipped.
}
}
// 2. The recommended walk: reaches every content element, but not table cell text.
foreach (var element in document.EnumerateContent())
{
if (element is ExtractionDocumentTableElement table)
{
foreach (var cell in table.Cells)
{
// cell can be null (covered position).
// cell.Content is an ExtractionDocumentSection: there is no built-in way to get its text.
}
}
}The full probe that produced the table is in the appendix.
Why this matters
- It breaks the core promise. The abstraction exists so you can swap providers without rewriting extraction code. Today, provider-dependent shapes plus a loop that silently drops content mean a provider swap can quietly degrade RAG quality, summaries, or data extraction, with no error to catch.
- The failure is silent and plausible. You get some text, headings included, so it looks right in a quick test. In RAG, the missing content shows up later as worse answers, and nobody traces it back to the loop.
- Everyone writes the same helper. Our samples needed a recursive
Leaves(element)walker, aTextOf(element)helper that joins section text (because "table cells hold a section too"), and anullcheck in every cell loop. The Data Ingestion bridge I wrote also walks nested sections and cell sections manually. If every consumer writes these helpers, some will get them wrong, and the library is the right place to provide them. - Tables are where users get value. The capability that sets layout-aware providers apart (tables with merged cells and headers) is the part that is hardest to read through the API.
What's right today (keep it)
- The tree model. Nesting preserves heading → subheading structure, which chunkers, Markdown rendering, citations, and navigation need. Flattening loses information that can't be recovered.
Contentis local, not aggregated. This avoids duplicated text and keeps mutation predictable. Any aggregation should be computed, not stored.EnumerateContent()as a named method instead ofIEnumerable. This was the right call in [API Proposal]: DataIngestion: Document representation #6893, for discoverability and to document that sections aren't returned.- Consistency with
IngestionDocument. With Integrate shared Documents while retaining IngestionDocument #7904, MEDI forwardsSectionsandEnumerateContent()to the shared model, so any change here lands in both places. That's good, and it raises the stakes for getting it right before GA.
Options considered
A. Documentation only
Change the Sections summary from "Gets the ordered root sections of the document." to something like "Gets the root sections. Sections can nest, and providers differ in how deeply. To visit all content, use EnumerateContent()." Do the same for ExtractionDocumentSection.Elements, and add a README section titled "Reading a document" that covers the tree, the provider differences, and tables.
- Pros: Not breaking, nearly free, and it helps right away in IntelliSense.
- Cons: It doesn't fix table cells, and it doesn't remove the need for every consumer to write a recursive helper. Docs alone rarely prevent a loop that looks obviously correct.
B. Add EnumerateContent() to ExtractionDocumentSection
The #6893 review concluded this "isn't needed at this time." The table-cell shape is new evidence: every cell's content is a section, so reading a cell requires walking a section.
- Pros: Small, additive, and symmetric with the document method. It makes
cell.Contentusable as((ExtractionDocumentSection)cell.Content).EnumerateContent()without custom code, and it lets a chunker process one subtree. - Cons: Adds a little surface. Cell content is typed
ExtractionDocumentElement, so callers still need a type check. A helper on the cell (option D) would hide that.
C. Built-in text projection (computed, not stored)
Add a method that renders the plain text of an element and everything below it, including table cells. For example, GetText() on ExtractionDocument and ExtractionDocumentElement, or extension methods in the same namespace.
- Pros: Covers the most common need (text for a prompt, an embedding, or a search index) in one call that can't silently drop content. It's consistent with "content is local", because the aggregate is computed on demand. It mirrors the
GetMarkdown()aggregation thatIngestionDocumentElementhad in [API Proposal]: DataIngestion: Document representation #6893, which the shared model doesn't carry over. - Cons: It requires policy decisions: separators between elements and cells, whether to include image OCR text or alt text, and whether to include page headers and footers. A Markdown renderer is even more opinionated (merged cells, heading levels), so it might belong in Data Ingestion rather than in the abstractions.
D. Cell enumeration fixes
- Make
ExtractionDocumentTableCellCollection's enumerator yield only anchor cells, nevernull. Covered positions remain reachable through the indexer,GetCoveringCell, andIsCovered. - Optionally add a cell-level text helper,
cell.GetText(), as part of C. - Pros: Removes a
NullReferenceExceptiontrap. Anchor-only enumeration matches how people think about cells ("each merged cell appears once"). - Cons: A behavior change. Anyone who relies on positional enumeration would need to switch to the indexer, but that is unlikely in a preview.
E. Rename Sections to RootSections
- Pros: The name itself explains the shape.
- Cons: It's breaking. It also diverges from
IngestionDocument.Sections, so it would need to be renamed in both places, coordinated with Integrate shared Documents while retaining IngestionDocument #7904. On its own, a rename still leaves consumers writing a recursive helper. The benefit is small next to A–D.
F. Make the shape consistent across providers (for example, always flat, or always nested)
- Pros: Code written against one provider would behave the same on all of them.
- Cons: Nesting reflects real capability differences: Document Intelligence returns deeper hierarchy than Content Understanding, and PdfPig has none. Forcing one shape either discards structure or invents it. The abstraction should make provider differences safe to consume, not hide them.
G. Analyzer for nested Sections/Elements loops
- Pros: Catches the bug at compile time.
- Cons: It's a heuristic that is hard to make precise (legitimate code inspects one level on purpose) and costs more to maintain. It's worth reconsidering only if A–D don't move the needle.
H. Flatten the model
Not recommended. It loses hierarchy, conflicts with "content is local", and would undo #6893 and #7904.
Recommendation
In priority order, all before GA:
| Priority | Change | Breaking? |
|---|---|---|
| P0 | A. Docs on Sections, Elements, and Cells, plus a README "Reading a document" section that shows the provider differences and the right loops |
No |
| P0 | B. ExtractionDocumentSection.EnumerateContent() |
No (additive) |
| P0 | D. Cells enumerator yields anchor cells only |
Behavior change (preview) |
| P1 | C. Computed plain-text projection for the document, elements, and cells; leave Markdown rendering to Data Ingestion or a separate helper | No (additive) |
| P1 | Conformance: give provider authors a test fixture that has nested sections and section-valued cells, so consumer guidance and samples are tested against both shapes | No |
| Skip for now | E (rename), F (normalize shape), G (analyzer), H (flatten) | n/a |
Sketch of the additive API (names are placeholders):
namespace Microsoft.Extensions.Documents;
public sealed class ExtractionDocumentSection : ExtractionDocumentElement
{
public List<ExtractionDocumentElement> Elements { get; }
// New: depth-first, non-section content of this subtree, same semantics as ExtractionDocument.EnumerateContent().
public IEnumerable<ExtractionDocumentElement> EnumerateContent();
}
public static class ExtractionDocumentTextExtensions
{
// New: computed (never stored) plain text of the node and all descendants, including table cells
// in row-major order, each merged cell once. Policy for separators and image text documented.
public static string GetText(this ExtractionDocument document);
public static string GetText(this ExtractionDocumentElement element);
public static string GetText(this ExtractionDocumentTableCell cell);
}
public sealed class ExtractionDocumentTableCellCollection : IEnumerable<ExtractionDocumentTableCell>
{
// Changed: enumerates anchor cells only (no nulls). Positional access stays on the indexer.
public IEnumerator<ExtractionDocumentTableCell> GetEnumerator();
}With these changes, the code in the repro becomes:
string text = document.GetText(); // everything, including table cells
foreach (var element in document.EnumerateContent())
{
if (element is ExtractionDocumentTableElement table)
{
foreach (var cell in table.Cells) // never null
{
Console.WriteLine($"[{cell.RowIndex},{cell.ColumnIndex}] {cell.GetText()}");
}
}
}Open questions
- Should
GetText()live on the abstractions (for every consumer) or in a helpers package? I lean toward the abstractions, because a correct traversal is part of the contract. - What should the default text policy be for images (OCR
TextversusAlternativeText) and forpageHeader/pageFooterroles? I suggest including OCR text, excluding alt text, and including headers and footers, with an options overload later if needed. - Should
EnumerateContent()take an option to descend into table cells? I lean toward no: keep it structural and letGetText()and cell helpers handle cell content. - Does Integrate shared Documents while retaining IngestionDocument #7904 change any of this? It forwards
SectionsandEnumerateContent(), so the additions would benefit MEDI automatically. TheCellsbehavior change should land in the same preview.
Appendix: probe
Probe source (C# file-based app) and output
#!/usr/bin/env dotnet
// Run from the root of luisquintanilla/dotnet-document-extraction-samples.
#:include common/Providers.cs
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Documents;
foreach (var p in args)
{
using var extractor = Providers.Create(p, out _);
var doc = await Samples.ExtractFileAsync(extractor!, Path.Combine("books", "pages", "tokens-explained-hero.pdf"));
static int Chars(ExtractionDocumentElement e) => e.Content is TextContent t ? t.Text.Trim().Length : 0;
static int Deep(ExtractionDocumentElement e) => Chars(e) + (e is ExtractionDocumentSection s ? s.Elements.Sum(Deep) : 0);
var naive = doc.Sections.SelectMany(s => s.Elements).Where(e => e is not ExtractionDocumentSection).ToList();
var all = doc.EnumerateContent().ToList();
var tables = all.OfType<ExtractionDocumentTableElement>().ToList();
Console.WriteLine($"{p}: two-level {naive.Count} elements/{naive.Sum(Chars)} chars; " +
$"EnumerateContent {all.Count}/{all.Sum(Chars)}; " +
$"cell text {tables.Sum(t => t.Cells.Sum(c => c?.Content is null ? 0 : Deep(c.Content)))} chars; " +
$"null cells {tables.Sum(t => t.Cells.Count(c => c is null))}");
}pdfpig: rootSections=1 totalSections=1 maxDepth=1
two-level 35 / 1035 chars; EnumerateContent 35 / 1035 chars (Text:35)
cu: rootSections=1 totalSections=2 maxDepth=2
two-level 4 / 88 chars; EnumerateContent 13 / 666 chars (Text:11, Table:1, Image:1)
table 6x5: 26 cells, 4 nulls from enumerator, every cell.Content is a section; cell text 447 chars
docintel: rootSections=1 totalSections=4 maxDepth=3
two-level 4 / 137 chars; EnumerateContent 13 / 666 chars (Text:11, Table:1, Image:1)
table 6x5: 26 cells, 4 nulls from enumerator, every cell.Content is a section; cell text 447 chars
mistral: rootSections=1 totalSections=1 maxDepth=1
two-level 3 / 1285 chars; EnumerateContent 3 / 1285 chars (Text:2, Image:1)
Related: #6893 (original document representation and the EnumerateContent decision), #7587 (DocumentExtractionClient proposal), #7904 (shared Documents model in MEDI), #7961 (partial-extraction errors).
Activity
luisquintanilla commentedon Nov 19, 2026
A few refinements after reviewing this against the goal of keeping
Microsoft.Extensions.Documentsa set of flexible exchange types rather than an implementation contract.None of the recommendations constrain providers.
ExtractionDocumentSection.EnumerateContent(), and anchor-onlyCellsenumeration are pure functions of, or ergonomics fixes to, the exchange types.The burden lands on consumers. Every freedom the contract gives producers is a case every consumer has to handle:
This is the same pattern as
AIContentin Microsoft.Extensions.AI. There,ChatMessage.TextandChatResponse.Textgive consumers a shape-agnostic way to be correct without constraining producers. That's the precedent for the additions proposed here.With that lens, three corrections to the original post:
GetText()policy minimal. Open question 2 suggested including OCR text and page headers and footers, and excluding alt text. That's more opinion than an exchange type should carry. MirrorChatMessage.Textinstead: concatenateTextContentin document order, including table cells (each anchor cell once), with a simple documented separator. Leave richer rendering (Markdown, image text policy, header/footer filtering) to consumers or Data Ingestion.ExtractionDocumentTableCell.Contentis typed as anyExtractionDocumentElement. The Content Understanding and Document Intelligence providers in the .NET AI Community Toolkit choose to wrap cell text in a section. That's valid, because a cell can hold several paragraphs. The helper is still needed, because consumers have to handle any legal shape, but the shape itself comes from the providers. If provider-side changes turn out to be worthwhile, that's a separate conversation in CommunityToolkit/AI.ExtractionDocument.ContentandExtractionDocumentSection.Contentcan carry text, andEnumerateContent()intentionally skips them (per its remarks). None of the providers I tested populate them on this page, but the contract allows it. SoGetText()should include document and section local payloads in order, or the "can't silently drop content" goal doesn't hold.luisquintanilla commentedon Nov 19, 2026
I prototyped the proposal and measured it against live providers.
Setup. Ten hard single-page PDFs from ParseBench-2: charts, complex table headers, dense and financial tables, a form, headings, layout and CJK text. Each page was extracted with the five providers in the samples repo, and each result was read five ways. The number is the share of the page's ground-truth words that each way returns.
EnumerateContent+is TextElementdocument.GetText()Compare within a row, not across rows; this isn't a provider ranking.
GetText()reads 99%.The prototype is in #7963. The harness, pinned corpus and per-page numbers are in luisquintanilla/dotnet-document-extraction-samples
validation/parsebench(luisquintanilla/dotnet-document-extraction-samples#3). Running it uncovered a separate bug, CommunityToolkit/AI#95: Document Intelligence fails on one page whose reported table cells overlap.duguankui commentedon Nov 20, 2026
Thanks for the prototype. Feedback from Vault, a consumer with its own Markdown renderer, router and chunker, checked against #7963.
GetText() vs HasText. Yes, it covers the gap. Our
HasTextreads a table cell only when the cell's content is a text element, so a section-valued cell reads as empty. We haven't hit it because the only engine behind that check (PdfPig) writes text-element cells, but a Document Intelligence-shaped tree would. Two differences:HasTextalso countsAlternativeText, and it tests for a letter or digit, because OCR engines do return whitespace-only blocks. The PR says "non-empty blocks"; is thatIsNullOrEmpty? A whitespace-only block would add a blank line.Separator. Please use "\n". Vault stores the rendered text and keeps offsets into it (source spans, chunk locations, a hash) and runs on Windows and Linux.
Environment.NewLinemakes the same document produce different text per OS. Our renderer already joins with "\n".Headers and footers. Including them is fine for us as long as per-element filtering stays easy. For search and embeddings we exclude them: our Markdown has no furniture (Docling's
export_to_markdown/export_to_textalso default to the body layer only, and Document Intelligence writes them as comments in its Markdown). One thing for the README:EnumerateContent()+element.GetText()can't see the document's or a section's local payload, sodocument.GetText()is not the join of the per-element texts when a producer sets one. Our renderer treats a section's payload as a paragraph, though none of our engines set one.AlternativeText. In Vault it is the printed caption: the PDF engine binds a caption line to the figure, and we index it. A printed caption is document text; a model-generated description (what
ImageAlternativeTextEnricherwrites) isn't, and the contract can't tell them apart. Excluding by default is defensible, but theAlternativeTextdocs should say which one it means, and an overload for consumers who want captions would be welcome later.Anchor-only Cells. No impact. We read cells by indexer, and our two loops over
Cellsalready tolerate null. Fine as a preview break; no need for a separateAnchors.