Skip to content

AzureContentUnderstanding: ExtractAsync throws when a chart figure's content is a JSON string #86

Open

Description

@Laurianti

The Content Understanding provider fails on documents with some charts. The cause is a defect in Azure.AI.ContentUnderstanding 1.1.0, the version in Directory.Packages.props: Azure/azure-sdk-for-net#57370. The fix is proposed in Azure/azure-sdk-for-net#64660.

What happens

For some charts the service returns the content of a chart figure, the Chart.js config, as an escaped JSON string instead of an object. The service team confirmed it in Azure/azure-sdk-for-net#57370. The SDK cannot deserialize the result, so operation.Value in ContentUnderstandingExtractionClient.ExtractAsync throws, and no part of the document is returned:

System.InvalidOperationException: The requested operation requires an element of type 'Object', but the target element has type 'String'.
   at System.Text.Json.JsonElement.EnumerateObject()
   at Azure.AI.ContentUnderstanding.DocumentChartFigure.DeserializeDocumentChartFigure(JsonElement element, ModelReaderWriterOptions options)
   ...
   at Azure.Core.ProtocolOperationHelpers.ConvertOperation`2.get_Value()
   at CommunityToolkit.DocumentExtraction.AzureContentUnderstanding.ContentUnderstandingExtractionClient.ExtractAsync(Stream document, String identifier, String mediaType, DocumentExtractionOptions options, CancellationToken cancellationToken)

Only analyzers that return chart figures can hit it, such as prebuilt-documentSearch: in our run, its chart came back as an object. The default, prebuilt-document, returns no figures.

How it was checked

A test in AzureContentUnderstanding.UnitTests replays a real prebuilt-layout result whose figure is changed into a chart figure, as prebuilt-documentSearch returns it. The provider sources are the ones on main, on .NET 10.

  • With 1.1.0 and the chart content as an object, ExtractAsync returns the document.
  • With 1.1.0 and the chart content as a string, ExtractAsync throws the exception above.
  • With a build of the SDK that has the fix, both cases pass, and so do the other 136 tests of the project.

In the toolkit

When a release of Azure.AI.ContentUnderstanding includes the fix, the version in Directory.Packages.props can move to it, with a test on a chart figure whose content is a string.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions