Skip to content

Data files persisted without complex columns in the table schema hit "unexpected target column" on read #2618

Closed

Description

@jordepic

Apache Iceberg Rust version

None

Describe the bug

If we write a data file and then evolve our table's schema to add a complex type field like list, map, struct, proceeding to read the data file back will error out.

To Reproduce

  1. Create a table and write a data file (without column xs).
  2. Evolve the schema to add xs list (or a map/struct column).
  3. Read the table.
  4. Reading the older file errors with unexpected target column type List(...).

Expected behavior

A column present in the table schema but absent from a data file is filled with a typed NULL array of the correct (possibly nested) Arrow type via new_null_array, for nested types just as for primitives. The read should succeed with the added column all-NULL for older files.

Willingness to contribute

None

Activity

  1. added a commit that references this issue on Jun 14, 2026
    66d84e0
  2. added a commit that references this issue on Jun 30, 2026
  3. added a commit that references this issue on Oct 6, 2026
    f2409e9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions