Skip to content

Ingestion: Snowflake nested column support (structured-type children + DQ dialect) #36383

Open

Description

@TeddyCr

Parent epic: #36278 — follow-up to #36382

Snowflake needs two separate chunks of work before DQ tests can run on nested columns:

Part 1 — Ingestion metadata

Snowflake imports StructuredType (snowflake/metadata.py:163,167) but never populates Column.children. Parse structured OBJECT/ARRAY types recursively into child columns so nested paths exist on the entity.

Part 2 — DQ dialect spike + interface

Dialect spike first, before writing the interface: prove how nested access compiles with sqlalchemy-snowflake structured types (path vs :variant syntax, quote behavior). Then add the nested override to data_quality/interface/sqlalchemy/snowflake/. Record findings in the supporting ADR.

Verification

Test with a real Snowflake instance for part 2; ingestion unit tests for children parsing.

Metadata

Metadata

Assignees

Type

No type

Projects

  • Status
    No status
  • Status
    Release Backlog 🚧

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions