Skip to content

feat: Support Map type in approx_distinct - #23526

Merged
Jefffrey merged 2 commits into
apache:mainfrom
mkleen:map_approx_distinct
Jul 14, 2026
Merged

Jefffrey merged 2 commits into
apache:mainfrom
mkleen:map_approx_distinct

Conversation

@mkleen

@mkleen mkleen commented Jul 13, 2026 •

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

  • Support the Arrow type Map type for approx_distinct

What changes are included in this PR?

  • Enable HLLAccumulator and HllGroupsAccumulator to support Map
  • Tests for the non-grouped and grouped path as part of aggregate.slt

Are these changes tested?

Yes and results are compared to DuckDB:

D CREATE TABLE approx_distinct_map_test AS SELECT * FROM (VALUES
    (1, MAP {'a': 1, 'b': 2}), (1, MAP {'a': 1, 'b': 2}), (1, MAP {'c': 3}),
    (2, MAP {'d': 4}), (2, NULL),
    (3, NULL), (3, NULL),
    (4, MAP {'e': 5})
  ) AS t(g, m);

D SELECT approx_count_distinct(m) FROM approx_distinct_map_test WHERE g = 1;
┌──────────────────────────┐
│ approx_count_distinct(m) │
│          int64           │
├──────────────────────────┤
│            2             │
└──────────────────────────┘

D SELECT g, approx_count_distinct(m) FROM approx_distinct_map_test GROUP BY g ORDER BY g;
┌───────┬──────────────────────────┐
│   g   │ approx_count_distinct(m) │
│ int32 │          int64           │
├───────┼──────────────────────────┤
│     1 │                        2 │
│     2 │                        1 │
│     3 │                        0 │
│     4 │                        1 │
└───────┴──────────────────────────┘

D SELECT approx_count_distinct(m) FROM approx_distinct_map_test;
┌──────────────────────────┐
│ approx_count_distinct(m) │
│          int64           │
├──────────────────────────┤
│            4             │
└──────────────────────────┘

Are there any user-facing changes?

Yes, approx_distinct supports now Map but no breaking changes.

@github-actions github-actions Bot added sqllogictest SQL Logic Tests (.slt) functions Changes to functions implementation labels Jul 13, 2026
@mkleen
mkleen force-pushed the map_approx_distinct branch from 6eab1a7 to 8b235b7 Compare July 13, 2026 08:27
@mkleen
mkleen marked this pull request as ready for review July 13, 2026 08:28

@nuno-faria nuno-faria left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mkleen, LGTM.

@Jefffrey
Jefffrey added this pull request to the merge queue Jul 14, 2026
@Jefffrey

Copy link
Copy Markdown
Contributor

thanks @mkleen & @nuno-faria

Merged via the queue into apache:main with commit 84eaef9 Jul 14, 2026
38 checks passed
@mkleen
mkleen deleted the map_approx_distinct branch July 19, 2026 19:55
kosiew pushed a commit to kosiew/datafusion that referenced this pull request Aug 12, 2026
## Which issue does this PR close?

- Relates to apache#22989 but does
not close it. More types are coming.


## Rationale for this change

- Support the Arrow types `Map` type for `approx_distinct`

## What changes are included in this PR?

- Enable `HLLAccumulator` and `HllGroupsAccumulator` to support `Map`
- Tests for the non-grouped and grouped path as part of `aggregate.slt` 

## Are these changes tested?

Yes and  results are compared to DuckDB: 

```
D CREATE TABLE approx_distinct_map_test AS SELECT * FROM (VALUES
    (1, MAP {'a': 1, 'b': 2}), (1, MAP {'a': 1, 'b': 2}), (1, MAP {'c': 3}),
    (2, MAP {'d': 4}), (2, NULL),
    (3, NULL), (3, NULL),
    (4, MAP {'e': 5})
  ) AS t(g, m);

D SELECT approx_count_distinct(m) FROM approx_distinct_map_test WHERE g = 1;
┌──────────────────────────┐
│ approx_count_distinct(m) │
│          int64           │
├──────────────────────────┤
│            2             │
└──────────────────────────┘

D SELECT g, approx_count_distinct(m) FROM approx_distinct_map_test GROUP BY g ORDER BY g;
┌───────┬──────────────────────────┐
│   g   │ approx_count_distinct(m) │
│ int32 │          int64           │
├───────┼──────────────────────────┤
│     1 │                        2 │
│     2 │                        1 │
│     3 │                        0 │
│     4 │                        1 │
└───────┴──────────────────────────┘

D SELECT approx_count_distinct(m) FROM approx_distinct_map_test;
┌──────────────────────────┐
│ approx_count_distinct(m) │
│          int64           │
├──────────────────────────┤
│            4             │
└──────────────────────────┘
```

## Are there any user-facing changes?

Yes, `approx_distinct` supports now `Map` but no breaking changes.

Co-authored-by: Nuno Faria <nunofpfaria@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation sqllogictest SQL Logic Tests (.slt) v55.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants