Skip to content

Improve filter predicates with Utf8View literals #10998

Closed

Description

@alamb

Part of #10918

Is your feature request related to a problem or challenge?

DataFusion's predicate creation doesn't work as ideally for Utf8View constants

Specifically, I added a test i #10997

explain SELECT column1_utf8 from test where column1_utf8 = arrow_cast('Andrew', 'Utf8View');
logical_plan
01)Filter: CAST(test.column1_utf8 AS Utf8View) = Utf8View("Andrew")
02)--TableScan: test projection=[column1_utf8]

This plan is non ideal as it casts the column rather than the constant

Describe the solution you'd like

Instead of

CAST(test.column1_utf8 AS Utf8View) = Utf8View("Andrew")

the filter should be

test.column1_utf8 =  Utf8("Andrew")

So the expected plan should look like

explain SELECT column1_utf8 from test where column1_utf8 = arrow_cast('Andrew', 'Utf8View');
logical_plan
01)Filter: test.column1_utf8 = Utf8("Andrew")
02)--TableScan: test projection=[column1_utf8]

Describe alternatives you've considered

No response

Additional context

No response

Activity

  1. doki23 commented on Jun 27, 2024

    @doki23
    Contributor

    I wonder why #11043 isn't able to close this issue.

  2. alamb commented on Jun 27, 2024

    @alamb
    ContributorAuthor

    I wonder why #11043 isn't able to close this issue.

    I agree -- #11043 should have closed it -- thank you @doki23

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions