Repository navigation
Support pruning on string columns using LIKE #507
Description
Activity
- changed the title
[-]Support pruning on string columns using starts_with[/-][+]Support pruning on string columns using LIKE[/+]on Jan 6, 2025 Filed the following ticket to support
starts_with: 🎣Reacted by Adrian Garcia BadaraccoI think we also need follow up tickets for:
- NOT LIKE
- Case insensitive matching
I think we also need follow up tickets for:
- NOT LIKE
- Case insensitive matching
Sounds good -- can you please file them (and the more hints you leave in the ticket the more likely it is for someone else to be able to do it)?
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
From looking at the pruning logic, it appears that we can prune inputs based on
col = 'string_value', but we can't prune oncol LIKE 'value%'.We should be able to support
LIKEandNOT LIKE.Describe the solution you'd like
Extend the pruning logic to generate a pruning filter for
LIKEandNOT LIKE.Describe alternatives you've considered
None
Additional context
Spark will push down a
LIKEfilter if it can be converted to astarts_with, normally by checking if the string doesn't start with%.