You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
rlike: consider arrow regexp_is_match kernel and fix non-StringArray panic #5102
RLike::is_match in native/spark-expr/src/predicate_funcs/rlike.rs loops over a StringArray calling regex::Regex::is_match per row into a BooleanBuilder. Arrow's regexp_is_match kernel does the same thing: it is built on the same Rust regex crate, so the documented divergence from Java regex semantics is unchanged, and null propagation matches.
The kernel would also make the array path more robust: it supports Utf8/LargeUtf8/Utf8View, while the current code hard-expects a StringArray and would panic on other string layouts.
Describe the potential solution
Replace the row loop with regexp_is_match(array, &StringArray::new_scalar(pattern), None).
Trade-offs to resolve:
The struct currently pre-compiles the Regex once at plan build; the kernel takes a pattern string and compiles per batch. For complex patterns this is a measurable regression; there is no arrow API accepting a pre-built Regex. Either accept per-batch compilation or keep the loop and only fix the downcast robustness.
The dictionary fast path (run on dictionary values then take) and the scalar-input branch stay as wrappers.
Additional context
Found during an audit of native code that replicates existing arrow-rs kernels.
Activity
sam-1112 commentedon Aug 1, 2026
take
fix: support Utf8/LargeUtf8/Utf8View in native RLike without panicking (
fix: support Utf8/LargeUtf8/Utf8View in native RLike without panicking (