Add mask functions to allow gandiva evaluation.

[lriggs]

DX-117112: HiveMaskConvertlets bypasses Gandiva mask functions, forcing Java execution

Why

Dremio's HiveMaskConvertlets rewrites every SQL MASK* call into mask_internal before
function resolution. mask_internal had no native implementation, so all string masking ran in Java and every MASK* split reported Split Evaluated in Gandiva: false.
Gandiva's existing mask / mask_first_n / mask_last_n / mask_show_first_n /
mask_show_last_n were unreachable dead code.

Redirecting the convertlet to those existing natives was the obvious fix and turns out to be
wrong: they do not compute the same thing as Hive. This PR makes the native functions match
Hive and adds the one signature Dremio actually emits.

Changes

1. Only Lu, Ll and Nd are masked

Both mask families now mask uppercase letters, lowercase letters and decimal digits, and pass
every other Unicode general category through — matching Hive's GenericUDFMaskBase and
Dremio's MaskTransformer.

• gdv_mask_first_n_utf8_int32 / gdv_mask_last_n_utf8_int32 dropped an undocumented
  case 10 (Nl) and now use named UTF8PROC_CATEGORY_* enums instead of bare integers.
• mask_utf8_utf8_utf8_utf8 dropped Lt, Lo, Nl and No.

This is a bug fix, not a preference. Three pieces of evidence:

• The two families disagreed with each other, and both contradictions were pinned by
  passing tests: gdv_function_stubs_test.cc asserted mask_first_n("世界您", 4) leaves CJK
  untouched, while projector_test.cc asserted mask("A的Ççd-123") masks 的 to x.
• The only written statement of intent anywhere in the tree is ARROW-17070's commit message —
  "'Masking' according to Hive specification (a-z : x, A-Z : X, 0-9 : n)" — which the code
  did not implement. docs/ has nothing, and neither the registrations nor the declarations
  carry comments.
• Provenance explains the drift: three commits, two authors, seven months apart, no shared
  spec. ARROW-14482 added the windowed family with the bare-integer switch; ARROW-17070 added
  the show_* wrappers citing Hive but delegating to it; ARROW-17121 added mask() the next
  day and deliberately expanded to Lt/Lo/Nl/No. {1,2,9,10} is exactly the adjacent
  initcap helper's category set minus LT, which is the likely origin of the stray Nl.

Lo is the widest-reaching: CJK, Japanese, Korean, Hebrew, Arabic, Thai, Devanagari. Cased
scripts (Cyrillic, Greek, accented Latin) were already correct and are unaffected.

2. An empty replacement argument means "use the default", not "delete"

Also matching getCharArg. This additionally fixes an under-allocation: the output buffer was
sized max(upper_len, lower_len, num_len) * data_len, which is 0 when all three arguments
are empty even though pass-through characters still get written. SimpleArena::Allocate(0)
returns the arena cursor without advancing it, so those bytes landed in space the next row's
allocation would reuse — silent cross-row corruption in a batch rather than a crash, which is
why the previous test asserted the truncated ":)" and passed.

3. New: mask with otherChar, and native mask_internal

mask(utf8, utf8, utf8, utf8, utf8) -> utf8
    text, upper, lower, digit, other

mask_internal(utf8, utf8, int32, utf8, utf8, utf8, utf8) -> utf8
              text, mode, char_count, upper, lower, digit, other

The 1–4 argument mask overloads now delegate to a shared mask_impl with a null other,
meaning pass-through, so their behavior is unchanged apart from change 1.

mask_internal is a single entry point for all five Hive modes (FULL, FIRST_N, LAST_N,
SHOW_FIRST_N, SHOW_LAST_N) with caller-supplied replacements. It implements the full
getCharArg contract: an empty argument takes the per-class default, an argument parsing to
-1 leaves that class unmasked, and anything longer is truncated to its first character.
Character counts are in codepoints.

This is what lets Dremio vectorize the shapes that actually matter. Apache Ranger's built-in
mask types generate mask_show_last_n(col, 4, 'x', 'x', 'x', -1, '1') and
mask_show_first_n(...) — custom replacement characters, which the (utf8, int32) natives
with hardcoded X/x/n cannot express.

Behavior changes to existing functions

expression	before	after
mask('Dž') — Lt	X	Dž
mask('世'), mask('ㅏ') — Lo	x	世, ㅏ
mask('Ⅷ') — Nl	n	Ⅷ
mask('½'), mask('²') — No	n	½, ²
mask_first_n('Ⅷ', 1) — Nl	n	Ⅷ
mask(s, '', '', '')	deletes matched characters	masks with X/x/n

No change for ASCII, or for any Lu/Ll/Nd character in any script.

Four expectations in projector_test.cc were updated accordingly (TestMaskAll,
TestMaskUpperLower, TestMaskUpper, TestMaskDefault), plus one in
gdv_function_stubs_test.cc for the empty-argument change. Those five are the entire blast
radius — every other mask assertion was already Hive-compatible.

Tests

gdv_function_stubs_test.cc:

• TestMaskInternalModes — all five modes, empty input, unknown-mode error, case sensitivity
• TestMaskInternalCharCountBoundaries — INT32_MIN, negative, 0, at length, past length,
  INT32_MAX, across all four windowed modes
• TestMaskInternalReplacementArguments — custom replacements, the -1 spelling and its
  -01 variant, -2 as an ordinary character, multi-character truncation, empty-means-default,
  and the Ranger MASK_SHOW_LAST_4 shape
• TestMaskInternalUnicode — the category matrix, codepoint-counted windows, multi-byte
  replacements, truncated UTF-8
• TestMaskOtherChar — the new 5-argument overload through both the ASCII fast path and the
  utf8proc path, including mask('王小明', …, '*') => "***"
• TestMaskUnicodeCategories — one case per disputed category in both families, plus a
  cross-family consistency assertion that mask and mask_first_n(…, len) now agree

tests/projector_test.cc:

• TestMaskInternal — end to end in the shape an engine emits: text column plus literal mode,
  char_count and replacements. This is what exercises the generated call, so it is the real
  check on the 15-parameter stub mapping.
• TestMaskOtherChar — the 5-argument overload end to end
• TestMaskNullInput — closes a gap: none of the six existing mask projector tests had a
  single false in its validity vector, so kResultNullIfNull was untested

187 gandiva-internals-test and 249 gandiva-projector-test cases pass.

cmake --build cpp/debug --target gandiva-internals-test gandiva-projector-test -j 8
cpp/debug/debug/gandiva-internals-test --gtest_filter='TestGdvFnStubs.*Mask*'
cpp/debug/debug/gandiva-projector-test --gtest_filter='TestProjector.*Mask*'

Notes for reviewers

Known divergence from the Java implementation. Character counts are in codepoints, while
CharSequenceWrapper counts UTF-16 code units. This is visible only when a char_count
boundary falls inside a surrogate pair — mask_first_n('🌍Abc', 2) gives 🌍Xbc here and
🌍Abc in Java. FULL mode has no char_count and is unaffected. Documented in the stub.

Not covered. mask_internal for int32, int64 and date. Those keep running in Java,
including Ranger's MASK_DATE_SHOW_YEAR. The int path is a handful of integer divisions and
the date path three field assignments, so the vectorization win is much smaller than for
strings; worth adding on evidence rather than speculatively.

Unrelated pre-existing issue this does not address. Because only Lu/Ll/Nd are masked
and the other default is pass-through, mask('王小明') returns the input unchanged. Ranger's
built-in "Redact" policy is plain mask(col), so a redaction policy over uncased-script data
provides no protection. That is Hive-compatible behavior and is tracked separately; changing it
here would have meant diverging from Hive in a masking function, which is worse than a
documented gap.

Dremio-side change required

Only the function-list-gandiva.csv golden, which gains two lines:

mask,GandivaFunctionHolder [functionName=mask  returnType=varchar  parameters=[varchar  varchar  varchar  varchar  varchar]]
mask_internal,GandivaFunctionHolder [functionName=mask_internal  returnType=varchar  parameters=[varchar  varchar  int32  varchar  varchar  varchar  varchar]]

No convertlet change, no new support key and no new Java functions: mask_internal is already
what the convertlet emits and already has a Java implementation, so GandivaPushdownSieve
starts pushing it down on signature match alone, and every degraded path — constant folding, a
Gandiva-less build, exec.disabled.gandiva-functions, oversized CASE — still falls back to
the existing Java mask_internal.