Repository navigation
The data type of predicate in the row filter should be same in the binary expr #3469
Description
@alamb @thinkharderdev
I find a issue caused by this rewrite in my work #3396
if we have two parquet, one has c1,c2, the other has c1,c3.
If the filter is c2 = 1, then it will produce a expr NULL = 1 for the parquet with c1,c3 column.
After I remove the binary type coercion in the physical phase, the binary physical expr of NULL = INT(1) can't be created, because we don't need to support type coercion in creation physical expr.
Originally posted by @liukun4515 in #3380 (comment)
Activity
liukun4515 commentedon Sep 14, 2022
why not build the
build_row_filterfrom thephy expr?liukun4515 commentedon Sep 14, 2022
If the table schema is c1,c2,c3, the file schema is c1,c3 with filter c2 = lit.
Why not push the
predicate = falseto the parquet file directly to replace the c2 = NULLalamb commentedon Sep 14, 2022
seems like a reasonable thing to me
liukun4515 commentedon Sep 15, 2022
cc @thinkharderdev
alamb commentedon Sep 15, 2022
I think the more general flow here is to take a predicate like
c2 = litand a schema ofc1,c3and replacec2withNULL-- which is what the code changed in #3470 is.Then the predicate is
NULL = litwhich can be reduced tofalseif we run the constant evaluator on itchore(deps): bump pygithub from 2.8.1 to 2.9.1 (apache#24299)
chore(deps): bump pygithub from 2.8.1 to 2.9.1 (apache#24299)