Repository navigation
Datafusion 48 Clickbench Q6 and Q0 regression #16444
Description
Describe the bug
When upgrading to Datafusion 48 our continous benchmarking infra detected a 25x regression in Q6 and 10x in Q0 of Clickbench. This query was previously answered all from stats which might not longer be the case if the slowdown is this significant
To Reproduce
No response
Expected behavior
No response
Additional context
No response
Activity
AdamGS commentedon Jun 18, 2025
I think this behavior is expected, it's explicitly called out in #16080.
AdamGS commentedon Jun 18, 2025
There's also an issue (#16158) to change the default config, I'll take it.
robert3005 commentedon Jun 18, 2025
One thing that's not clear to me is whether statistics will be fetched if you have a query that could be ansewered by statistics or will they not and you will run the full query
AdamGS commentedon Jun 18, 2025
For
ListingTableas it is now - the only thing taken into account is the config value.blaginin commentedon Jun 19, 2025
#16447 is merged so closing this
alamb commentedon Jun 20, 2025
To be clear, I think the PR that changed this was:
SessionContext::register_parquetobeycollect_statisticsconfig #16080