Repository navigation
Reduce the number of distinct binaries to reduce CI / compile time #18142
Description
Activity
take
Reacted by Alex Huang- added a commit that references this issue
on Nov 4, 2025 - added a commit that references this issue
on Nov 5, 2025 - added a commit that references this issue
on Nov 6, 2025 - added a commit that references this issue
on Nov 7, 2025 - added a commit that references this issue
on Nov 8, 2025 - added a commit that references this issue
on Nov 8, 2025 - added a commit that references this issue
on Nov 9, 2025 13 remaining items
- added a commit that references this issue
on Nov 20, 2025 - added a commit that references this issue
on Nov 21, 2025 - added 5 commits that reference this issue
on Nov 23, 2025 @alamb As far as I can tell, the work described in this issue has been completed and the related changes have already been merged. I don’t see any remaining open tasks here.
If everything is covered, perhaps this issue can be closed? If there’s still work to be done, I’d be happy to continue and help move it forward.
I agree - this was epic work. Thank you @cj-zhukov 🙏
Reacted by Sergey Zhukov
Is your feature request related to a problem or challenge?
We have been running out of diskspace when running CI jobs
For example
Rust / cargo examples (amd64) (pull_request)failing on branch-50 #18135Part of this is due to the fact that each binary that is created (example, test, etc) gets an entire
copy of the DataFusion codebase compiled into it.
For example, on my machine, after running
./ci/scripts/rust_example.sheach example binary is 50-60MB in size:(venv) andrewlamb@Andrews-MacBook-Pro-3:~/Software/datafusion$ du -s -h target/ci/examples/ 52M target/ci/examples/tracing 52M target/ci/examples/simple_udf 55M target/ci/examples/thread_poolsDescribe the solution you'd like
Thus I think it will be helpful to reduce the number of examples
Describe alternatives you've considered
Two options, depending on the example:
Tasks