Repository navigation
[Discuss] Release cadence / patch releases / Long Term Supported (lts) minor releases #5269
Description
Activity
Is there a risk this would encourage people to only update once every 3 months? I'm not sure we're at a level of API stability where performing 3 months of upgrades in one go wouldn't be a fraught experience?
Is there a risk this would encourage people to only update once every 3 months? I'm not sure we're at a level of API stability where performing 3 months of upgrades in one go wouldn't be a fraught experience?
Yes, that is definitely a risk.
- changed the title
[-]Discuss release cadence / patch releases / LTS[/-][+][Discuss] Release cadence / patch releases / Long Term Supported (lts) minor releases[/+]on Dec 20, 2024 I think the upgrade situation is better than it was previously. For example we now document major upgrades and
However, it still requires lots of work
This has came up on the sync call today. 52 specifically has caused us a bunch of issues at InfluxData (our customers found several regressions on production in our InfluxDB cloud instance) and the fact that we have needed multiple patchsets I think also speaks to challenge of this release.
- Release DataFusion 52.1.0 (minor/patch) Release (Jan 2026) #19784
- Release DataFusion 52.2.0 (minor/) Release (Feb 2026) #20287
- Release DataFusion 52.3.0 (minor/) Release (Mar 2026) #20681
- Release DataFusion 52.4.0 (minor/) Release (Mar 2026) #20855
Thus I know I would personally be willing to expend energy to help make a LTS version happen
I think there's an opportunity for a 52 postmortem when all is said and done, so I don't want to conflate the discussion too much. That said, from my perspective there were some stars that might have aligned to make 52 a tough release from which we're still recovering:
- Significant API churn with 52 (
SchemaAdapter, etc.). This meant Comet couldn't test 52 before release, and maybe other downstream consumers have been slow to adopt it and find more issues in it. - Released during the holidays.
- Slightly longer release window: DF 51.0 on November 19, 52.0 on January 12, but both were ~540 commits.
All of that said, re: LTS releases:
- We're struggling a bit to get 52.3 and 53 released at the same time, demonstrating the challenges of maintaining parallel releases. How would we handle cutting LTS releases while simultaneously cutting new major versions?
- Selfishly: DataFusion Comet depends on projects (iceberg-rust) that may pin to an LTS. I am hoping to find a way to disentangle the iceberg-rust components that Comet needs from DataFusion dependency, but realistically they'd still be tied to the Arrow-rs version. If iceberg-rust pins to an LTS and Comet wants to keep up with DataFusion, then we'd likely have to fork iceberg-rust.
- What would the backporting criteria be? Presumably it's only for regressions, but even that gets a little fuzzy w.r.t. performance differences. The hope would be that it wouldn't add significant reviewing workload, but we also want to be disciplined with what gets backported.
Reacted by Bhargava Vadlamani, KAZUYUKI TANIMURA and Marko Milenković- Significant API churn with 52 (
Let's move this conversation to
I think we should consider LTS as described here: #16622 (comment)
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Describe the solution you'd like
Describe alternatives you've considered
Keep things as they are.
Additional context