CoalesceBatches physical optimizer rule says it should be last but it isn't

datafusion/datafusion/physical-optimizer/src/optimizer.rs

Lines 123 to 129 in e323357

    
           // The CoalesceBatches rule will not influence the distribution and ordering of the 
        
           // whole plan tree. Therefore, to avoid influencing other rules, it should run last. 
        
           Arc::new(CoalesceBatches::new()), 
        
           Arc::new(CoalesceAsyncExecInput::new()), 
        
           // Remove the ancillary output requirement operator since we are done with the planning 
        
           // phase. 
        
           Arc::new(OutputRequirements::new_remove_mode()),

Can see here it states CoalesceBatches rule should be last but seems this isn't the case anymore; either comment is outdated and it doesn't need to be last anymore, or we should in-fact move it down to respect the comment.

Side note: it would be nice to add the comment for CoalesceAsyncExecInput too, to be consistent with how it's done for other rules. Something like:

// Coalesce in order to reduce number of async function invocations for efficiency.

FWIW I am working to organize some work to remove this execution plan entirely:

[EPIC] Remove CoalesceBatchesExec operator #18779

The rule was removed in

Remove CoalesceBatches optimizer rule #19591

So I think this issue is now obselete

Navigation Menu

Repository navigation

`CoalesceBatches` physical optimizer rule says it should be last but it isn't #18148

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	// The CoalesceBatches rule will not influence the distribution and ordering of the
	// whole plan tree. Therefore, to avoid influencing other rules, it should run last.
	Arc::new(CoalesceBatches::new()),
	Arc::new(CoalesceAsyncExecInput::new()),
	// Remove the ancillary output requirement operator since we are done with the planning
	// phase.
	Arc::new(OutputRequirements::new_remove_mode()),

CoalesceBatches physical optimizer rule says it should be last but it isn't #18148

Description

Activity

Jefffrey commented on Oct 18, 2025

alamb commented on Nov 17, 2025

alamb commented on Jan 6, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

`CoalesceBatches` physical optimizer rule says it should be last but it isn't #18148