You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Round Robin Polling Between Multiple Winners of Loser Tree in SPM #12231
Is your feature request related to a problem or challenge?
Assume a scenario where SPM (SortPreservingMergeExec) polls from 3 partitions, each of them gives the same value for the order expression for a very long time. Currently, we poll from the same partition until a new value comes from the other partitions. Continuously polling the same partition gets all upstream operator buffers and channels grow. However, we can execute a round-robin fashioned polling mechanism for the tied partitions.
Describe the solution you'd like
Loser tree can re-order the tied values of partitions.
Describe alternatives you've considered
No response
Additional context
This is the PR resolving a similar problem in the initialization phase: #12230
Activity
athultr1997 commentedon Oct 26, 2024
@berkaysynnada Any idea why this happens ?
berkaysynnada commentedon Oct 27, 2024
Because of eagerly polling mechanisms like RepartitionExec channels