Repository navigation
Check the final physical plan in the planner and name the rule that broke it - #26461
Open
jayzhan211 wants to merge 1 commit into
Open
jayzhan211 wants to merge 1 commit into
jayzhan211 wants to merge 1 commit into
Conversation
github-actions
Bot
added
documentation
jayzhan211
marked this pull request as ready for review
October 18, 2026 03:14
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #26461 +/- ##
=======================================
Coverage 82.89% 82.90%
=======================================
Files 1154 1154
Lines 457656 457711 +55
Branches 457656 457711 +55
=======================================
+ Hits 379391 379450 +59
+ Misses 55261 55252 -9
- Partials 23004 23009 +5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
|
Thank you, I have a quick look and I think its overall beneficial. Will have a in-depth review after this PR getting some other +1s. Specifically, I think the benefits is prevent internal tests or simple extensions from building invalid optimizer pipeline; but for more advanced extension (that overrides sanity check), we might need additional API to set a custom sanity checker. |
alamb
reviewed
Oct 18, 2026
alamb
left a comment
alamb
left a comment
Contributor
There was a problem hiding this comment.
I quickly looked at this and it seems like a good idea. I will try and find more time to review later on.
My only concern is plan speed -- I will run some planning benchmarks to check
| | 19 | `EnsureCooperative` | - | Wraps non-cooperative plan parts so long-running tasks yield fairly. | | ||
| | 20 | `FilterPushdown(Post)` | post-optimization phase | Pushes dynamic filters at the end of optimization, after plan references stop moving. | | ||
| | 21 | `SanityCheckPlan` | - | Validates that the final physical plan meets ordering, distribution, and infinite-input safety requirements. | | ||
| After the last rule, including any rule added by the user, the physical planner |
Contributor
There was a problem hiding this comment.
I feel like this document is outdated and redundand and hard to find -- maybe as a follow on PR we can fold it into the documentation in #26351
| // given query plan; i.e. it only acts as a final | ||
| // gatekeeping rule. | ||
| Arc::new(SanityCheckPlan::new()), | ||
| // There is no `SanityCheckPlan` rule at the end of this list. The |
Contributor
There was a problem hiding this comment.
how about we deprecate SanityCheckplan rule so it is clearer we intend to remove it
Contributor
|
run benchmark sql_planner |
1 similar comment
Contributor
|
run benchmark sql_planner |
|
🤖 Benchmark running (GKE) | trigger CPU Details (lscpu)Comparing planner-final-plan-check (2689cbb) to 290ad15 (merge-base) diff Run configurationrun benchmark sql_plannerResults will be posted here when complete File an issue against this benchmark runner |
|
🤖 Benchmark running (GKE) | trigger CPU Details (lscpu)Comparing planner-final-plan-check (2689cbb) to 290ad15 (merge-base) diff Run configurationrun benchmark sql_plannerResults will be posted here when complete File an issue against this benchmark runner |
|
🤖 Benchmark completed (GKE) | trigger Instance: Comparing planner-final-plan-check (2689cbb) to 290ad15 (merge-base) diff Run configurationrun benchmark sql_plannerCPU Details (lscpu)Details
Resource Usagesql_planner — base (merge-base)
sql_planner — branch
File an issue against this benchmark runner |
|
🤖 Benchmark completed (GKE) | trigger Instance: Comparing planner-final-plan-check (2689cbb) to 290ad15 (merge-base) diff Run configurationrun benchmark sql_plannerCPU Details (lscpu)Details
Resource Usagesql_planner — base (merge-base)
sql_planner — branch
File an issue against this benchmark runner |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Related: #26171 (option 3), #26351, #26360, #25688.
Rationale for this change
SanityCheckPlanis the last rule inPhysicalOptimizer::new(), so rules appended withSessionStateBuilder::with_physical_optimizer_rulerun after it and are never checked. Onmain, an appended rule that removes a window's sort silently returns wrongrow_number()values. Custom pipelines have to keep the rule last by hand, and a custom list without it runs plans whose requirements do not hold. When the check does fail, the error only saysSanityCheckPlan, so finding the rule that broke the plan has needed a manual bisect each time (for example #26391, #17077, #25921).@alamb suggested this change in the review of #26107: remove
SanityCheckPlanas an optimizer pass and always run it after the optimizer is done (comment).#26171 agreed (option 3) that a custom pipeline must keep the plan valid, while optimization coverage is not guaranteed. This PR makes that validity part checked for every pipeline. The review on #26360 asked for the same shape: one rule,
EnsureRequirements, makes the plan valid, and every later rule keeps it valid.What changes are included in this PR?
SanityCheckPlanleaves the default rule list.DefaultPhysicalPlanner::optimize_physical_planruns the same check (unbounded inputs, ordering, distribution,check_invariants(Executable)) after the last rule, including appended rules. It replaces the planner's ownInvariantChecker(Executable)walk, whichSanityCheckPlanrepeated.The planner keeps the plan after each rule (one
Arcper rule). When the ordering or distribution requirements fail, the error names the rule that broke them:or, when the plan was already invalid as
EnsureRequirementsproduced it:Plans before
EnsureRequirementsneed not satisfy the requirements, so only plans from its last run onwards are searched. The rule is recognized by its name; a separate enforcement phase (feat: run physical requirement enforcement first, as a PhysicalAnalyzer phase #25688) would give a cleaner boundary. Errors about unbounded inputs keep the plainSanityCheckPlancontext, so existing messages do not change.sanity_checkerexposescheck_plan_tree_sanity,check_input_requirementsandinput_requirements_hold, which the planner uses.The upgrade guide, the optimizer rule reference and
explain.sltare updated.What is the testing strategy for this PR?
sanity_checker.rs: a rule appended after the default rules that removes sorts is reported by name, both with the default list and with a custom list; a list withoutEnsureRequirementsreports that no rule produced a valid plan; the default list passes.memory_limit/mod.rs: the two sort merge join tests ran aSortMergeJoinExecover unsorted streams through a custom rule list withoutEnsureRequirements. They now sort the generated data by the join keys and declare that order, so the plan is valid and the tests still measure the join's memory use.Are there any user-facing changes?
with_physical_optimizer_rule, or by a custom rule list, are now checked. A plan whose requirements do not hold, such as a sort merge join over unsorted input, fails to plan instead of running and possibly returning wrong results.PhysicalOptimizer::new()no longer containsSanityCheckPlan, andEXPLAIN VERBOSEno longer printsphysical_plan after SanityCheckPlan. Code that looks the rule up in the default list no longer finds it.SanityCheckPlanitself remains available.This changes the public behavior of the default rule list, so it needs the
api changelabel.