Skip to content

[9.0] LINQ: drop ordering keys that read nothing from the document (backport of #5958) - #5960

Merged
jeremydmiller merged 10 commits into
9.0from
fix/5950-order-by-closed-value-9.0
Nov 15, 2026
Merged

jeremydmiller merged 10 commits into
9.0from
fix/5950-order-by-closed-value-9.0

Conversation

@jeremydmiller

@jeremydmiller jeremydmiller commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

Merge after #5956. This branch is stacked on #5956 (the 9.0 backport of #5949), which is itself stacked on #5948 (the 9.0 backport of #5943). Until those merge, the diff here also shows their commits. The #5950 change is the single commit 847df7035.

Backport of #5958 to 9.0. Fixes #5950 on 9.x.

Cherry-picked with -x from 8e0e856ec. The only adaptation: the migration note goes under 9.60.0, as an asterisk bullet like the rest of 9.0's guide.

CI (started by hand): https://github.com/JasperFx/marten/actions/runs/43758808897 (head 11a726779). The stack merges current origin/9.0 (through #5956), which brings in #5951 (HiLo mt_get_next_hi) and #5980 (a docs-only change). With those merged in, the top of the chain (#5971) passes full LinqTests net10.0 (3036/3036) and DocumentDbTests' Writing.Identity.Sequences namespace (33/33). #5956 also carries the one-line isComputedProjection fix that keeps an array index in a top-level Select() on the select visitor.

Root cause

Ordering resolves an OrderBy/ThenBy key through MemberFinder.Determine(lambda, "Invalid OrderBy() expression"). A key that reads nothing from the document failed in one of two ways:

  • It threw. A literal, arithmetic or an index (x => 1, x => Probe.Value * 2, x => Probe.Grid[1][0]) produced Invalid OrderBy() expression. That is the report.
  • It silently read the document. For a plain static member or captured local (x => Probe.Value, x => local), MemberFinder visits the lambda's own parameter list, so it saw a parameter "at the front" and resolved the static's name against the document. OrderBy(x => Probe.Name) became ORDER BY d.data ->> 'Name' and quietly ordered by the document's own Name. When no document member had that name, every row read null and the result happened to look right.

The fix

OrderingOperator drops a key whose selector body is a closed value before it becomes an Ordering. A closed value is a static member, a captured variable, a literal, or arithmetic or indexing over them; the test is the same IsClosedValue() that #5946 uses. This matches LINQ to Objects: the key is the same for every element and the sort is stable, so the remaining keys keep their relative order, and OrderBy(const).ThenBy(x => x.Number) orders by Number. A query whose keys are all closed has no ORDER BY.

  • OrderBy("Name") is not affected. It is Marten's dynamic-member overload, whose argument is a string rather than a key selector.
  • Ordering SQL that doesn't use a closed key is unchanged. A throwaway test dumped the SQL for 45 shapes that don't use one (where/order/select over members, nested members, indexes, dictionaries, ToLower, case-insensitive ordering, anonymous types, records, Distinct). The SQL is byte-identical on the base and on this branch. A test also asserts that OrderBy(static).ThenByDescending(x => x.Name).ThenBy(static).ThenBy(x => x.Number) sends exactly the SQL of OrderByDescending(x => x.Name).ThenBy(x => x.Number).
  • Compiled queries: a closed key in QueryIs() is dropped the same way, whatever its value.
  • Docs: a note under "Ordering Results" in operators.md, and a 9.60.0 migration note. markdownlint and cspell report 0 issues.

Tests

Bug_5950_ordering_by_a_closed_value has 20 cases, each compared with LINQ to Objects in order:

  • Key kinds: static, local, literal, string literal, indexed static, and arithmetic keys.
  • Position: each before, between and after document keys, ascending and descending.
  • A closed key next to other features: an index key from LINQ: indexing a document array fails in OrderBy/Select, and jagged document arrays fail in Where #5949, a Where, First, Skip/Take, a comparer, and a compiled query.
  • All keys closed: the SQL has no ORDER BY.
  • Name collision: a static whose name matches a document member.
  • OrderBy("Name"): still orders.

Red, then green on 9.0 (net10.0, marten_9_z)

  • On the base ([9.0] LINQ: index into document arrays in OrderBy, Select and jagged Where (backport of #5953) #5956 head a886a6ad7, fix reverted): 9 of 20 failed. The same 11 pass on the base as on master, for the same reason: static or local keys whose names match no document member read null on every row.
  • On this branch: 20 / 20 passed.
  • The SQL for the same 45 shapes that don't use a closed key is byte-identical between the base and this branch.
  • markdownlint over docs/ reports 0 issues, as it does on 9.0. cspell reports 0.

Full suite on this branch

  • LinqTests net10.0 against marten_9_z: 2250 / 2250

Found along the way, not fixed here

No CollectionUsage / SelectMany code is touched.

🤖 Generated with Claude Code

jeremydmiller and others added 3 commits November 15, 2026 11:54
…ixes #5949)

An index into an array or list on the document translated only in a Where()
comparison with a single index. OrderBy(x => x.Values[1]) was refused
("Invalid OrderBy() expression": MemberFinder treats ArrayIndex as
invalid), Select(x => x.Values[1]) threw ArgumentNullException ('array')
because SelectorVisitor rebuilt the index node from null children, and a
jagged x.Grid[1][0] failed everywhere: SimpleExpression rebuilt the inner
index node the same way, and ChildDocumentAtIndex could not index again.

- ChildDocumentAtIndex indexes an element that is itself an array: a scalar
  element through the same SimpleCastMember/StringMember factories value
  collections use, a complex one as another ChildDocumentAtIndex.
- OrderBy/ThenBy, Select (scalar, anonymous type, record) and bool
  predicates resolve a chain containing an index through the SimpleExpression
  walk Where() comparisons already used. Chains without an index resolve
  exactly as before; the SQL for 45 such shapes is byte-identical.
- A negative index throws BadLinqExpressionException instead of becoming a
  negative jsonb index, which PostgreSQL reads from the end of the array.
- A bool[] / List<bool> on the document could not be resolved at all
  (BooleanMember read the member type of an ElementMember); it can now, and
  Any(f => f) over one registers a filter instead of throwing.
- An indexed string element's case-insensitive ordering lowers the real
  locator, not the placeholder the constructor built.

Docs: "Reading an Element by Position" in child-collections.md, and a
migration note for the negative-index behavior change (under 9.60.0 on 9.0).

9.0 adaptation: BadLinqExpressionException is Marten.Exceptions on 9.0 (no
union members either), and the migration note goes under 9.60.0.

(cherry picked from commit 6d2c14e)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…5950)

OrderBy(x => Settings.SortKey).ThenBy(x => x.Number) threw "Invalid
OrderBy() expression": Ordering resolves its key through MemberFinder, which
refuses a chain with no lambda parameter. In LINQ to Objects such a key is
the same value for every element and the sort is stable, so it leaves the
order to the other keys and the ThenBy() acts as the primary ordering.

OrderingOperator now drops an OrderBy/OrderByDescending/ThenBy/
ThenByDescending key whose body is a closed value (a static member, a
captured variable, a literal, or arithmetic or indexing over them, the
same IsClosedValue() test #5943 uses) before it becomes an Ordering. The
remaining keys keep their relative order, and a query whose keys are all
closed has no ORDER BY. OrderBy("Name"), which names a member as a string,
is not a key selector and is unaffected; the SQL of every other ordering is
unchanged.

9.0 adaptation: the migration note goes under 9.60.0, as an asterisk
bullet like the rest of 9.0's guide.

(cherry picked from commit 8e0e856)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jeremydmiller and others added 5 commits November 15, 2026 13:04
…y-indexing-9.0

# Conflicts:
#	docs/migration-guide.md
…er-by-closed-value-9.0

# Conflicts:
#	docs/migration-guide.md
…5949 x #5942)

#5942 (merged on 9.0 as #5954) sends a top-level projection it classifies as
computed through the scalar translator and refuses it when that fails. Every
BinaryExpression counted as computed, which took in #5949's array indexes:
Select(x => x.Values[1]), Select(x => x.Grid[1][0]) and Select(x => x.Grid[1])
were refused instead of reaching the select visitor that translates them.
An ArrayIndex is now not a computed projection; arithmetic over one still is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jeremydmiller
jeremydmiller merged commit 075e6dc into 9.0 Nov 15, 2026
31 checks passed
@jeremydmiller
jeremydmiller deleted the fix/5950-order-by-closed-value-9.0 branch November 15, 2026 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant