Repository navigation
fix(parser): support three-part identifiers and parenthesized join groups (#521) - #585
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Claude finished @ajitpratap0's task in 22s —— View job Review of PR #585: three-part identifiers and parenthesized join groups (#521)I reviewed the diff by reading it. I did not build the branch or run the tests, so I'm relying on the test results reported in the PR description. Verdict: looks good. I found nothing blocking. What I checked
Minor suggestions (non-blocking)
Test coverage: The new tests are thorough. They cover the exact issue query on MySQL and MariaDB, equivalence with the flat query across all dialects, round-trip, |
18c73f2 to
d76351f
Compare
|
Claude finished @ajitpratap0's task in 39s —— View job Review of PR #585: three-part identifiers and parenthesized join groups (#521)I reviewed the diff by reading it, including the newer hardening commit. I did not build the branch or run the tests, so I'm relying on the results reported in the PR description. Verdict: looks good. I found nothing blocking, and one item below is worth checking before merge. Addressed since the earlier review
Worth checking
Minor notes (non-blocking)
Performance and security: No new allocations on the hot path. Recursion is bounded by Test coverage: Thorough. It covers the exact issue query on MySQL and MariaDB, equivalence with the flat query in all dialects, round-trip, extraction, and the rejection cases. Item 1 above is the main gap. |
…n groups (#521) MySQL view definitions (SHOW CREATE VIEW output such as sakila.customer_list) failed to parse for two reasons, in every dialect: - A column reference with three or more parts (`sakila`.`city`.`city`, db.public.users.id) stopped after the second part with "E2002 expected FROM, semicolon, or end of statement, got PERIOD". parsePrimaryExpression now folds every extra qualifier into Identifier.Table as a dotted string (Table "sakila.city", Name "city"), mirroring TableReference.Name. Identifier.SQL() already prints dots unquoted, so no AST change is needed. The two-part path is untouched and still makes a single allocation. - A leading parenthesized join group (FROM ((a JOIN b ON ..) JOIN c ON ..) JOIN d ON ..) was always treated as a derived table and failed with "expected SELECT in derived table, got LPAREN". parseFromClause now recognises "(" followed by an identifier or "(" as a joined table, parses it recursively (depth-guarded) and flattens its joins into the statement's join chain. Joins associate to the left, so the AST is identical to the equivalent flat query and SQL()/formatter output round-trips. Cases that flattening would get wrong (comma list after or inside a group) are rejected with E4001, and a parenthesized join on the right side of JOIN keeps its current error. Parser benchmarks show no allocs/op or B/op change. docs/SQL_COMPATIBILITY.md gains rows for both constructs. Fixes #521 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve
… follow-up)
- Identifier.SQL() quotes each dot-separated part of Table on its own, so
`my-db`.`t`.`c` renders as "my-db".t.c instead of the single identifier
"my-db.t".c. A dot in Table is always a part separator (documented on
Identifier.Table); the SQL() fast path agrees with the buffered path for
empty parts.
- Multi-part column references collect their parts and join once, and more
than 5 parts (SQL Server server.db.schema.table.column) is rejected with
E2004. A 50k-part chain now fails in milliseconds instead of running
quadratically.
- ExtractColumnsQualified splits a dotted qualifier at its last dot, so
db.t.col gives {Schema: db, Table: t, Name: col}, matching
ExtractTablesQualified's {Schema: db, Name: t}.
- An alias or ClickHouse FINAL after a parenthesized join group is rejected
with E4001 instead of a misleading "expected statement" error or FINAL
silently moving onto the group's first table.
- Qualifier errors no longer pass hint text as the SQL source, which printed
a bogus context snippet (new branch and pre-existing two-part branch).
- Tests for backtick (MySQL) and bracket (SQL Server) names directly after
"(" in a join group.
Part of #521
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve
|
Pre-merge review follow-ups (merge train), in the latest commits:
|
d76351f to
ca7c41c
Compare
|
Claude finished @ajitpratap0's task in 26s —— View job Review of PR #585: three-part identifiers and parenthesized join groups (#521)I reviewed this by reading the code. I tried to run probe tests for the open items below, but the sandbox denied Verdict: looks good. I found nothing blocking. One item is still open from the last review. Open item:
Still to add
Addressed since the earlier review
Notes
|
Join groups and derived tables inside a group parse; a parenthesized set operation used as a derived table was unsupported before #521 and still returns an error rather than a wrong tree. Pin both so a later change to either is deliberate. Part of #521 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve
|
Claude finished @ajitpratap0's task in 15s —— View job Review of PR #585: three-part identifiers and parenthesized join groups (#521)I reviewed this by reading the code. I did not build the branch or run the tests, so I'm relying on the results in the PR description. Verdict: looks good. I found nothing blocking. Both open items from the earlier reviews are resolved
Optional
Notes for the consolidated changelog PR (#566)
Performance and security
Test coverage: Thorough. It covers the exact issue query on MySQL and MariaDB, flat-query equivalence, round-trip, extraction, quoted names, and the rejection cases. |
#613) parseQualifiedName built dotted names with `name = name + "." + part`, so time and allocation grew quadratically with the number of parts: a ~100 KB `a.a.a...` chain in FROM, JOIN, INSERT, UPDATE, DELETE, CREATE/DROP/TRUNCATE, views or indexes cost hundreds of MB to GBs. The parts are now collected in a fixed stack array (it does not escape) and joined once with strings.Join, and names with more than maxQualifiedNameParts (5, the same limit as column references from #585) are rejected with E2004 "qualified name has more than 5 parts". Names with 1 to 5 parts produce identical strings and SQL() output. The FROM path surfaces the E2004 directly; the JOIN, DML and DDL callers keep discarding or re-wrapping the error as E2002 "expected table name", so those inputs are rejected fast but with the callers' existing message (callers intentionally untouched to keep the diff localized). Benchmarks (benchstat, Apple M4, machine heavily loaded by concurrent runs so ns/op is noisy; allocs and B/op are exact): ParserSimpleSelect/ComplexSelect/Insert/Update/Delete allocs and B/op unchanged QualifiedTableName/1part 8 allocs, 2.128Ki (unchanged) QualifiedTableName/3part 10 -> 9 allocs (-10%) QualifiedTableName/5part 12 -> 9 allocs (-25%) QualifiedTableName/chain4KB 2068 -> 30 allocs (-98.6%), 5032Ki -> 664Ki (-86.8%) Fixes #613 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve
…errors (#616) Several parser error builders were handed a human-readable sentence as their `sql` argument. That argument becomes Error.Context.SQL, so Error() printed the sentence as "line 1" of the query with a caret at the real column. The parser does not hold the source SQL, so these sites now pass "" (no snippet, as in #585) and carry the sentence, verbatim, as the hint. Code, message and location are unchanged. ExpectedTokenError (E2002): FROM, SAMPLE (x5), PREWHERE, WHERE and the SELECT column list guard. InvalidSyntaxError (E2004), same defect: CAST type params, INTERVAL, empty array subscript, ::TYPE(...) params, and the two strict-mode empty statement checks. Adds a table-driven regression test plus a go/ast guard that fails if a non-test parser file passes a non-empty string literal as a pkg/errors builder's `sql` parameter. Attaching the real source text to parser errors is left as a follow-up. Fixes #616 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve
…ntext (#613, #616) (#627) * perf(parser): join qualified object names once and cap them at 5 parts (#613) parseQualifiedName built dotted names with `name = name + "." + part`, so time and allocation grew quadratically with the number of parts: a ~100 KB `a.a.a...` chain in FROM, JOIN, INSERT, UPDATE, DELETE, CREATE/DROP/TRUNCATE, views or indexes cost hundreds of MB to GBs. The parts are now collected in a fixed stack array (it does not escape) and joined once with strings.Join, and names with more than maxQualifiedNameParts (5, the same limit as column references from #585) are rejected with E2004 "qualified name has more than 5 parts". Names with 1 to 5 parts produce identical strings and SQL() output. The FROM path surfaces the E2004 directly; the JOIN, DML and DDL callers keep discarding or re-wrapping the error as E2002 "expected table name", so those inputs are rejected fast but with the callers' existing message (callers intentionally untouched to keep the diff localized). Benchmarks (benchstat, Apple M4, machine heavily loaded by concurrent runs so ns/op is noisy; allocs and B/op are exact): ParserSimpleSelect/ComplexSelect/Insert/Update/Delete allocs and B/op unchanged QualifiedTableName/1part 8 allocs, 2.128Ki (unchanged) QualifiedTableName/3part 10 -> 9 allocs (-10%) QualifiedTableName/5part 12 -> 9 allocs (-25%) QualifiedTableName/chain4KB 2068 -> 30 allocs (-98.6%), 5032Ki -> 664Ki (-86.8%) Fixes #613 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve * fix(parser): stop passing hint text as the SQL source in E2002/E2004 errors (#616) Several parser error builders were handed a human-readable sentence as their `sql` argument. That argument becomes Error.Context.SQL, so Error() printed the sentence as "line 1" of the query with a caret at the real column. The parser does not hold the source SQL, so these sites now pass "" (no snippet, as in #585) and carry the sentence, verbatim, as the hint. Code, message and location are unchanged. ExpectedTokenError (E2002): FROM, SAMPLE (x5), PREWHERE, WHERE and the SELECT column list guard. InvalidSyntaxError (E2004), same defect: CAST type params, INTERVAL, empty array subscript, ::TYPE(...) params, and the two strict-mode empty statement checks. Adds a table-driven regression test plus a go/ast guard that fails if a non-test parser file passes a non-empty string literal as a pkg/errors builder's `sql` parameter. Attaching the real source text to parser errors is left as a follow-up. Fixes #616 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve * fix(parser): address adversarial review follow-ups for #627 627-1: the Snowflake permissive CREATE <kind> stub (STAGE, SEQUENCE, ...) discarded the parseQualifiedName error, so a 6+ part name parsed successfully with only its tail in RawSQL ("CREATE SEQUENCE f" for "CREATE SEQUENCE a.b.c.d.e.f"). The stub now returns the E2004 part-limit error; other name errors are still tolerated as before. 627-2: every non-FROM caller of parseQualifiedName (JOIN, INSERT, UPDATE, DELETE, CREATE TABLE/VIEW/MATERIALIZED VIEW/INDEX, ClickHouse MV TO, REFRESH MATERIALIZED VIEW, DROP, TRUNCATE, MySQL SHOW/DESCRIBE/REPLACE, Snowflake USE) replaced the E2004 with "expected table name, got IDENTIFIER", pointing at a valid identifier. The part-limit error now carries an errQualifiedNameTooLong cause, and callers pass it through unchanged via isQualifiedNameTooLong/qualifiedNameError. Tests now assert the E2004 message on every path. 627-3: the BETWEEN lower/upper, LIKE, REGEXP and IN-subquery wrappers passed the current token's text as the error's sql argument, printing a fake "1 | )" source line. They now pass "" and attach the operand error with WithCause. The #616 static guard now flags any sql argument other than the literal "" (allowlisting validate.go's real-source string(input)), not just non-empty string literals. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…resh performance baselines (#566) CHANGELOG [Unreleased] now covers every PR merged since v1.14.0, with a Behavior changes section for the observable changes in #597, #595, #584, #585, #588, #581, #582/#570, #517/#518 and #511, plus Added, Changed, Fixed, Performance, Security and Internal entries. The two existing entries (#526 and the tokenizer-pool dialect fix) are kept verbatim. performance_baselines.json is re-measured on an Apple M4 with BenchmarkPerformanceBaseline -count=10 -benchmem (median). Baselines are set at about 2.5x the M4 median rather than 1.5x: the CI Benchmark job on ubuntu-latest measures these queries about 1.5-3.2x slower than M4 (last 9 Tests runs on main, 0715bb9..61ca6c8), so 1.5x with the unchanged tolerances would fail CI's slowest runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve
…xes; refresh performance baselines (#566) (#598) * docs: consolidate CHANGELOG for the perf audit and batch-2 fixes; refresh performance baselines (#566) CHANGELOG [Unreleased] now covers every PR merged since v1.14.0, with a Behavior changes section for the observable changes in #597, #595, #584, #585, #588, #581, #582/#570, #517/#518 and #511, plus Added, Changed, Fixed, Performance, Security and Internal entries. The two existing entries (#526 and the tokenizer-pool dialect fix) are kept verbatim. performance_baselines.json is re-measured on an Apple M4 with BenchmarkPerformanceBaseline -count=10 -benchmem (median). Baselines are set at about 2.5x the M4 median rather than 1.5x: the CI Benchmark job on ubuntu-latest measures these queries about 1.5-3.2x slower than M4 (last 9 Tests runs on main, 0715bb9..61ca6c8), so 1.5x with the unchanged tolerances would fail CI's slowest runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve * docs(changelog): add batch-3 follow-up fixes (#599-#620) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve * docs(changelog): record #632 render-injection hardening and #624 quote guards Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve * docs(changelog): correct batch-3 entries after review; record measured costs - #628: SQLite still rejects count(*) [cnt]; only SQL Server keeps it. - #631: bare FORMAT / FORMAT <name> rules apply after SETTINGS. - Performance: record the measured costs of the batch-3 correctness fixes (Identifier and TableReference growth, quoting in SQL(), lineStarts reallocation, sorted Extract results). - #632: note the lıke, WITH CHECK OPTION and MINUS rendering changes. - Fix the [Unreleased] compare link (v1.14.0...HEAD) and add [1.14.0]. - Keep each section a single tight list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve * docs(changelog): describe the release as source-compatible with runtime behaviour changes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Summary
#521 [BUG] Valid MySQL query fails on three-part identifiers and parenthesized joins
Fixes #521. Column references with more than two parts now parse in every dialect:
db.table.col, db.t.* and catalog.schema.t.col. The extra qualifiers are folded into Identifier.Table as a dotted string, mirroring TableReference.Name, so SQL() reproduces the reference without any AST change. A parenthesized join group at the start of FROM, as in MySQL SHOW CREATE VIEW output, is now parsed recursively with a depth guard and its joins are flattened into the statement's join chain, so the AST and SQL() are identical to the flat query. Group shapes that flattening would get wrong (comma lists after or inside a group) are rejected with E4001. Verification: the new regression tests fail on the parent commit and pass on this branch. Differential testing of about 3,866 corpus statements in 9 dialects showed no changed output for any input that parsed before. In an alternating benchstat run, parser allocs/op and B/op are unchanged and ns/op is flat (geomean -0.86%). The race suite,task checkand golangci-lint all pass.Commits (1), one per issue:
5 files changed, 509 insertions(+), 12 deletions(-)Test Plan
CGO_ENABLED=1 go test -race -timeout 120s $(go list ./... | grep -v /cbinding)),go vet, andgolangci-linton this branch.benchstat -count=6before/after on the same machine; allocs/op and B/op are the deterministic signal, ns/op is directional on a loaded machine.#521 benchstat
#521 race gate: PASS.
CGO_ENABLED=1 go test -race -timeout 120s $(go list ./... | grep -v /cbinding)exited 0: 44 packages ok, no FAIL, no DATA RACE. Many results were cached, so I re-ran-race -count=1on pkg/sql/parser (3.9s), pkg/gosqlx and pkg/formatter, and all were ok.task check(fmt, vet, lint, test:race) also exited 0, andgit status --porcelainwas still empty afterwards.Breaking Changes
joins []ast.JoinClauseargument (one caller). Observable behaviour: SQL that used to fail (multi-part column references; left-nested parenthesized joins in FROM) now parses in all dialects. A parenthesized join group renders flattened, and its AST is identical to the flat join chain. Some invalid inputs get more specific error messages, including new E4001 unsupported-feature errors for comma lists combined with parenthesized joins. Nothing that parsed before parses differently.Notes
main.Fixes #521
Part of #566
🤖 Generated with Claude Code
https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve