Repository navigation
test(migration): guard against reserved column names on MariaDB and MySQL - #9991
Conversation
…ySQL Forward-port of the two tests that #5957 added on the release line. - ReservedColumnNamesTest checks every column the migrations define against the reserved words measured on MariaDB 11.3 and MySQL 8.0 (fixtures/). Doctrine quotes a column name only when its own keyword list has the word, and its MariaDB list misses row_number (#4909). - ImportPreviewRowMapperMigratedTest runs the mapper's queries against the migrated table: the source row lives in source_row, and both readers return rows in source order. The mapper test calls findByPreview with ListQuery::unbounded(), the signature on main. Refs #4909 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…am sources ReservedColumnNamesTest used two hand-committed lists, one for MariaDB 11.3 and one for MySQL 8.0, that had to be edited whenever a version moved. It now derives the list per server series from the upstream source: the keywords in the lexer's symbol table (sql/lex.h) that the grammar's identifier rule (MariaDB keyword_ident, MySQL ident_keyword in sql/sql_yacc.yy) does not allow as an identifier. The series are branches (MariaDB 10.5, 10.6, 10.11, 11.3, 11.4, 11.8 and main; MySQL 8.0, 8.4 and trunk), so new point releases and the words a coming series reserves are picked up without an edit. 11.3 stays because MariaDB 11.0 to 11.3 reserve ROW_NUMBER, the word behind #4909, and 11.4 does not. The parser lives in tests/Support/ReservedWordSource.php. For the versions measured earlier on live servers (MariaDB 11.3, MySQL 8.0) it returns exactly the words that were measured. The committed snapshots stay as the offline fallback, one per series, and are regenerated by fixtures/update-reserved-words.php instead of being edited by hand. Per series the test downloads with a 5 second connect timeout, falls back to the snapshot when the network does not answer (one failure ends the downloads for the rest of the run), and checks the union of both. A 404 or a layout the parser no longer reads fails the test instead of falling back, so a renamed branch or a changed upstream file cannot leave it checking stale lists unnoticed. Refs #4909 Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…e is not found The parser followed the identifier rule (keyword_ident / ident_keyword) and, when upstream renamed it, found nothing reachable and counted every lexer keyword as reserved (688 words for MariaDB 11.3, `status` and `date` included). That passed the 200-word lower bound and surfaced as an unrelated column failure in a migration. parse() now throws a RuntimeException naming the engine, the series and the rule when the start rule is missing, or when a rule it reaches has no definition. The plausibility check on a list gained an upper bound and a sentinel (`status` and `date` must not be reserved), and it also covers the committed snapshots. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…iling on them MariaDB `main` and MySQL `trunk` were merged into the same lookup as the released series, so the day either branch reserved a word an existing column uses, every open PR would fail without a change of its own. SOURCES now marks which series are development branches. Released series keep failing testNoMigrationColumnIsAReservedWord; a word only a development branch reserves is reported by a separate test through markTestIncomplete(), which phpunit.xml and the CI workflow do not turn into a failure. Both messages name, per column, the engine and series that reserve the word. The schema walk is shared by the three tests. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… download fallback ReservedWordSource::words() reads the snapshot on every run and merges it with the online list, so a word upstream stops reserving stays flagged until the snapshot is regenerated. The class docblocks, the generator's docblock and the header it writes said the snapshot is read only when the download fails. The generator's header text is corrected and the ten snapshots regenerated; their word lists are unchanged, only the header lines differ. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…wance exempts ALREADY_IN_USE exempts a word on every engine and series because Doctrine quotes it wherever it is reserved. Nothing asserted that: if an engine later reserved e.g. `groups` on MariaDB, where Doctrine's list lacks it, the allowance would hide exactly the #4909 failure. A new test checks, per engine, that each allowed word an engine reserves is in every Doctrine keyword list that engine's platforms can use (MariaDBKeywords, MariaDb102Keywords, MariaDb117Keywords; MySQL80Keywords, MySQL84Keywords: the classes of doctrine/dbal 3.10, which Nextcloud 32 to 35 all ship). It skips with a message naming the classes when they cannot be loaded. The docblock now says why one global list is acceptable: the premise is asserted per engine. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…side the check The test reads the schema built from the migrations' changeSchema() only, so a column added with raw SQL or in preSchemaChange/postSchemaChange never appears in it. The class docblock now says so and names the two migrations that do this today (Version1Date20260322120000 and Version1Date20260706100000) and why a source scan was not added: their column names are interpolated or spread over concatenated strings, so a scan would miss or misread them. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…e each list came from download() treated every non-200 as "offline" and latched it for the run: a 301 (redirects are not followed), a 429 or a 5xx ended the downloads silently, and the origin of each list was only printed when the assertion failed. Now 200 is used; a 404 or any 3xx throws with the URL, because the source moved and no snapshot repairs that; 429, 5xx, other statuses and connection errors fall back to the snapshot and keep the reason. The test writes one line on STDERR after the run naming which series came from upstream and which from the snapshot, with the download failure, so a snapshot-only run is visible while the suite stays green; the failure message carries the same line. The refresh script prints the reason too. curl_close() is gone (deprecated in PHP 8.5, a no-op since 8.0). Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…bare clone ReservedColumnNamesTest runs under phpunit-unit-local.xml on a host without a Nextcloud server; only the import-preview mapper test still needs the server's classes, and its own docblock says so. The paragraph claiming otherwise is removed. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…s, not only the branch head A series branch is only its newest point release, and upstream also un-reserves words: the MariaDB 10.11 and 11.4 branch heads no longer list ROW_NUMBER while every released 10.11.x and 11.4.x (and 11.8.x) does. The snapshots held head-only words, so the word that broke the import preview migration (#4909) was caught on MariaDB only through the 11.3 branch. The refresh script now builds the snapshot of a released series as the union over the branch head and every release tag of that series (tags listed through the GitHub API, `<engine>-<series>.<n>` only; MySQL 8.0 starts at 8.0.11, because 8.0.0 to 8.0.4 are development milestones with another lex.h layout). A development branch has no tags: head only. It goes one request at a time, takes an optional GITHUB_TOKEN for the rate limit, caches finished tags so an interrupted run resumes, and exits 1 naming the cause. Each header states the tag range, the tag count and the date. The test still reads the live head and merges the snapshot, which is now also the only source for the older point releases; the docblocks say so. A new assertion pins the defect class: ROW_NUMBER must be in the MariaDB 10.11, 11.3, 11.4 and 11.8 lists and the MySQL 8.0 and 8.4 lists. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
WilcoLouwerse
left a comment
There was a problem hiding this comment.
Verdict: APPROVE (Standard · full)
Self-review by the author, posted as a comment because GitHub does not accept an approval on one's own PR; a colleague's review is still needed. Tests only, nothing under lib/. The review found one blocker and three concerns, plus four minor points; all eight were fixed on the branch before this verdict (commits 3e4991da95..e72c6065b8), and a separate check of the fix commits found no defect.
Connected issues
| Issue | State | Effect of this PR |
|---|---|---|
| #4909 | closed (fixed by #5879 / #5957) | unaffected; this PR adds the guard on main |
🔴 Blocker (fixed)
- f8 — a series branch is only its newest point release; the 10.11 and 11.4 lists missed
row_number. Fixed ine72c6065b8.
🟡 Concerns (fixed)
- f1 — the plausibility guard passed when the grammar's identifier rule was not found. Fixed in
3e4991da95. - f2 — development branches were a hard gate. Fixed in
b5fb313482. - f3 — the text said the snapshot is read only when the download fails. Fixed in
cd64b7a9c8.
🟢 Minor
- f4 — the allowance's premise (Doctrine quotes these words) was untested. Fixed in
7f7ab230de. - f5 — columns created by raw SQL are not seen by the test. Stated in the class docblock, with the two migrations, in
166af53cae; no scan added. - f6 — any status other than 200 or 404 fell back silently. Fixed in
93d0afc7fa: 404 and redirects fail, outages fall back, one STDERR line names each list's origin. - f7 — the docblock said the test cannot run in a bare clone. Fixed in
645a5953e2. - Open, from the check of the fix commits, none blocking: the refresh script's tag cache in the system temp directory is created world-writable and written non-atomically (
ReservedWordSource::releaseWords()); when one series fails, the other snapshots are still rewritten (exit 1); two comments say the branch heads "no longer list"ROW_NUMBERwhere "no longer reserve" is meant; the snapshot headers do not say that pre-release tags (for example 11.4.0) are included.
Checks on the fixed head e72c606: ReservedColumnNamesTest on the host: exit 0, 5 tests, 93 assertions, online and with the download host unreachable, also with -c phpunit.xml; every snapshot equals the union over its release tags (159 tags, recomputed independently); mutation (a column renamed to row_number) fails the test. Not run locally: ImportPreviewRowMapperMigratedTest (needs a Nextcloud server), the full suite.
Gates: the CI checks on this head, the gate check included, had not finished when this was posted; no gate findings this round.
|
/backport to stable2.1 |
…on stable2.1 On this line MigratedSqliteDatabase has no fallback for a bare clone, so the sentence #9991 dropped still holds here. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…erved-words test(migration): read reserved words per database version from upstream sources (stable2.1 backport of #9991)
Quality Report — ConductionNL/openregister @
|
| Check | PHP | Vue | Security | License | Tests |
|---|---|---|---|---|---|
| lint | ✅ | ||||
| phpcs | ❌ | ||||
| phpmd | ❌ | ||||
| psalm | ❌ | ||||
| phpstan | ❌ | ||||
| phpmetrics | ✅ | ||||
| eslint | ❌ | ||||
| stylelint | ✅ | ||||
| build | ✅ | ||||
| check-specs | ✅ | ||||
| test-l10n | ❌ | ||||
| test-l10n-parity | ❌ | ||||
| format | ❌ | ||||
| check-schema-l10n | ✅ | ||||
| test-newman-route-ratchet | ✅ | ||||
| composer | ✅ | ✅ 169/169 | |||
| npm | ✅ | ✅ 639/639 | |||
| app:check-code | ⏭️ | ||||
| info.xml | ✅ | ||||
| REUSE | ✅ | ||||
| lockfile sync | ✅ | ||||
| PHPUnit | ❌ 56767/56972 · 27 skipped (counts from 1 of 8 legs) | ||||
| Newman | ❌ | ||||
| Playwright | ❌ 252/338 · 5 skipped · 150 did not run | ||||
| Hydra gates | ❌ |
Quality workflow — 2026-11-14 22:55 UTC
Download the full PDF report from the workflow artifacts.
Summary
Forward-port of the two tests that #5957 added to the release line, so
mainis guarded too. The fix itself is already onmainvia #5879; these tests were only onbeta/stable2.1.tests/Unit/Migration/ReservedColumnNamesTest.phpchecks every column the migrations define through the schema API against the words each supported MariaDB and MySQL series reserves. Doctrine quotes a column name only when its own keyword list for the platform has the word, and its MariaDB list missesrow_number, which broke MariaDB installs (Migration Version1Date20260915050000 fails on MariaDB: column name row_number is a reserved word #4909). CI does not run the migrations on MariaDB, so this test is the guard. A short allowance lists the reserved words already in use that Doctrine does quote; one test keeps that allowance from going stale and another checks that Doctrine really quotes each of them. Columns created by raw SQL are outside the check; the class docblock names the two migrations that do so.tests/Support/ReservedWordSource.php: the keywords insql/lex.hthat the grammar's identifier rule insql/sql_yacc.yydoes not allow as an identifier. The parser fails loudly when that rule is missing or the result is implausible. Released series: MariaDB 10.5, 10.6, 10.11, 11.3, 11.4, 11.8 and MySQL 8.0, 8.4. Development branches (MariaDBmain, MySQLtrunk) are reported as an incomplete test, not a failure, so a word a coming series reserves is visible without turning unrelated PRs red.tests/Unit/Migration/fixtures/is the union over its branch head and every release tag (159 tags in total), because a branch head is only the newest point release: MariaDB 10.11.x, 11.3.x, 11.4.x and 11.8.x reserveROW_NUMBER(from 10.7 on), while the 10.11 and 11.4 heads no longer reserve it. At run time the test checks the live branch head merged with the snapshot; when a download fails (connection error, 429, 5xx) the snapshot alone is used, so the test never needs network. A 404 or redirect fails the test on purpose: the source moved. One line on STDERR says where each list came from.php tests/Unit/Migration/fixtures/update-reserved-words.phpregenerates the snapshots (optionalGITHUB_TOKEN).tests/Unit/Db/ImportPreviewRowMapperMigratedTest.phpruns the mapper's queries as SQL against the migrated table: the source row lives insource_row, notrow_number, andfindByPreview/findPendingWritesreturn rows in source order.Only change against the
betaversion of the mapper test:findByPreviewis called withListQuery::unbounded(), the signature onmain.Checks
vendor/bin/phpunit -c phpunit-unit-local.xml tests/Unit/Migration/ReservedColumnNamesTest.phpon the host (PHP 8.4, no Nextcloud server), exit 0, OK (5 tests, 93 assertions, none skipped or incomplete), once online and once withRESERVED_WORDS_BASE_URL=https://invalid.invalid(every list from its snapshot). Also exit 0 with-c phpunit.xml.row_number: the test fails (exit 1) and names the column withmariadb 10.11, mariadb 11.3, mariadb 11.4, mariadb 11.8, mysql 8.0, mysql 8.4; reverted.php -lon the changed PHP files, no syntax errors.ImportPreviewRowMapperMigratedTest(needs Nextcloud's Doctrine classes and errors on the host), the full unit suite, Psalm, and a real MariaDB 10.6 server; CI runs the first three.phpcs.xmlcoverslib/only, so these files are outside it, as onbeta.Test plan
Refs #4909
🤖 Generated with Claude Code