Repository navigation

truncatesc: add new style time partitions - #6320

Open

markhannum wants to merge 6 commits into

bloomberg:mainfrom

markhannum:truncatesc-timepart

Open

truncatesc: add new style time partitions#6320
markhannum wants to merge 6 commits into
bloomberg:mainfrom
markhannum:truncatesc-timepart

markhannum commented Oct 8, 2026 •

edited

Loading

Contributor

truncatesc now covers new-style time partitions (the old cases used the legacy create time partition syntax and were disabled). It partitions a table, rolls it out, alters it, changes its retention, adds a time partition, collapses and drops, then truncates the log back through each step, checking data and partition metadata on the master and a physical replicant, and the master's rollout events.

Fixes for what it found:

Changing retention reordered the shards, so a later partitioned by none failed with "Table t1 already exists" and the master asserted. The original table now stays at shard 0.
Shards added by a retention increase got table version 0, so later alters of the partition failed with "stale version". They now get the partition's version.
A schema reload (e.g. log truncation) created a second timepart scheduler and never requeued rollout events. The scheduler is now reused and events are requeued, as on a master swing.
A manual partition's logical cron reads its counter every second without locks, so a schema reload could free the table under it (pre-existing crash in bdb_cursor_open). It now reads under the schema lock with fresh rootpages, and drops the scheduler lock while reading so it can't deadlock with schema-locked paths.

truncatesc is removed from the quarantine list (truncatesc_offline_generated stays quarantined).

Tested on a 3-node cluster: truncatesc plus the timepart, manual partition and logical cron tests.

markhannum mentioned this pull request

tests: failed partition collapse #6321

Open

markhannum force-pushed the truncatesc-timepart branch from 89c50bc to 56c2334 Compare

October 9, 2026 00:12

markhannum changed the title ~~truncatesc: test new-style time partitions, and fix what it found~~ truncatesc: add new style time partitions

markhannum force-pushed the truncatesc-timepart branch from 6867b97 to 7414964 Compare

October 9, 2026 04:58

roborivers suggested changes

View reviewed changes

roborivers left a comment

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

The first 10 failing tests are:
comdb2sys_queueodh_generated
consumer_non_atomic_default_consumer_generated **quarantined**
truncatesc_offline_generated [timeout] **quarantined**

roborivers suggested changes

View reviewed changes

roborivers left a comment

Cbuild submission: Error ⚠.
Regression testing: 2/731 tests failed ⚠.

The first 10 failing tests are:
recover_deadlock
consumer_non_atomic_default_consumer_generated **quarantined**
truncatesc_offline_generated [timeout] **quarantined**

markhannum force-pushed the truncatesc-timepart branch 2 times, most recently from afd4992 to 0affbe9 Compare

October 9, 2026 08:15

roborivers suggested changes

View reviewed changes

roborivers left a comment

Cbuild submission: Success ✓.
Regression testing: 2/731 tests failed ⚠.

The first 10 failing tests are:
consumer_non_atomic_default_consumer_generated **quarantined**
manual_partition

roborivers approved these changes

View reviewed changes

roborivers left a comment

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
consumer_non_atomic_default_consumer_generated **quarantined**
sc_redo_logicalsc_generated [timeout]
reco-ddlk-sql [timeout] **quarantined**
sql_logfill_autodisable [timeout]

markhannum force-pushed the truncatesc-timepart branch 2 times, most recently from c54688d to e0ee7be Compare

October 14, 2026 11:31

roborivers approved these changes

View reviewed changes

roborivers left a comment

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
guid
consumer_non_atomic_default_consumer_generated **quarantined**
reco-ddlk-sql [timeout] **quarantined**

markhannum force-pushed the truncatesc-timepart branch 2 times, most recently from b8e7636 to 9210158 Compare

October 14, 2026 15:19

roborivers approved these changes

View reviewed changes

roborivers left a comment

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
comdb2sys_queueodh_generated [db unavailable at finish]
consumer_non_atomic_default_consumer_generated **quarantined**

markhannum force-pushed the truncatesc-timepart branch 2 times, most recently from 86aba59 to f189de8 Compare

October 15, 2026 14:44

roborivers approved these changes

View reviewed changes

roborivers left a comment

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
comdb2sys [db unavailable at finish] **quarantined**
consumer_non_atomic_default_consumer_generated **quarantined**
reco-ddlk-sql [timeout] **quarantined**
sql_logfill_autodisable [timeout]

markhannum force-pushed the truncatesc-timepart branch from f189de8 to e58e001 Compare

October 15, 2026 16:16

roborivers approved these changes

View reviewed changes

roborivers left a comment

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
consumer_non_atomic_default_consumer_generated **quarantined**
reco-ddlk-sql [timeout] **quarantined**

markhannum force-pushed the truncatesc-timepart branch from e58e001 to 313131a Compare

October 16, 2026 13:49

roborivers approved these changes

View reviewed changes

roborivers left a comment

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
consumer_non_atomic_default_consumer_generated **quarantined**
sql_logfill_autodisable [timeout]

markhannum force-pushed the truncatesc-timepart branch from 313131a to ffe5e3f Compare

October 18, 2026 20:34

roborivers approved these changes

View reviewed changes

roborivers left a comment

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
comdb2sys_queueodh_generated [db unavailable at finish]
consumer_non_atomic_default_consumer_generated **quarantined**

markhannum force-pushed the truncatesc-timepart branch from ffe5e3f to 2df598b Compare

October 21, 2026 09:42

roborivers suggested changes

View reviewed changes

roborivers left a comment

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

The first 10 failing tests are:
constraints_partial_index_off_generated [failed with core dumped]
constraints_noreorder_generated [failed with core dumped]
sqlqueries
reco-ddlk-sql [timeout] **quarantined**
skipscan [timeout] **quarantined**

markhannum force-pushed the truncatesc-timepart branch from 2df598b to 066078e Compare

October 23, 2026 11:30

roborivers approved these changes

View reviewed changes

roborivers left a comment

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
reco-ddlk-sql [timeout] **quarantined**

markhannum force-pushed the truncatesc-timepart branch from 066078e to 15e132e Compare

October 23, 2026 16:14

markhannum force-pushed the truncatesc-timepart branch 2 times, most recently from 0389856 to 746b941 Compare

October 24, 2026 01:17

roborivers approved these changes

View reviewed changes

roborivers left a comment

Cbuild submission: Success ✓.
Regression testing: Success ✓.

roborivers approved these changes

View reviewed changes

roborivers left a comment

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
reco-ddlk-sql [timeout] **quarantined**

markhannum added 6 commits

October 26, 2026 14:46


          Keep the original table first when resizing a partition

70bcb2f

Collapsing the partition expects it at shard 0.

Signed-off-by: Mark Hannum <mhannum@bloomberg.net>


          Give shards added by a retention increase the partition's version

860e2ae

Later alters of the partition failed with "stale version".

Signed-off-by: Mark Hannum <mhannum@bloomberg.net>


          Requeue partition rollouts after a schema reload

d601f45

Reuse the timepart scheduler on reload instead of leaking a new one.

Signed-off-by: Mark Hannum <mhannum@bloomberg.net>


          Lock the schema when the logical cron reads its counter

f0d2439

A schema reload freed the table it was reading. The read drops the scheduler
lock so it can't deadlock with schema-locked paths, and keeps the old clock on error.

Signed-off-by: Mark Hannum <mhannum@bloomberg.net>


          tests: truncatesc covers new-style time partitions

dad01c7

Signed-off-by: Mark Hannum <mhannum@bloomberg.net>


          tests: unquarantine truncatesc

4bb347d

Signed-off-by: Mark Hannum <mhannum@bloomberg.net>

markhannum force-pushed the truncatesc-timepart branch from 746b941 to 4bb347d Compare

October 26, 2026 18:46

roborivers suggested changes

View reviewed changes

roborivers left a comment

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

The first 10 failing tests are:
reco-ddlk-sql [timeout] **quarantined**

This branch has not been deployed

No deployments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet