Repository navigation
[docs] Improve consistency and configuration guidance - #1284
Conversation
Add an operations guide for consistency setup, recovery, retention, cleanup, and migration, and organize the configuration reference by user task. Consolidate shared recovery guidance and retain feature-specific contracts. Related to apache#1234. Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6) Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-6 AI-Contributed/Feature: 1153/1153 AI-Contributed/UT: 0/0
Restore six feature pages and remove optional Actions tutorial expansions. Retain the shared-guide extraction, navigation, and corrections to overstated recovery guarantees. Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6) Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-6 AI-Contributed/Feature: 141/141 AI-Contributed/UT: 0/0
Reduce repeated mode and recovery explanations, smooth transitions, and keep cleanup configuration in the relevant procedure step. Preserve replay conditions, backend defaults, code examples, and migration requirements. Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6) Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-6 AI-Contributed/Feature: 159/159 AI-Contributed/UT: 0/0
Update feature and setup references to reach the relevant recovery, retry, and Trace Log sections directly after moving the shared guidance. Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6) Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-6 AI-Contributed/Feature: 20/20 AI-Contributed/UT: 0/0
Remove the premature idempotency and reconciliation recommendation from the mode comparison. The recovery section explains its context and limits. Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6) Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-6 AI-Contributed/Feature: 2/2 AI-Contributed/UT: 0/0
State that Flink Agents has no Fluss equivalent of the Kafka cleanup mechanisms. Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6) Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-6 AI-Contributed/Feature: 5/5 AI-Contributed/UT: 0/0
Connect recovery decisions, cleanup conditions, and upgrade steps to their causes and consequences. Group related setup details and preserve the existing technical conditions, comparison tables, examples, and headings. Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6) Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-6 AI-Contributed/Feature: 99/99 AI-Contributed/UT: 0/0
Explain how to manage Fluss records after identifying the absence of Kafka-style cleanup mechanisms in Flink Agents. Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6) Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-6 AI-Contributed/Feature: 3/3 AI-Contributed/UT: 0/0
Keep the fresh-start requirement in the 0.4 migration note and remove the redundant subsection and source-position guidance. Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6) Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-6 AI-Contributed/Feature: 12/12 AI-Contributed/UT: 0/0
Keep the cleanup procedure focused on four key steps and recovery restrictions. Place complete commands and classpath requirements in a collapsed examples block. Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6) Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-6 AI-Contributed/Feature: 113/113 AI-Contributed/UT: 0/0
Place the collapsed command examples immediately after the four cleanup steps, ahead of failure handling and recovery restrictions. Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6) Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-6 AI-Contributed/Feature: 30/30 AI-Contributed/UT: 0/0
wenjin272
left a comment
There was a problem hiding this comment.
Thanks for improving the consistency and configuration guidance! I left two comments on the placement of upgrade/migration guidance and the remaining Kafka cleanup details.
| checkpoint-aligned cleanup for the Fluss Action State Store. Instead, use Fluss's | ||
| own log retention settings to manage record retention and cleanup. | ||
|
|
||
| ## Upgrades and migration |
There was a problem hiding this comment.
I think we can remove the “Upgrades and migration” section from this guide. We plan to design compatibility for the Action State Store and Flink operator state in the next release. For now, the guide should describe current recovery requirements without implying broader upgrade compatibility. The existing 0.4 format change can go into release-specific migration notes.
There was a problem hiding this comment.
Removed the general upgrade discussion and kept current recovery requirements in the recovery section. The known 0.4 format change now lives under Migration notes > Flink Agents 0.4 on the same page, keeping the Action State Store guidance together.
There was a problem hiding this comment.
Sorry, my earlier wording was unclear. Please remove the 0.4 migration note from this guide. I’ll mention the breaking change in the release notes; we don’t need to provide a migration guide for the current 0.x releases. The removal of the general upgrade discussion looks good—thanks!
There was a problem hiding this comment.
Removed the 0.4 migration note and updated the Deployment text and PR description accordingly.
| Per-key tombstones and checkpoint-aligned cleanup are mutually exclusive. Tombstones are not tied to the selected recovery boundary and could invalidate a checkpoint that checkpoint-aligned cleanup promises to retain. | ||
|
|
||
| To migrate a job that previously emitted tombstones, first stop it cleanly so its Kafka producer closes and all accepted tombstone sends finish. Restart from the newest recovery point that is valid under the tombstone mode's existing recovery trade-off, with `kafkaActionStateTombstoneEnabled=false` and no cleanup control topic configured. After that attempt completes a new checkpoint or savepoint, stop it again, create and apply the first cleanup plan from that new recovery point, and restart from the same point with `kafkaActionStateCleanupControlTopic` configured. The new marker is after every old tombstone; selecting an older recovery point cannot provide the same guarantee because replay may still encounter those tombstones. | ||
| `KafkaActionStateCleanupTool` requires the following configuration and recovery |
There was a problem hiding this comment.
Could we move the remaining “Checkpoint-aligned Kafka cleanup” subsection into the Consistency guide, including its prerequisites, topic policies, and CLI options? The cleanup procedure currently sends readers back to Configuration for these details. Keeping them together would make the procedure self-contained, while Configuration retains concise option descriptions and a link to the guide.
There was a problem hiding this comment.
Moved the cleanup prerequisites, topic policies, and CLI options into Consistency. Configuration now keeps concise option descriptions and a direct link to the procedure.
Keep current recovery requirements in the recovery section and isolate the known 0.4 format change in version-specific migration notes. Consolidate Kafka cleanup prerequisites, topic requirements, and CLI guidance in Consistency. Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6) Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-6 AI-Contributed/Feature: 105/105 AI-Contributed/UT: 0/0
Keep the guide focused on current consistency and recovery behavior. Remove the 0.4 migration notes and align Deployment's link description. Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6) Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-6 AI-Contributed/Feature: 11/11 AI-Contributed/UT: 0/0
wenjin272
left a comment
There was a problem hiding this comment.
LGTM once the conflicts are resolved.
Preserve the upstream Subagents rewrite and retain the checkpoint/store prerequisite and direct links to Consistency in its recovery section. Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6) Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-6 AI-Contributed/Feature: 8/1093 AI-Contributed/UT: 0/472
Linked issue: #1234
Purpose of change
Consistency and recovery guidance is spread across deployment and feature pages, while the configuration reference mixes option lookup with long feature tutorials. Add a dedicated Operations guide and separate usage guidance from the configuration reference.
Tests
git diff --check upstream/mainpassed.Runtime tests were not rerun for this documentation-only change. Java examples were not compiled, and live-backend cleanup and cross-version recovery were not exercised.
API
No public API or configuration key, type, or default changes.
Documentation
doc-neededdoc-not-neededdoc-includedWas this patch authored or co-authored using generative AI tooling?
Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6)