Skip to content

[docs] Improve consistency and configuration guidance - #1284

Merged
wenjin272 merged 14 commits into
apache:mainfrom
joeyutong:codex/docs-consistency-configuration-1234
Oct 20, 2026
Merged

wenjin272 merged 14 commits into
apache:mainfrom
joeyutong:codex/docs-consistency-configuration-1234

Conversation

@joeyutong

@joeyutong joeyutong commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Linked issue: #1234

Purpose of change

Consistency and recovery guidance is spread across deployment and feature pages, while the configuration reference mixes option lookup with long feature tutorials. Add a dedicated Operations guide and separate usage guidance from the configuration reference.

  • Add a Consistency guide covering mode selection, Java/Python/YAML setup, reuse of saved Action completions and durable call outcomes during recovery, limits on external effects, and Action State Store retention and cleanup.
  • Organize the configuration reference into six task-based groups, sort keys within each group, and clarify loading precedence, parameter descriptions, and backend constraints. Preserve all 49 option names, types, and defaults.
  • Move checkpoint-aligned Kafka cleanup prerequisites, procedures, and CLI guidance to Consistency and Trace Log usage guidance to Monitoring. Link feature pages directly to the shared guidance and clarify when external effects may repeat.

Tests

  • Built the full site with Hugo 0.110.0 extended: 75 pages.
  • Validated 3372 internal links and fragments across 80 generated HTML files, including retained anchors.
  • Checked all 49 option names, types, and defaults against Java/Python definitions, and reviewed parameter descriptions against runtime behavior.
  • Checked example syntax and API identifiers, and verified that cleanup prerequisites render in a collapsed block before the four-step procedure and command examples in a collapsed block immediately after it.
  • git diff --check upstream/main passed.

Runtime tests were not rerun for this documentation-only change. Java examples were not compiled, and live-backend cleanup and cross-version recovery were not exercised.

API

No public API or configuration key, type, or default changes.

Documentation

  • doc-needed
  • doc-not-needed
  • doc-included

Was this patch authored or co-authored using generative AI tooling?

  • Yes
  • No

Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6)

Add an operations guide for consistency setup, recovery, retention, cleanup,
and migration, and organize the configuration reference by user task.
Consolidate shared recovery guidance and retain feature-specific contracts.

Related to apache#1234.

Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6)
Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6
AI-Contributed/Feature: 1153/1153
AI-Contributed/UT: 0/0
@github-actions github-actions Bot added doc-included Your PR already contains the necessary documentation updates. fixVersion/0.4.0 priority/major Default priority of the PR or issue. labels Oct 20, 2026
joeyutong and others added 2 commits October 20, 2026 10:38
Restore six feature pages and remove optional Actions tutorial expansions.
Retain the shared-guide extraction, navigation, and corrections to overstated
recovery guarantees.

Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6)
Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6
AI-Contributed/Feature: 141/141
AI-Contributed/UT: 0/0
Reduce repeated mode and recovery explanations, smooth transitions, and keep
cleanup configuration in the relevant procedure step. Preserve replay conditions,
backend defaults, code examples, and migration requirements.

Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6)
Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6
AI-Contributed/Feature: 159/159
AI-Contributed/UT: 0/0
@github-actions github-actions Bot added doc-included Your PR already contains the necessary documentation updates. and removed doc-included Your PR already contains the necessary documentation updates. labels Oct 20, 2026
joeyutong and others added 3 commits October 20, 2026 11:09
Update feature and setup references to reach the relevant recovery, retry,
and Trace Log sections directly after moving the shared guidance.

Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6)
Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6
AI-Contributed/Feature: 20/20
AI-Contributed/UT: 0/0
Remove the premature idempotency and reconciliation recommendation from the
mode comparison. The recovery section explains its context and limits.

Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6)
Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6
AI-Contributed/Feature: 2/2
AI-Contributed/UT: 0/0
State that Flink Agents has no Fluss equivalent of the Kafka cleanup mechanisms.

Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6)
Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6
AI-Contributed/Feature: 5/5
AI-Contributed/UT: 0/0
@github-actions github-actions Bot added doc-included Your PR already contains the necessary documentation updates. and removed doc-included Your PR already contains the necessary documentation updates. labels Oct 20, 2026
joeyutong and others added 3 commits October 20, 2026 11:37
Connect recovery decisions, cleanup conditions, and upgrade steps to their
causes and consequences. Group related setup details and preserve the existing
technical conditions, comparison tables, examples, and headings.

Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6)
Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6
AI-Contributed/Feature: 99/99
AI-Contributed/UT: 0/0
Explain how to manage Fluss records after identifying the absence of Kafka-style
cleanup mechanisms in Flink Agents.

Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6)
Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6
AI-Contributed/Feature: 3/3
AI-Contributed/UT: 0/0
Keep the fresh-start requirement in the 0.4 migration note and remove the
redundant subsection and source-position guidance.

Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6)
Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6
AI-Contributed/Feature: 12/12
AI-Contributed/UT: 0/0
@github-actions github-actions Bot added doc-included Your PR already contains the necessary documentation updates. and removed doc-included Your PR already contains the necessary documentation updates. labels Oct 20, 2026
@joeyutong joeyutong self-assigned this Oct 20, 2026
joeyutong and others added 2 commits October 20, 2026 14:37
Keep the cleanup procedure focused on four key steps and recovery restrictions.
Place complete commands and classpath requirements in a collapsed examples block.

Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6)
Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6
AI-Contributed/Feature: 113/113
AI-Contributed/UT: 0/0
Place the collapsed command examples immediately after the four cleanup steps,
ahead of failure handling and recovery restrictions.

Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6)
Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6
AI-Contributed/Feature: 30/30
AI-Contributed/UT: 0/0
@joeyutong
joeyutong marked this pull request as ready for review October 20, 2026 06:55
@github-actions github-actions Bot added doc-included Your PR already contains the necessary documentation updates. and removed doc-included Your PR already contains the necessary documentation updates. labels Oct 20, 2026

@wenjin272 wenjin272 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for improving the consistency and configuration guidance! I left two comments on the placement of upgrade/migration guidance and the remaining Kafka cleanup details.

checkpoint-aligned cleanup for the Fluss Action State Store. Instead, use Fluss's
own log retention settings to manage record retention and cleanup.

## Upgrades and migration

@wenjin272 wenjin272 Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can remove the “Upgrades and migration” section from this guide. We plan to design compatibility for the Action State Store and Flink operator state in the next release. For now, the guide should describe current recovery requirements without implying broader upgrade compatibility. The existing 0.4 format change can go into release-specific migration notes.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the general upgrade discussion and kept current recovery requirements in the recovery section. The known 0.4 format change now lives under Migration notes > Flink Agents 0.4 on the same page, keeping the Action State Store guidance together.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, my earlier wording was unclear. Please remove the 0.4 migration note from this guide. I’ll mention the breaking change in the release notes; we don’t need to provide a migration guide for the current 0.x releases. The removal of the general upgrade discussion looks good—thanks!

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the 0.4 migration note and updated the Deployment text and PR description accordingly.

Per-key tombstones and checkpoint-aligned cleanup are mutually exclusive. Tombstones are not tied to the selected recovery boundary and could invalidate a checkpoint that checkpoint-aligned cleanup promises to retain.

To migrate a job that previously emitted tombstones, first stop it cleanly so its Kafka producer closes and all accepted tombstone sends finish. Restart from the newest recovery point that is valid under the tombstone mode's existing recovery trade-off, with `kafkaActionStateTombstoneEnabled=false` and no cleanup control topic configured. After that attempt completes a new checkpoint or savepoint, stop it again, create and apply the first cleanup plan from that new recovery point, and restart from the same point with `kafkaActionStateCleanupControlTopic` configured. The new marker is after every old tombstone; selecting an older recovery point cannot provide the same guarantee because replay may still encounter those tombstones.
`KafkaActionStateCleanupTool` requires the following configuration and recovery

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we move the remaining “Checkpoint-aligned Kafka cleanup” subsection into the Consistency guide, including its prerequisites, topic policies, and CLI options? The cleanup procedure currently sends readers back to Configuration for these details. Keeping them together would make the procedure self-contained, while Configuration retains concise option descriptions and a link to the guide.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved the cleanup prerequisites, topic policies, and CLI options into Consistency. Configuration now keeps concise option descriptions and a direct link to the procedure.

joeyutong and others added 2 commits October 20, 2026 19:18
Keep current recovery requirements in the recovery section and isolate the
known 0.4 format change in version-specific migration notes. Consolidate Kafka
cleanup prerequisites, topic requirements, and CLI guidance in Consistency.

Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6)
Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6
AI-Contributed/Feature: 105/105
AI-Contributed/UT: 0/0
Keep the guide focused on current consistency and recovery behavior.
Remove the 0.4 migration notes and align Deployment's link description.

Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6)
Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6
AI-Contributed/Feature: 11/11
AI-Contributed/UT: 0/0

@wenjin272 wenjin272 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once the conflicts are resolved.

Preserve the upstream Subagents rewrite and retain the checkpoint/store
prerequisite and direct links to Consistency in its recovery section.

Generated-by: Codex CLI 0.163.0-alpha.10.2 (GPT-6)
Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6
AI-Contributed/Feature: 8/1093
AI-Contributed/UT: 0/472
@wenjin272
wenjin272 merged commit 47daff5 into apache:main Oct 20, 2026
29 checks passed
@joeyutong
joeyutong deleted the codex/docs-consistency-configuration-1234 branch October 30, 2026 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-included Your PR already contains the necessary documentation updates. fixVersion/0.4.0 priority/major Default priority of the PR or issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants