Skip to content

fix(streaming): bound workers and drain writes on shutdown - #6811

Open
iutx wants to merge 1 commit into
erda-project:masterfrom
iutx:fix/clickhouse-storage-lifecycle
Open

iutx wants to merge 1 commit into
erda-project:masterfrom
iutx:fix/clickhouse-storage-lifecycle

Conversation

@iutx

@iutx iutx commented Oct 30, 2026

Copy link
Copy Markdown
Member

What this PR does / why we need it:

bound workers and drain writes on shutdown

Which issue(s) this PR fixes:

  • Fixes #your-issue_number
  • [Erda Cloud Issue Link](paste your link here)

Specified Reviewers:

/assign @luobily

ChangeLog

Language Changelog
🇺🇸 English Use fixed workers and a bounded queue to preserve backpressure and wait for accepted batches to finish processing during shutdown.
🇨🇳 中文 优化 streaming 批量写入失败重试以及修复安全关闭

Need cherry-pick to release versions?

Add comment like /cherry-pick release/1.0 when this PR is merged.

For details on the cherry pick process, see the cherry pick requests section under CONTRIBUTING.md.

Use fixed workers and a bounded queue to preserve backpressure and wait for accepted batches to finish processing during shutdown.

Fix retries after failed sends, skip backoff after the final attempt, return rejected writes to exporters, and reduce per-batch logging.

Keep profile-agent enabled and cover concurrent shutdown, cancellation, retries, configuration validation, and error propagation with tests.
@erda-bot
erda-bot requested a review from luobily October 30, 2026 02:13
@iutx
iutx marked this pull request as ready for review October 30, 2026 02:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Development

Successfully merging this pull request may close these issues.

2 participants