Skip to content

Per-host status writes rewrite the full CHI .status, causing repeated update conflicts under reconcile concurrency #2064

Description

@atsarevskiy

Problem

As each host finishes reconciling, the operator writes the entire CHI .status back with a read-modify-write (pkg/controller/chi/kube/cr.go, statusUpdateProcess): Get the current CHI → EnsureStatus().CopyFrom(...) → full Update, wrapped in statusUpdateRetry (up to maxRetryAttempts = 50, with a sleep between attempts).

When many hosts reconcile concurrently, they all update the same single CHI object. Kubernetes allows only one update to an object at a time (optimistic concurrency), so the others get:

Operation cannot be fulfilled on clickhouseinstallations.clickhouse.altinity.com "<chi>":
the object has been modified; please apply your changes to the latest version and try again

…and then re-Get and retry. Under concurrency this leads to repeated write conflicts: a single host's status write can take several attempts (observed attempt 2, 3, 4, 6, …) with backoff between each.

Impact

At higher reconcile concurrency (>50) the per-host status write can become a significant per-host cost (observed in the tens of seconds), most of it spent on conflict retries and backoff rather than the update itself. It also results in repeated full-object writes to the apiserver. Because the contention comes from concurrency, the cost tends to increase as reconcile concurrency is raised.

Fix ideas

  • Use a status subresource Patch (JSON/strategic-merge patch of just the changed fields) instead of a read-modify-write of the whole .status.
  • Or coalesce/batch host-completion status updates rather than one full-object write per completing host (for example, a single writer that flushes accumulated per-host transitions).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions