Skip to content

chore(infra): do not partition AuditEvents by time — record why, and what to do instead #505

Closed

Description

@mforce

Parked from the #494 review, so the reasoning is not rediscovered.

Question

AuditEvents is append-only with no purge sweep, so it only grows. #494 adds ~1 row per daily entry per day (DailyEntry.Create + DailyEntry.Submit). Should we get ahead of it by range-partitioning the table by month?

Answer: no, and not merely "not yet"

Time partitioning is the wrong axis for this table. Partition pruning only helps a query that filters on the partition key. The dominant read — AuditEventRepository.GetProvenanceAsync, run on every Flocks / Egg grades / Daily entries / Sales / Expenses page load — filters on AccountId + EntityType + EntityId and carries no date predicate at all. Partitioning by month converts one index lookup on IX_AuditEvents_AccountId_EntityId into one lookup per partition, getting worse every month. It would degrade the exact query #494 introduced.

Three further costs, none of them the deciding one:

  1. Postgres requires the partition key in every unique constraint, so the PK moves from Id to (Id, OccurredAtUtc). Converting an existing table means create-new + copy + rename, not an ALTER.
  2. Partition maintenance becomes a new failure mode with teeth. If next month partition does not exist, INSERT fails; audit writes share the transaction with the change they record (F23: Audit log for critical changes — same-transaction append-only events + admin viewer #93), so every audited mutation fails with it. We run no partition-maintenance job today.
  3. It drags in a migration (feat(eggs): make cracked and dirty eggs sellable stock via condition grades (#396) #407), a docs/schema/ regeneration (chore(schema): generate PostgreSQL schema documentation #417), and a sim-harness pass (Sim harness (#243) rotted silently: 4 breakages, no CI ever ran it #370).

Current size, for the record

~2 rows per flock per day. 10 flocks -> ~7.3k rows/year; 100 flocks -> ~73k rows/year, roughly 20 MB/year. Not a problem at any plausible farm size.

If it ever does bite, in order

  1. Archive or roll up rows older than the retention the farm actually needs.
  2. Partition by AccountId — the column the provenance query does filter on, so pruning would apply. Only meaningful once the deployment is genuinely multi-tenant.

Separate, and real

The Audit page pages with OFFSET, which degrades on deep pages as the table grows. Independent of partitioning and of #494; worth keyset pagination on its own merits if it ever shows up in practice.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions