Skip to content

Enforce function-level coverage risk in CI #193

Closed

Description

@HemSoft

identify-fingerprint: identify:quality:.github/workflows/ci.yml:function-level-coverage-risk-gate-omitted

Summary

Measure function-level coverage and complexity for production Swift, publish the results, and prevent new uncovered complexity from entering through otherwise-green pull requests.

Affected area

  • .github/workflows/ci.yml
  • new scripts/function-risk/ measurement and policy files
  • focused parser and failure-fixture tests
  • a committed baseline and FUNCTION-RISK.md
  • CHANGELOG.md under Unreleased / Developer Experience

Evidence

  • The current CI workflow runs smoke checks and one ordinary xcodebuild test. It does not enable coverage, save an xcresult, publish function measurements, or enforce a coverage-risk policy.
  • A fresh coverage-enabled run on Xcode 26.6 passed all 451 tests but measured CodexBarMac.app at 14,003 of 22,135 executable lines, or 63.26%. The passing suite alone does not expose where untested control flow accumulates.
  • Xccov reports several production workflows at 0%: ProviderSettingsView.saveOpenCodeCredential() has 54 executable lines, signInWithCodex() has 47, signInWithCopilot() has 46, signInWithClaude() has 42, and LaunchAtLoginManager.refreshFromSystem() has 42. Large SwiftUI bodies are also unexecuted. This issue does not claim that every UI declaration needs unit coverage; the baseline must distinguish reviewed platform and presentation exceptions from silent gaps.
  • The iOS sibling's issue #306 and merged implementation provide a working bounded design: function line coverage joined to pinned complexity measurements, an explicit baseline, CRAP thresholds, unmatched-symbol handling, artifacts, and failure fixtures. The Mac needs its own measurements and exceptions rather than copying iOS baseline values.
  • Immediately before filing, the repository had 2 open issues and a filing budget of 18. A fresh exact-fingerprint and semantic search across every open and closed issue and pull request found no function-risk, CRAP, or per-function coverage gate. Closed coverage issues such as Cover browser OAuth token exchanges end to end #141, Cover concurrent credential-refresh coalescing #147, and Cover concurrent Gemini tier-fetch coalescing #149 add tests for specific paths but do not maintain a repository-wide regression policy.
  • ./test.sh, the coverage-enabled 451-test run, Xcode static analysis, current-head CI, and git diff --check are clean at c9a5581. This is a missing regression signal, not a current test failure.

Acceptance criteria

  • Enable code coverage for the existing Mac test job and save its xcresult at a unique runner path.
  • Measure production functions with a pinned, documented complexity tool and xccov function executable-line coverage. Publish the function identity, source path, complexity, covered and executable lines, coverage fraction, CRAP score, and any unmatched or excluded declaration.
  • Use CRAP = CC^2 * (1 - coverage)^3 + CC. Document whether the available tooling measures line or branch coverage and do not present line coverage as branch coverage.
  • Commit a reviewed baseline from the current Mac source and Xcode toolchain. Give each existing high-risk or unmeasurable production declaration an explicit reason and bounded follow-up policy; never regenerate the baseline automatically in CI.
  • Fail the required test job for new production functions above CRAP 30, increases above reviewed high-risk ceilings, malformed evidence, missing tracked production files, or unexplained measurement gaps. Publish scores from 15 through 30 as a review queue.
  • Treat generated, synthesized, preview-only, and platform-inactive declarations explicitly. Do not silently mark them covered or force the whole app to 100% coverage.
  • Add deterministic tests for completed clean evidence, a new uncovered complex function, worsened baseline risk, malformed inputs, duplicate identities, missing source extraction, and hash-bound exceptions. Prove a disposable harmless regression fails the gate, then remove it.
  • Upload a compact artifact on pass and failure, preserve the full xcresult on test failure, and document a reproducible local command using Xcode 26.6.
  • Keep all existing tests and smoke checks passing and add a CHANGELOG.md Developer Experience entry.

Activity

  1. HemSoft commented on Sep 9, 2026

    @HemSoft
    CollaboratorAuthor
    No description provided.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions