Skip to content

INFR: Coverage measurement excludes all numba-jitted code and counts the test tree #871

Closed

Description

@mmcky

Problem

.coveragerc lists @jit, @jit\(.*nopython=True, @njit, @overload and @guvectorize\(.*nopython=True under [report] exclude_lines. The intent is sound — jit-compiled code is not traced by CPython, so its lines would otherwise read as uncovered. But coverage.py excludes the entire decorated function when a decorator line matches an exclusion pattern, not just the decorator line. The result is that the hottest, most algorithmically intricate code in the library is not merely uncounted — it is unmeasurable.

Measured by walking the AST and comparing against what coverage report attributes to each file:

Module AST statements Inside @jit Statements coverage sees Reported
optimize/scalar_maximization.py 79 77 2 100%
optimize/root_finding.py 212 202 10 100%
optimize/nelder_mead.py 113 108 4 100%
optimize/linprog_simplex.py 169 150 18 100%
markov/utilities.py 66 63 2 100%
Package total 5,954 1,625 (27%) 4,019 —

The entire quantecon.optimize subpackage — 2,396 LOC of numerical optimisation — reports 100% while measuring almost nothing. The code is not untested (it has 1,052 LOC of tests); it is unmeasured: no instrument can tell which branches of the numerical kernels are exercised, or detect regression in that coverage.

A second, independent defect compounds it: [run] source = quantecon includes quantecon/**/tests/, so the test files count toward the percentage. CI computes and Coveralls displays 94%; source-only coverage is 87%.

Proposed fix

Run the suite twice in CI: once normally, and once with NUMBA_DISABLE_JIT=1, which makes jitted functions execute as ordinary Python and become fully traceable; combine with coverage combine. This is the standard approach for numba projects and requires no library-code changes. At the same time, narrow source/omit so the test tree is excluded from the headline figure.

Expect the reported percentage to fall when this lands — that is the point; the new number will be true and becomes the baseline for future coverage work.

Acceptance criteria

  • CI runs the suite a second time under NUMBA_DISABLE_JIT=1 and combines the results
  • The jit-decorator patterns are removed from exclude_lines, or scoped to the non-instrumented pass only
  • source/omit excludes quantecon/**/tests/ from the reported figure
  • The new baseline percentage is recorded in this thread
  • quantecon/optimize/* reports statement counts consistent with AST statement counts

From the July 2026 technical-debt audit (AI-assisted; claims verified against 28d4b3b on 2026-07-25).

Activity

  1. added a commit that references this issue on Aug 7, 2026
    0c5b97a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions