Skip to content

Plan: Add terminal rendering regression tests (3.12.3) - #809

Draft
leynos wants to merge 2 commits into
mainfrom
3-12-3-terminal-rendering-regression-tests
Draft

leynos wants to merge 2 commits into
mainfrom
3-12-3-terminal-rendering-regression-tests

Conversation

@leynos

@leynos leynos commented Sep 26, 2026 •

Copy link
Copy Markdown
Owner

Summary

This PR adds the ExecPlan for roadmap task 3.12.3, which verifies the
--color, --emoji, --progress, and --accessibility policies by
driving the real netsuke binary through pipes and pseudo-terminals (PTYs).

Key findings

A baseline PTY probe of the current binary shows that the policies are not
honoured end to end:

  • --color never --verbose and NO_COLOR=1 --verbose both emit 416 ANSI
    colour sequences through tracing on a terminal.
  • --color never still colours miette diagnostics and the tracing error
    line on a failing run.
  • --color never --help emits 113 colour sequences from clap.
  • --progress auto and --progress always resolve identically.

Proposed approach

  • A pure display-plan core (src/display_plan.rs) resolves every rendering
    decision once from the four policies and terminal facts gathered through
    existing injected seams; tracing, miette, clap, and the status
    reporter become adapters that apply the plan.
  • A Unix PTY harness in test_support built on the existing nix
    dev-dependency (no new crate), plus a byte classifier for colour, redraw,
    hyperlink, and glyph facts.
  • Exhaustive enumeration of the 5,184-point policy × fact domain, property
    tests for the string-facing lemmas, an end-to-end PTY/pipe matrix, BDD
    scenarios, and insta contract snapshots.

Decisions needing approval

  • D3: keep --progress auto ≡ always and document it.
  • D4: whether an empty NO_COLOR counts as set (recommend aligning with
    the convention: empty means unset).
  • D8: map colour, Unicode, and hyperlinks onto miette; defer the
    narratable handler.
  • D9: clap help colour follows a startup hint from --color and
    NETSUKE_COLOR.
  • D11: --color always styles output even when piped.

The draft was reviewed and revised by a five-member expert panel
(accessibility, testing and verification, architecture, terminal systems,
plan quality); see the revision note at the end of the plan.

Validation

Documentation-only change: make check-fmt, make markdownlint (including
spelling), and make nixie pass.

References

🤖 Generated with Claude Code

Summary by Sourcery

Define and seek approval for a comprehensive plan to verify and repair Netsuke's terminal rendering policies through real binary output tests.

Enhancements:

  • Add a draft ExecPlan for end-to-end terminal rendering regression coverage across colour, emoji, progress, and accessibility policies, including PTY and pipe testing, display-plan resolution, emitter integration, and contract verification.

Documentation:

  • Document the proposed terminal rendering test strategy, baseline findings, implementation milestones, validation requirements, and open policy decisions.

Tests:

  • Plan exhaustive policy-domain checks, byte-level output classification, Unix PTY harness coverage, end-to-end binary tests, BDD scenarios, property tests, and contract snapshots.

leynos and others added 2 commits September 27, 2026 00:41
Add the initial ExecPlan for roadmap task 3.12.3, which verifies the
`--color`, `--emoji`, `--progress`, and `--accessibility` policies by
driving the real binary through pipes and pseudo-terminals.

The draft records a baseline probe showing that `--color never` and
`NO_COLOR` do not suppress ANSI colour in verbose `tracing` output,
`miette` diagnostics, or `clap` help on a terminal, and proposes a
pure display-plan core with adapters bound to its decisions.

The plan is a DRAFT awaiting expert-panel revision and user approval.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Fold in findings from a five-member review (accessibility, testing and
verification, architecture, terminal systems, and plan quality).

- Split the colour decision: `--color never` and `NO_COLOR` leaks are
  defects to fix; `--color always` styling pipes is new open decision
  D11.
- Gather terminal facts once through existing injected seams, reduce
  environment strings in one `EnvSignals` type, delete the ambient
  wrappers, and enforce single resolution with a `disallowed-methods`
  lint (D12).
- Cover the missed consumers (`runner/help.rs`, `ExecutionContext`, and
  indicatif's own terminal check) and model `TERM=dumb`/unset `TERM`.
- Isolate child processes with `env_clear()`, harden the PTY harness
  against hangs and flakes, widen the byte classifier, and add
  completeness guards to the exhaustive sweep.
- Write each red cell and scenario in the EP-M3 commit that fixes it,
  and correct BDD step phrases, documentation coverage, and the Verus
  rationale.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 26, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai

sourcery-ai Bot commented Sep 26, 2026

Copy link
Copy Markdown
Contributor

Reviewer's Guide

This documentation-only PR adds a DRAFT ExecPlan that diagnoses current terminal-rendering policy gaps and lays out an approval-gated architecture and verification programme using a pure display-plan core, Unix PTY/pipe tests, exhaustive/property-based checks, BDD scenarios, and contract snapshots.

File-Level Changes

Change Details Files
Adds a draft execution plan for validating terminal rendering policies against observable binary output.
  • Defines expected behaviour for colour, emoji, progress, accessibility, and terminal/pipe handling.
  • Records baseline PTY findings, open product decisions, constraints, risks, and residual gaps.
  • Decomposes implementation into harness, regression, display-plan, adapter, and documentation milestones.
docs/execplans/3-12-3-terminal-rendering-regression-tests.md
Proposes a pure display-plan architecture to centralize rendering decisions and feed presentation adapters.
  • Introduces planned DisplayPolicies, TerminalFacts, EnvSignals, and DisplayPlan interfaces.
  • Resolves policy and terminal facts once, then passes the plan to tracing, miette, clap, and status-reporting adapters.
  • Preserves presentation/build separation and removes ambient resolution wrappers.
docs/execplans/3-12-3-terminal-rendering-regression-tests.md
Specifies a cross-platform regression-testing strategy using pipes, Unix PTYs, property tests, exhaustive domain checks, BDD scenarios, and snapshots.
  • Builds a nix-based PTY harness with isolated environments, concurrent stream draining, timeouts, process-group cleanup, and stress validation.
  • Adds a byte classifier for ANSI colour, redraw, hyperlinks, escapes, glyphs, carriage returns, and bidi isolates.
  • Enumerates the 5,184-point policy/fact domain and derives end-to-end expectations from the pure display plan.
  • Adds planned contract snapshots and behavioural scenarios for policy combinations and terminal transports.
docs/execplans/3-12-3-terminal-rendering-regression-tests.md
Plans adapter changes to make all controlled emitters honour the resolved rendering policy.
  • Configures tracing ANSI output from the plan after startup configuration is resolved.
  • Maps diagnostic colour, Unicode, and hyperlink settings into miette.
  • Passes a startup colour hint to clap help/version rendering and defines handling for pre-configuration errors.
  • Aligns indicatif progress targets and reporter selection with accessibility, terminal, JSON, and progress decisions.
docs/execplans/3-12-3-terminal-rendering-regression-tests.md
Defines approval gates, implementation sequencing, and validation requirements for the draft plan.
  • Requires explicit approval of decisions D3, D4, D8, D9, and D11 before implementation.
  • Sets production-scope, dependency, snapshot, flakiness, and timing escalation thresholds.
  • Requires formatting, type, lint, test, Markdown, and documentation validation at milestone boundaries.
docs/execplans/3-12-3-terminal-rendering-regression-tests.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant