Skip to content

feat(id): bounded raw input token assembler (inert, #787 slice B) - #5512

Merged
dezren39 merged 2 commits into
mainfrom
queue/7947-787-input-assembler
Nov 5, 2026
Merged

dezren39 merged 2 commits into
mainfrom
queue/7947-787-input-assembler

Conversation

@dezren39

@dezren39 dezren39 commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Slice B of the landed #787 design (doc/2026-11-01_design_id787.md §4.2 and §6 row B).

Part of #787. This is one of six slices and does not close the issue; C–F remain.

What this adds

A new, inert Rust module pkgs/id/src/input_framing.rs (plus its registration in
src/lib.rs) that turns arbitrary transport chunks into complete logical key tokens:

  • Retains an incomplete UTF-8 scalar, CSI or SS3 sequence across push calls and
    emits one token per complete key, independent of how the transport split the bytes.
    Plain text emits one token per Unicode scalar.
  • Token bytes are never rewritten; what the assembler emits is exactly what the
    transport received, so the durable Input(Vec<u8>) boundary is untouched.
  • Bounds: 256-byte cap per pending token and a 5-second incomplete-token deadline
    (§4.2 safety defaults). Overlong tokens, invalid UTF-8, incomplete tokens at
    EOF/deadline and queue loss are fatal; the caller must close the session, because
    a missing prefix cannot be resynchronised.
  • Connection-local state: one InputAssembler per raw connection, so interleaved
    connections cannot mix partial tokens.
  • ReplyFilter seam only: push_filtered drops recognized control replies while
    preserving key order, for F13.4's demultiplexer to implement in slice E. No filter
    is provided that swallows unrecognized keys.

No activation. world_ssh does not call the module, the durable Input event,
journal format, guest ABI, browser encoder and lease/negotiation paths are unchanged.

Tests

cd pkgs/id && cargo test --lib input_framing::tests
# 16 passed; 0 failed

Coverage follows the §6 done check: every split point and byte boundary of the fixture
corpus, coalesced keys, split probe-reply + key order, cap boundary (exactly 256 passes,
257 fails), unterminated over-cap, deadline, EOF with/without pending bytes, invalid
UTF-8 and overlong forms, malformed escapes, queue loss and interleaved connections.

cargo fmt --all -- --check passes. Default-feature targeted crate tests pass (16/16),
and cargo check --all-targets passes with the crate's default features. The isolated
module also passes clippy-driver with the crate's Clippy groups and repo-specific
allow-list. Both cargo clippy --lib and cargo clippy --lib --all-features could not
complete: they failed with E0514 incompatible-compiler metadata errors in existing
dependency artifacts (e.g. p2panda_core, thiserror, anyhow) before reaching the
crate lint pass. cargo check --all-targets --all-features also cannot compile in this
checkout because the RustEmbed input web/dist has not been generated. The
--no-default-features test variant cannot compile on current main because of
pre-existing world/petname feature-gated references in chat.rs/invite.rs; the
default-feature test variant passed.

No automated CI jobs were triggered for this PR. .github/workflows/id-check.yml has
only schedule and workflow_dispatch triggers (its header still mentions PRs), so
package-level PR CI coverage is absent on this branch; the required code check was run
locally as described above. roc-test passes all 83 Roc expectations.

Slice status after this PR

@dezren39
dezren39 requested a review from a team as a code owner November 4, 2026 14:44
@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 29 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 8c31c867-14d4-44ee-a861-57d1339dbb9b
Commits

Reviewing files that changed from the base of the PR and between 329b23b and 24a57d5.

Files selected for processing: 2
  • pkgs/id/src/input_framing.rs
  • pkgs/id/src/lib.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-04T14:46:51.364068Z fcc276a PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fcc276a709

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

];

#[test]
fn plain_text_emits_one_token_per_scalar() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Prefix the new test names with test_

All 16 new test functions omit the test_ prefix explicitly required by the scoped naming convention in pkgs/id/AGENTS.md. Rename these functions to follow that documented convention; the same correction applies throughout this test module.

AGENTS.md reference: pkgs/id/AGENTS.md:L140-L140

Useful? React with 👍 / 👎.

@dezren39
dezren39 merged commit a9a7748 into main Nov 5, 2026
1 check passed
@dezren39
dezren39 deleted the queue/7947-787-input-assembler branch November 5, 2026 01:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant