Repository navigation
test: add conformance suite and semver check - #60
Merged
Merged
Conversation
Adds the two things missing from the crate's confidence story: a public API surface check, and one shared contract that every backend is held to. `tests/conformance.rs` runs an identical set of checks against all five backends: capabilities are self-consistent and stable, the declared frame is accepted with a score in range, a wrong sample rate and an impossible frame length are rejected with the documented error, `reset()` restores a freshly constructed detector, a rejected call leaves streaming state untouched, the FrameAdapter preserves scores across chunk boundaries, and `timings()` tracks scored frames. The checks are phrased to fit every backend rather than the strictest one. WebRTC accepts 10/20/30 ms frames, so the contract asserts that the declared frame size works and that an impossible length is refused, not that only one length is accepted. FireRedVAD takes 160-sample calls but needs 400 samples before its first frame, so the timing contract is expressed in the steady state — which is what the trait already documents `frames` to mean. WebRTC is swept across all four sample rates and all three frame durations, and Silero across both of its rates. A `semver` CI job runs cargo-semver-checks with all features against the published release, so an accidental break in the public API fails the PR rather than being noticed after publishing. `make semver` runs it locally. Verified it fails as intended by removing a public method. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012PF9YKsA6GfchGTPDAWYzL
Merged
wavekat-eason
added a commit
that referenced
this pull request
Aug 30, 2026
## 🤖 New release * `wavekat-vad`: 0.1.16 -> 0.1.17 (✓ API compatible changes) <details><summary><i><b>Changelog</b></i></summary><p> <blockquote> ## [0.1.17](v0.1.16...v0.1.17) - 2026-08-30 ### Added - add pure-Rust earshot VAD backend ([#55](#55)) ### Other - add earshot section to the README ([#59](#59)) - add conformance suite and semver check ([#60](#60)) - cover earshot backend and frame adapter ([#57](#57)) </blockquote> </p></details> --- This PR was generated with [release-plz](https://github.com/release-plz/release-plz/). --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Eason WaveKat <eason@wavekat.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the two things missing from the crate's confidence story, per the gaps found while reviewing #55: a public API surface check, and one shared contract that every backend is held to.
tests/conformance.rsEvery backend implements
VoiceActivityDetector, and the README tells callers they can swap one for another. That promise rests on behaviour the trait cannot enforce at compile time. Until now each backend was tested only in its own module, in its own style, and to very different depths — earshot had 15 tests, webrtc 6 — so nothing would notice a backend quietly diverging on error types, reset semantics, or whattimings()counts.This runs one identical set of checks against all five:
capabilities_are_self_consistentframe_duration_msmatchesframe_sizeatsample_rate; capabilities do not drift as audio is processedthe_declared_frame_is_accepted0.0..=1.0a_wrong_sample_rate_is_rejectedInvalidSampleRate, echoing the offending ratean_impossible_frame_length_is_rejectedInvalidFrameSizewith the truegotlength and a usableexpectedreset_restores_a_fresh_detectorreset()reproduces a freshly constructed detector's score sequence exactlya_rejected_call_does_not_disturb_the_streamthe_frame_adapter_preserves_scoresFrameAdapterreproduce frame-aligned scores, with the carry always under one frametimings_count_scored_framesreset()does not clear timingsWebRTC is swept across all four sample rates and all three frame durations, and Silero across both of its rates.
Fitting the contract to all five backends
The checks are phrased to fit every backend rather than the strictest one — two cases forced that, and both are worth recording:
capabilities(). So the contract asserts that the declared frame size is accepted and that an impossible length is refused, rather than that only one length works.frames == callsand failed here — the backend was right and the check was wrong.ProcessTimings::framesis documented as "the number of frames that produced a result (excludes buffering-only frames)", so the check is now expressed in the steady state: once warm, each further call adds exactly one frame.All five backends pass.
semverCI jobrelease-plz.tomlcarries only tag and release settings, and there was no API check in CI. The "✓ API compatible changes" note on a release PR comes from release-plz's own check at release time — real, but not a PR gate. Nothing would have failed a PR that made a public type private or changed a signature.The new job runs
cargo-semver-checkswithall-features, so the feature-gated backend modules are covered too, against the published release.make semverruns the same thing locally.It is not decoration: I confirmed it fails as intended by removing
pubfromFrameAdapter::process_latest, which reportsinherent_method_missingandsemver requires new major version. Locally the run takes ~26s with all features, ~11s with defaults.Verification
cargo fmt --all --checkandcargo clippy --all-targets -- -D warningsclean on both the workspace and the full feature set. Tests pass across all eight feature combinations CI builds: empty, each backend alone,serde, and the full combination.🤖 Generated with Claude Code
https://claude.ai/code/session_012PF9YKsA6GfchGTPDAWYzL