Repository navigation
Conversation
Data-quality inspection and the data-pipeline half of the HG-DAgger
recovery-data feature (the interactive `rd infer --intervene` collection
loop follows in a later commit).
- rd inspect: pre-flight quality check over raw SVO2 recordings — flags
frame-stutter (clean/borderline/bad/broken via a shrink + max-gap
classifier) and black left-eye frames, with a per-episode inspect.json
cache (mtime-keyed) and a task-level inspect_report.json aggregate.
Camera-name agnostic (any cameras/*.svo2 view is scanned).
- convert: per-frame control_source / is_intervention / sample_weight tags
written into each lowdim pickle, resampled onto the camera grid by NEAREST
timestamp (never np.interp, which would smear the 0/1 takeover flag);
--w-intervention upweights human-takeover frames; episode_kind and
intervention_ratio carried raw -> converted; --append numbers new episodes
after the highest existing one and rebuilds split_all.json.
- shardify: --keep {all,interventions,interventions_context} + --keep-context
subsetting on control_source, and episode_kind / is_intervention /
sample_weight propagated into every sample's metadata so a merged shard set
stays splittable for the ABC 80:10:10 base/intervention mix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the data-quality inspection user-guide page and inspector API reference, and wire rd inspect into the README / guide index / nav. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
paarthshah-tri
self-requested a review
July 20, 2026 21:39
Covers the three pure functions this PR introduces, with no ZED SDK, SVO2
files, or robot attached:
- inspector._classify / _result_from_scan: the shrink / max-dt /
frame-count verdict ladder, exclusive threshold boundaries, worst-axis
-wins, degenerate scans, and the black-left-eye override (which beats
an otherwise-clean timing verdict while preserving timing metrics for
diagnosis); _is_episode_dir on a temp tree.
- converter._resample_flag_nearest: control_source must be resampled by
nearest timestamp and never interpolated — one test asserts np.interp
on the same inputs smears the flag into fractions while this does not.
Also ties, out-of-range clamping, 100 Hz -> 30 Hz downsampling, dtype.
- shardify._keep_anchor: all / interventions / interventions_context,
the forward look-ahead window, keep_context=0 still keeping one
lead-in frame, end-of-episode clamping, and the documented footgun
that --keep interventions empties a pure-teleop dataset.
This is the repo's first tests/ directory; there is no test job in CI, so
they run locally with `uv run pytest tests/ -v`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Port YAM's local inference loop and HG-DAgger interactive correction onto TRI raiden. TRI has no local inference path (rd serve is remote-only and cannot host a per-step obs/action/takeover rendezvous), so this adds a new `rd infer` command that runs a model in-process and layers the intervention state machine on top — mirroring how the YAM fork separates the two. New raiden/inference.py: - ModelBridge ABC (load/predict/reset). predict returns the loop's action type: (14,) joint [left(7), right(7)] or (20,) ee_pose (IK'd on-device). - RaidenInferenceLoop(RaidenPolicyServer): autonomy loop at --action-hz plus the --intervene HG-DAgger state machine — a single leader-CAN input thread (TOP button toggles policy<->teleop, BOTTOM ends an episode), shadow tracking of the leaders onto the measured follower pose, seam-free joint-delta takeover with a bumpless gripper clutch, bridge.reset() handback, whole-rollout recording with per-frame control_source, and a load-bearing Ctrl+C teardown (join input thread -> home -> de-energise -> heavy save) so the GIL-heavy save can't starve the motor threads and drop CAN. - run_session() looped collection, reachable via `rd infer --session`. Surgical, behaviour-preserving hooks on the shared base: - server.py: _wants_leaders() gate (default False), _make_obs(timestamp=) kwarg, and a _grab_paused Event honoured in the ZED capture loop so SVO2 toggles don't deadlock grab(). rd serve is unchanged when unused. - robot/controller.py: YAMLeaderRobot.zero_torque_mode (i2rt command path, so a passive leader is genuinely backdrivable), pure joint_delta_takeover + _grip_clutch, and return_to_home_and_disconnect. - recorder.py: DaggerRecorder (borrows the loop's live ZED handles, writes control_source int8 + episode_kind='dagger'/intervention_ratio in the exact layout convert/shardify consume); _next_recording_dir now numbers max+1. - cli.py: `rd infer` (InferCommand + dispatch + help). Arm order is TRI's left-then-right end to end (joint_delta_takeover repacked) so joint and ee_pose modes command the same arms. NOTE: this touches leader CAN, zero-torque, and ZED SVO2 toggling and cannot be exercised without the physical robot — it needs on-robot validation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the recovery-data-collection user-guide page (operator loop, round organization for the 80:10:10 mix, ModelBridge how-to) and the inference API reference, and wire rd infer into the README / guide index / nav. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The predict() contract documented the gripper as (0=open, 1=closed). The
actual convention throughout raiden is the opposite — 0 = closed, 1 = open:
- controller.py: "Button 0 -> close (0.0), Button 1 -> open (1.0)"
- the SpaceMouse path closes with max(0.0, actual - d), opens with
min(1.0, actual + d)
- _GRIPPER_SAFETY_THRESHOLD stops closing when the command is *below*
the actual position
- YAMLeaderRobot.get_info() returns 1 - encoder_position, so squeezing
the trigger lowers the command
- FOLLOWER_HOME_POS homes the gripper at 1.0 (open)
This is the spec a bridge author reads to decide their policy's output,
so an inverted description silently inverts the gripper on hardware.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The leader trigger's physical throw saturates a few percent short of the passive encoder's hardcoded range, so a full squeeze only drives the gripper command down to ~0.04 rather than 0.0. During an HG-DAgger takeover _grip_clutch passes the trigger through absolutely, so the follower never fully closes: it holds a visible gap and drops small parts — precisely the precision tasks interactive correction targets. Snap a trigger within _GRIP_FULL_CLOSE_SNAP of the bottom to a complete close. Applied in _grip_clutch (reached only via joint_delta_takeover from the inference loop) rather than in the shared leader read, so rd teleop and rd record are unaffected. Also corrects the direction of the saturation example in the docstring and states the gripper convention explicitly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Exercises the pure _grip_clutch / joint_delta_takeover functions with no CAN bus, motors, or arms: the clutch holds the policy's grip until the passive trigger crosses it, never moves the gripper in an unintended direction at takeover, latches through a reversal, and reaches a full close via the new snap. joint_delta_takeover is covered for the zero-delta takeover edge and for an un-grabbed arm holding its grip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
geyan21
force-pushed
the
feat/hgdagger-infer
branch
from
July 20, 2026 21:40
455c500 to
8cc4243
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
rd infer, a local inference loop for deploying a trained policy on the robot host, with an optional HG-DAgger interactive-correction mode (--intervene): the policy runs autonomously, the operator grabs a leader arm the moment it starts to fail, corrects, and hands back — and the whole rollout is recorded as one episode with a per-framecontrol_sourceflag (consumed by the convert/shardify plumbing from #6). Follows the ABC recipe: record whole rollouts and mix/upweight corrections at train time, rather than training on interventions alone (seedocs/guide/recovery.md).Why a new command instead of extending
rd serverd serveis a remote policy server (chiral protocol): fire-and-forget streaming, leaders off, hard-estop on disconnect. HG-DAgger needs a per-step rendezvous between observation, action, and takeover state, plus powered leader arms — a different control topology. So the loop lives in a newraiden/inference.pyhosting the model in-process behind a smallModelBridgeABC (load/reset/predict), while reusingRaidenPolicyServer's cameras, depth, proprio, and motor control. The hooks added toserver.pyare default-off, sord servebehavior is unchanged.Design highlights
np.savez_compressed); the reverse order starves the 100–250 Hz motor threads past the DM4310 watchdog and drops CAN mid-motion.grab()+enable_recording()deadlocks the ZED SDK).--max-joint-delta) applies in policy mode only — a deliberate human correction is not a fault.[left(6), l_grip, right(6), r_grip]), matchingrd serveandee_poseIK output.❓ Question for maintainers: leader trigger calibration
On our rig the leader trigger's physical throw saturates a few percent short of the passive encoder's hardcoded range, so a full squeeze only drives the gripper command down to ~0.04 instead of 0.0 — the follower never fully closes, holding a visible gap and dropping small parts. That is invisible in teleop but bites in HG-DAgger, where a takeover exists precisely to fix a failed grasp.
_grip_clutchtherefore snaps a trigger within_GRIP_FULL_CLOSE_SNAP(0.05) of the bottom to a complete close. It is deliberately confined to the takeover path, not the sharedYAMLeaderRobot.get_info()where it would also changerd teleop/rd recordfor every user based on our rig's calibration.If your triggers reach a true full squeeze, this snap is a harmless no-op on the bottom 5% and you may prefer to drop it — happy to remove it or make it a flag. Worth confirming on your hardware either way, since the underlying saturation would affect teleop grasps too.
Usage
rd infer --bridge my_model.bridge:MyBridge --ckpt-path /path/to/ckpt \ --intervene --dagger-task box_fold_dagger_r1 --dagger-instruction "fold the box flap"Full operator guide (takeover loop, round organization for the 80:10:10 mix, writing a
ModelBridge):docs/guide/recovery.md.Status / test plan
tests/test_controller.py(new) covers the takeover control logic hardware-free — 9 tests over_grip_clutchandjoint_delta_takeover: the clutch holds the policy's grip until the passive trigger crosses it, never moves the gripper in an unintended direction at takeover, latches through a reversal, reaches a full close via the snap, andjoint_delta_takeoveris checked for the zero-delta takeover edge and for an un-grabbed arm holding its grip. This is the repo's firsttests/directory; no test job exists in CI yet, so they run locally viauv run pytest tests/ -v.jointandee_posemodesrd convert/rd shardify --keep interventionsruff format --check+ruff checkclean;mkdocs build --strictclean with the docs-CI dependency set.🤖 Generated with Claude Code