Skip to content

Add rd infer: local inference loop with HG-DAgger interactive correction - #7

Draft
geyan21 wants to merge 8 commits into
TRI-ML:mainfrom
geyan21:feat/hgdagger-infer
Draft

geyan21 wants to merge 8 commits into
TRI-ML:mainfrom
geyan21:feat/hgdagger-infer

Conversation

@geyan21

@geyan21 geyan21 commented Jul 20, 2026 •

Copy link
Copy Markdown

Stacked on #6 — this branch is based on it, so the diff includes its commits until it merges. The rd infer changes proper are the last two commits.

Summary

Adds rd infer, a local inference loop for deploying a trained policy on the robot host, with an optional HG-DAgger interactive-correction mode (--intervene): the policy runs autonomously, the operator grabs a leader arm the moment it starts to fail, corrects, and hands back — and the whole rollout is recorded as one episode with a per-frame control_source flag (consumed by the convert/shardify plumbing from #6). Follows the ABC recipe: record whole rollouts and mix/upweight corrections at train time, rather than training on interventions alone (see docs/guide/recovery.md).

Why a new command instead of extending rd serve

rd serve is a remote policy server (chiral protocol): fire-and-forget streaming, leaders off, hard-estop on disconnect. HG-DAgger needs a per-step rendezvous between observation, action, and takeover state, plus powered leader arms — a different control topology. So the loop lives in a new raiden/inference.py hosting the model in-process behind a small ModelBridge ABC (load / reset / predict), while reusing RaidenPolicyServer's cameras, depth, proprio, and motor control. The hooks added to server.py are default-off, so rd serve behavior is unchanged.

Design highlights

  • Instant, seam-free takeover — during autonomy the powered leaders shadow the followers' measured pose, so pressing a trigger hands you the arm exactly where it is; the correction is a joint-space delta (leader and follower are kinematically identical YAM arms), giving a zero seam by construction. Handback re-observes and re-predicts from the pose you left.
  • Bumpless gripper clutch — the leader trigger is unmotorized and can't be shadowed; the policy's grip command is held until your trigger crosses it, then tracks absolutely.
  • Safe shutdown ordering — Ctrl+C homes and de-energises the arms before the heavy save (SVO2 flush + np.savez_compressed); the reverse order starves the 100–250 Hz motor threads past the DM4310 watchdog and drops CAN mid-motion.
  • Recording shares the live ZED handles — one camera feeds the policy and records SVO2 simultaneously; recording toggles quiesce the grab loop first (a concurrent grab() + enable_recording() deadlocks the ZED SDK).
  • Single leader-bus owner — all leader reads/writes stay on one input thread; the policy-runaway estop path stops that thread before commanding the leaders.
  • The per-step joint-delta limit (--max-joint-delta) applies in policy mode only — a deliberate human correction is not a fault.
  • Action convention is left-then-right ([left(6), l_grip, right(6), r_grip]), matching rd serve and ee_pose IK output.

❓ Question for maintainers: leader trigger calibration

On our rig the leader trigger's physical throw saturates a few percent short of the passive encoder's hardcoded range, so a full squeeze only drives the gripper command down to ~0.04 instead of 0.0 — the follower never fully closes, holding a visible gap and dropping small parts. That is invisible in teleop but bites in HG-DAgger, where a takeover exists precisely to fix a failed grasp.

_grip_clutch therefore snaps a trigger within _GRIP_FULL_CLOSE_SNAP (0.05) of the bottom to a complete close. It is deliberately confined to the takeover path, not the shared YAMLeaderRobot.get_info() where it would also change rd teleop / rd record for every user based on our rig's calibration.

If your triggers reach a true full squeeze, this snap is a harmless no-op on the bottom 5% and you may prefer to drop it — happy to remove it or make it a flag. Worth confirming on your hardware either way, since the underlying saturation would affect teleop grasps too.

Usage

rd infer --bridge my_model.bridge:MyBridge --ckpt-path /path/to/ckpt \
         --intervene --dagger-task box_fold_dagger_r1 --dagger-instruction "fold the box flap"

Full operator guide (takeover loop, round organization for the 80:10:10 mix, writing a ModelBridge): docs/guide/recovery.md.

Status / test plan

tests/test_controller.py (new) covers the takeover control logic hardware-free — 9 tests over _grip_clutch and joint_delta_takeover: the clutch holds the policy's grip until the passive trigger crosses it, never moves the gripper in an unintended direction at takeover, latches through a reversal, reaches a full close via the snap, and joint_delta_takeover is checked for the zero-delta takeover edge and for an un-grabbed arm holding its grip. This is the repo's first tests/ directory; no test job exists in CI yet, so they run locally via uv run pytest tests/ -v.

⚠️ Draft. The offline logic is reviewed and the data contract with convert/shardify is covered by the tests referenced in #6, but the hardware loop needs on-robot validation before this is merge-ready:

  • takeover / handback seam (no jump) in joint and ee_pose modes
  • leader shadow tracking + gripper clutch feel
  • Ctrl+C ordering: home → de-energise → save, arms never go limp
  • SVO2 start/stop against live grab loops (no deadlock, no dropped frames)
  • runaway-delta estop path with the leader input thread active
  • recorded episode round-trips through rd convert / rd shardify --keep interventions

ruff format --check + ruff check clean; mkdocs build --strict clean with the docs-CI dependency set.

🤖 Generated with Claude Code

geyan21 and others added 2 commits July 20, 2026 13:42
Data-quality inspection and the data-pipeline half of the HG-DAgger
recovery-data feature (the interactive `rd infer --intervene` collection
loop follows in a later commit).

- rd inspect: pre-flight quality check over raw SVO2 recordings — flags
  frame-stutter (clean/borderline/bad/broken via a shrink + max-gap
  classifier) and black left-eye frames, with a per-episode inspect.json
  cache (mtime-keyed) and a task-level inspect_report.json aggregate.
  Camera-name agnostic (any cameras/*.svo2 view is scanned).
- convert: per-frame control_source / is_intervention / sample_weight tags
  written into each lowdim pickle, resampled onto the camera grid by NEAREST
  timestamp (never np.interp, which would smear the 0/1 takeover flag);
  --w-intervention upweights human-takeover frames; episode_kind and
  intervention_ratio carried raw -> converted; --append numbers new episodes
  after the highest existing one and rebuilds split_all.json.
- shardify: --keep {all,interventions,interventions_context} + --keep-context
  subsetting on control_source, and episode_kind / is_intervention /
  sample_weight propagated into every sample's metadata so a merged shard set
  stays splittable for the ABC 80:10:10 base/intervention mix.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the data-quality inspection user-guide page and inspector API
reference, and wire rd inspect into the README / guide index / nav.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@paarthshah-tri
paarthshah-tri self-requested a review July 20, 2026 21:39
geyan21 and others added 6 commits July 20, 2026 14:40
Covers the three pure functions this PR introduces, with no ZED SDK, SVO2
files, or robot attached:

  - inspector._classify / _result_from_scan: the shrink / max-dt /
    frame-count verdict ladder, exclusive threshold boundaries, worst-axis
    -wins, degenerate scans, and the black-left-eye override (which beats
    an otherwise-clean timing verdict while preserving timing metrics for
    diagnosis); _is_episode_dir on a temp tree.
  - converter._resample_flag_nearest: control_source must be resampled by
    nearest timestamp and never interpolated — one test asserts np.interp
    on the same inputs smears the flag into fractions while this does not.
    Also ties, out-of-range clamping, 100 Hz -> 30 Hz downsampling, dtype.
  - shardify._keep_anchor: all / interventions / interventions_context,
    the forward look-ahead window, keep_context=0 still keeping one
    lead-in frame, end-of-episode clamping, and the documented footgun
    that --keep interventions empties a pure-teleop dataset.

This is the repo's first tests/ directory; there is no test job in CI, so
they run locally with `uv run pytest tests/ -v`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Port YAM's local inference loop and HG-DAgger interactive correction onto
TRI raiden. TRI has no local inference path (rd serve is remote-only and
cannot host a per-step obs/action/takeover rendezvous), so this adds a new
`rd infer` command that runs a model in-process and layers the intervention
state machine on top — mirroring how the YAM fork separates the two.

New raiden/inference.py:
- ModelBridge ABC (load/predict/reset). predict returns the loop's action
  type: (14,) joint [left(7), right(7)] or (20,) ee_pose (IK'd on-device).
- RaidenInferenceLoop(RaidenPolicyServer): autonomy loop at --action-hz plus
  the --intervene HG-DAgger state machine — a single leader-CAN input thread
  (TOP button toggles policy<->teleop, BOTTOM ends an episode), shadow
  tracking of the leaders onto the measured follower pose, seam-free
  joint-delta takeover with a bumpless gripper clutch, bridge.reset() handback,
  whole-rollout recording with per-frame control_source, and a load-bearing
  Ctrl+C teardown (join input thread -> home -> de-energise -> heavy save) so
  the GIL-heavy save can't starve the motor threads and drop CAN.
- run_session() looped collection, reachable via `rd infer --session`.

Surgical, behaviour-preserving hooks on the shared base:
- server.py: _wants_leaders() gate (default False), _make_obs(timestamp=)
  kwarg, and a _grab_paused Event honoured in the ZED capture loop so SVO2
  toggles don't deadlock grab(). rd serve is unchanged when unused.
- robot/controller.py: YAMLeaderRobot.zero_torque_mode (i2rt command path,
  so a passive leader is genuinely backdrivable), pure joint_delta_takeover
  + _grip_clutch, and return_to_home_and_disconnect.
- recorder.py: DaggerRecorder (borrows the loop's live ZED handles, writes
  control_source int8 + episode_kind='dagger'/intervention_ratio in the exact
  layout convert/shardify consume); _next_recording_dir now numbers max+1.
- cli.py: `rd infer` (InferCommand + dispatch + help).

Arm order is TRI's left-then-right end to end (joint_delta_takeover repacked)
so joint and ee_pose modes command the same arms.

NOTE: this touches leader CAN, zero-torque, and ZED SVO2 toggling and cannot
be exercised without the physical robot — it needs on-robot validation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the recovery-data-collection user-guide page (operator loop, round
organization for the 80:10:10 mix, ModelBridge how-to) and the inference
API reference, and wire rd infer into the README / guide index / nav.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The predict() contract documented the gripper as (0=open, 1=closed). The
actual convention throughout raiden is the opposite — 0 = closed, 1 = open:

  - controller.py: "Button 0 -> close (0.0), Button 1 -> open (1.0)"
  - the SpaceMouse path closes with max(0.0, actual - d), opens with
    min(1.0, actual + d)
  - _GRIPPER_SAFETY_THRESHOLD stops closing when the command is *below*
    the actual position
  - YAMLeaderRobot.get_info() returns 1 - encoder_position, so squeezing
    the trigger lowers the command
  - FOLLOWER_HOME_POS homes the gripper at 1.0 (open)

This is the spec a bridge author reads to decide their policy's output,
so an inverted description silently inverts the gripper on hardware.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The leader trigger's physical throw saturates a few percent short of the
passive encoder's hardcoded range, so a full squeeze only drives the
gripper command down to ~0.04 rather than 0.0. During an HG-DAgger
takeover _grip_clutch passes the trigger through absolutely, so the
follower never fully closes: it holds a visible gap and drops small
parts — precisely the precision tasks interactive correction targets.

Snap a trigger within _GRIP_FULL_CLOSE_SNAP of the bottom to a complete
close. Applied in _grip_clutch (reached only via joint_delta_takeover
from the inference loop) rather than in the shared leader read, so
rd teleop and rd record are unaffected.

Also corrects the direction of the saturation example in the docstring
and states the gripper convention explicitly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Exercises the pure _grip_clutch / joint_delta_takeover functions with no
CAN bus, motors, or arms: the clutch holds the policy's grip until the
passive trigger crosses it, never moves the gripper in an unintended
direction at takeover, latches through a reversal, and reaches a full
close via the new snap. joint_delta_takeover is covered for the
zero-delta takeover edge and for an un-grabbed arm holding its grip.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@geyan21
geyan21 force-pushed the feat/hgdagger-infer branch from 455c500 to 8cc4243 Compare July 20, 2026 21:40

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant