isaac/CLAUDE.md

# Context for Claude

Working notes for this repo: how the cell is put together, where the rest of the assets
live, and the traps that are not visible from the code. Read this before changing anything
in `robozon_sorter/sim/` — several constants here are load-bearing and look arbitrary.

---

## 1. What this is

A conveyor sorting cell in Isaac Sim 6.0 (Kit 110). Goods ride a belt, three stereo pairs
measure each one under a camera portal, and a pusher diverts the round ones onto a branch
belt and into a bin.

```
infeed belt ──▶ camera portal ──▶ laser gate ──▶ pusher ──▶ Belt_01 ──▶ bin
   x=+2.3          x=-0.75          x=-3.74      x=-3.90              y=+1.3…2.0
                (CRE-ROI v2b)                  (class D only)
```

Goods travel in **−X** at 1 m/s. Belt surface is at **z = 1.781** everywhere on the main
run and the branch.

**Classes:** B = sortable, C = oversize (any dim >450×320×320 mm or <10 mm), D = round
(`K = r_in/R_out > 0.8`). Only D is diverted.

---

## 2. Where things are

### This repo
`scene/sorter.usd` is the real authored cell, not a procedural rebuild. It was exported
from `90_degree.usd` with two changes: conveyor references re-pointed to
`../assets/conveyors/`, and the 48-object library dropped in favour of six bundled test
meshes with baked geometry.

Do not "simplify" the scene by regenerating it from code. That was tried and rejected —
the result looked nothing like the real cell.

`scene/plow_cell.usd` is the same authored build carried over whole, but as the **bare
mechanics**: conveyors, the Y-split pusher and the plow, with no camera portal, no laser
gate and no item library. Cameras, speed scenarios and laser sensors get added on top of
it. Rebuild it with `scripts/build_plow_cell.py`, which re-points the S3 conveyor
references at `assets/conveyors/`, points the plow at `assets/plow/`, and strips the baked
`targetPosition.timeSamples` track off the pusher drive — time samples outrank the
attribute default, so while they are present nothing can *control* that drive.

The plow meshes matter: `assets/meshes/plow_*.usd` are 8-point placeholder boxes that
`sorter.usd` still uses, while `assets/plow/plow_*.usd` is the real geometry (72k / 23k
points). `scripts/smoke_plow_cell.py` asserts the point count precisely because a stub
composes without error and just looks wrong.

### Remote machine (where Isaac Sim runs)

```
host      dasha@46.39.224.77
isaac     /home/whatevenif/isaacsim          (python.sh lives here)
project   /home/dasha/robozon-sorter          (deployed copy, ~308 MB, runnable)
assets    /home/dasha/isaac_assets            (the original working tree)
```

Connect with an SSH ControlMaster (plain repeated ssh exhausts local ephemeral ports on
long sessions) and tunnel the Kit python server:

```bash
ssh -M -S ~/.ssh/cm/dasha -o ControlPersist=900 -fN dasha@46.39.224.77
ssh -fN -L 8226:127.0.0.1:8226 dasha@46.39.224.77       # isaacsim.code_editor.python_server
```

Then send code with the `isaac-sim-remote` skill's `isaacsim_send.py`. Named contexts keep
state between calls, which is how the long experiments were run.

### More meshes, if six are not enough

| What | Where | Notes |
|---|---|---|
| 31 classified objects | `/home/dasha/isaac_assets/categories.json` | 13 C, 11 D, 7 B; has `zone`, `obb_extents_m`, `k_round`, `label_ru` |
| the objects themselves | `/home/dasha/isaac_assets/robozon_conveyor_scaled.usd` under `/World/CVObjects` | 48 prims, 31 of them classified |
| raw source geometry | `/home/dasha/.objaverse/hf-objaverse-v1/glbs/` | 19 `.glb`, 123 MB — what CVObjects reference |
| already-exported | `/home/dasha/isaac_assets/export_meshes/` | flattened USDs, the six in this repo came from here |
| conveyor art | `/home/dasha/isaac_assets/Props/Conveyors/` | 283 MB, mirrors the Omniverse S3 tree |

To add a test mesh, flatten it out of the source scene so it carries no `.glb` reference:

```python
ns = Usd.Stage.CreateInMemory()
root = UsdGeom.Xform.Define(ns, "/Item")
root.GetPrim().GetReferences().AddReference(
    "/home/dasha/isaac_assets/robozon_conveyor_scaled.usd", f"/World/CVObjects/{name}")
ns.SetDefaultPrim(root.GetPrim())
ns.Flatten().Export(f"assets/meshes/{name}.usd")
```

Then add an entry to `assets/meshes/manifest.json` with `zone` and `gt_dims_mm`. Objects in
that scene sit at **0.49× real size** (1/3-scale meshes × 1.4706), which is why
`config.DIM_SCALE = 1/2.041` converts metres to real millimetres.

### Scene backups
`/home/dasha/isaac_assets/backups/` — `robozon_prescale_2252.usd` is the clean pre-scale
original, useful if the working scene ever gets damaged.

---

## 3. Mechanics

`sim/scene.py` opens the USD and re-applies the runtime settings USD does not carry.
`sim/mechanics.py` is the per-item behaviour. `sim/spawner.py` hooks a PhysX step callback
so the cell runs itself from the Play button.

### Belts
Driven by `PhysxSurfaceVelocityAPI` on kinematic slabs, not by the authored
`ConveyorBeltGraph` nodes — those carry no velocity and only fight the explicit setting, so
they are switched off on load.

**`surfaceVelocity` is expressed in the body's LOCAL frame.** `Belt_01`
(`/World/ConveyorTrack_03/Belt_01`) is rotated: its local X points along world −Y. Carrying
goods toward the bin therefore needs `(−speed, 0, 0)`. Setting the intuitive `(0, +speed, 0)`
drags them sideways and they sit there looking stuck. Resolve the local axes with
`XformCache.GetLocalToWorldTransform(prim).TransformDir(...)` before setting it.

### Pusher
A kinematic blade moved directly from script. It is **not** joint-driven: the authored
`PusherSlide` prismatic drive is unusable because USD drive-target writes reach PhysX about
a second late, so the blade never finishes its stroke while the item is still in reach.
The joint is disabled and `DiverterAnimGraph` is switched off (it rewrites diverter targets
every tick).

Blade collisions are filtered against every belt so it sweeps through the side rails
instead of grinding on them.

**Speed and physics rate are a matched pair: 2.5 m/s at 120 Hz.** Do not change one alone.

* The blade covers 0.70 m of belt; the beam trips with the item at x = −3.61, so the whole
  cycle must fit in 0.64 m of travel = 0.64 s at 1 m/s.
* Stroke is 0.72 m → extension alone needs >1.12 m/s, extend+retract >2.25 m/s.
* Measured: 1.0 / 1.5 / 2.0 / 2.5 m/s all deliver to the bin. **3.0 m/s throws the item**
  (ends up at y≈122, z≈−2070) — the kinematic blade injects too much impulse.
* Raising the solver rate to 480 Hz to "smooth" the impulse also breaks the landing. It was
  tried; the item stopped reaching the bin. 120 Hz is the validated setting.


### Plow sorting station (`plow_cell.usd`)

Two-way sort at the plow, verified by measurement. Rebuild the geometry with
`scripts/narrow_plow.py` then `scripts/place_plow_lanes.py`.

**Arm width is 600 mm**, not the authored 730. The wider arm overhung both rails and
clipped goods it should have passed. `narrow_plow.py` does not guess which local axis
carries the length - it tries each and measures; the answer is **local Y, scale 0.8219**.
Hinge, drive and limits untouched.

**Lane geometry, all verified against the belt (main belt x -7.00..-6.00, y +-0.45, top
z 1.7805):**

| | span | contact |
|---|---|---|
| lane B, perpendicular, travel -Y | x -7.03..-6.58, y -2.45..-0.45 | height 0.0 mm, Y 0.0 mm |
| lane C, 45 deg, travel (-X,+Y) | x -8.12..-6.39, y +0.45..+2.18 | height 0.0 mm, Y 0.1 mm |

Three placement mistakes were made and corrected, all of which looked fine in the tree and
wrong in the viewport:

1. **Height.** These tracks carry an authored `-0.1` z offset, putting their belts at
   z 1.681 - 100 mm below the run, so they read as separate furniture. Use `LANE_DROP = 0`.
2. **X position.** Lane B was first placed at x -7.48..-7.03, entirely *past* where the
   main belt stops (-7.00). A plow sweeps goods sideways while they are still on the belt,
   so a lane must run **alongside** it inside the arm's span (x -7.12..-6.52), never beyond
   the end. Same for C: at x -6.90 its near corner sat behind the plow and +Y deflections
   had nothing to land on; -6.55 puts the corner at the arm tip.
3. **A 45 deg lane does not meet a straight edge at its centreline.** Its near corner runs
   ahead by 159 mm (measured), so the lane needs that much offset or it cuts into the belt.

**An angled lane needs a transition DECK, not a corner patch.** Offsetting by 159 mm makes
its near corner touch, but only that one corner - the other stands off by
width/sqrt2 = 318 mm. Patching just that triangle is not enough either: the plow can put a
item anywhere across the discharge width, so anything pushed wide still drops through the
second gap. `place_plow_lanes.add_transition()` therefore decks the whole corner - the
convex span of the belt edge across the junction (x -7.00..-6.00) and both end-face
corners, giving y 0.45..0.768 - coplanar with both belt surfaces. A square lane like B
meets flush along its whole face, so the builder returns nothing rather than emitting a
zero-area collider.

**Every lane also needs a corner deck**, flush or not. The belt is wider than the lane -
the run reaches x=-6.00 while lane B stops at -6.58 - so the right angle between them is
open air, and anything the plow pushes sideways in that leftover span drops through.
`add_corner_deck()` fills it with a triangular fillet (B: legs 575 mm, corners
(-6.575,-0.45), (-6.00,-0.45), (-6.575,-1.025); C: legs 391 mm), turning the right angle
into a chute. Deck plus fillet together leave no open surface across the discharge.

Lanes start at y = +-0.45, which clears the arm's swept envelope: a 600 mm arm at +-35 deg
reaches 0.60*sin(35) = **0.344 m** either side.

**The conveyor shell collides.** `SM_ConveyorBelt_*_02` has `collision=True`, and that
includes the blue side rails - they physically block goods from leaving the belt, which is
exactly what "nothing reaches the bins" looks like. `plow_sort.open_junction()` clears the
collider on the three shells at the junction, the way a real plow station has its rails cut
away. Each `Belt` keeps its own collider, so nothing falls through.

**Plow delivery is geometrically impossible as currently placed - measured.** The arm is
600 mm and pivots at the belt centre (-7.05, 0), so its lateral reach is
0.60*sin(35) = **0.344 m**. Both lanes start at **y = +-0.45**. That leaves a **106 mm dead
band**: the blade can push an item to 0.344 and no further, the lane begins at 0.45, and the
item runs off the belt end and stops at x ~= -7.0 with v = 0, or falls. Every trace in
`runs/gt_run5.json` shows exactly that - barrel drifts from y=-0.09 to y=+0.148 and halts.

The arm sits at z 1.810..1.890 while the lane surfaces are at 1.7805, i.e. **29 mm above
them**, so it would sweep *over* a lane edge rather than into it. The earlier reasoning that
lanes had to start beyond the swept envelope was therefore unnecessary, and it is what
opened the dead band. Closing it means one of: lanes in to ~y=0.33, a longer arm, or a
larger swing (the joint limit is +-35 authored).

**The angle does not track the command.** Measured in a demo run: an item commanded 0.0 saw
the arm at **+30.5**, one commanded +30.0 saw **-34.6**. Both then went to the same lane. So
on top of the reach gap the control sign and the settling are wrong. Two things are known
about this path and both matter:

* `Plow.target(deg)` writes the whole angle at once; calling it *and* `step_toward()` leaves
  the ramp nothing to do and the blade snaps. Use one or the other, and `plow_sort` uses the
  ramp.
* `Plow.home()` does not settle the arm - it sets a target, and the compliant drive needs
  time. Consecutive trials therefore start from wherever the previous one left the blade
  (measured: -26.4 and -14.4 at the start of runs meant to begin at 0).

* A **positive command deflects to -Y (lane B)**, opposite to the natural reading. Measured,
  not assumed, exactly as the module docstring warns.

Until reach and tracking are fixed the plow does not deliver: `scripts/run_demo.py` reports
the classification honestly and a delivery rate near zero.

### Retract interlocks
The blade returns only when (a) the pushed item has cleared to y > 0.5 and (b) no other item
is inside the blade's footprint. Retracting blindly sweeps the blade back through the next
item and knocks it over — that was a real observed failure, not a hypothetical.

### Plow (`plow_cell.usd` only)
The second diverter, at x = −7.05. Mechanically the opposite of the pusher: a **dynamic
arm on a revolute joint with an angular force drive** (axis Z, limits ±35°, stiffness
120000, damping 1500, 12 kg with gravity disabled). Being force-driven it is compliant —
it yields on contact instead of teleporting through cargo — so `sim/plow.py` commands a
drive *target* rather than writing a transform.

Two consequences: a target is a request, not a position (read `Plow.angle`, which measures
the arm's simulated pose, never assume it arrived), and the rate is not free. The authored
graph swings 30° in **7 ms** (72 rad/s), which is a display animation, not a sortable
motion — at that rate the blade lands as an impulse, the same failure the pusher shows
above 2.5 m/s. `Plow.step_toward` ramps the target at `config.PLOW_RATE` (180 °/s) instead;
`config.PLOW_RATE_AUTHORED` keeps the original figure for reference.

The scene keeps its authored `DiverterAnimGraph`, so pressing Play alone demonstrates the
cell. `plow_cell.prepare(..., script_control=True)` switches that graph off — it has to go,
or it rewrites the drive target every tick and overwrites anything Python commands.

There is also a stray second `ConveyorTrack_01` at stage root, outside `/World`, left over
from how the cell was assembled; it composes as a duplicate belt in the same place and is
deactivated on load rather than deleted, so the file stays as authored.

### Laser gate
A genuine `raycast_closest` across the belt, not a coordinate test. The beam starts at
y = −0.24, which is deliberately **clear of the blade's retracted footprint** (the blade
spans y −0.33…−0.27). Start it any further out and the ray simply reads the blade and the
gate never sees cargo.

---

## 4. Vision (CRE-ROI v2b)

`cv/pipeline.py`. Per item, once, while it sits under the portal: FastSAM segment-everything
→ keep the blob covering the projected inspection point in every view → ROI crop to a fixed
320 px side → **one batched CRE-Stereo pass over all three crops** → fuse, dropping views
whose 3D centroid disagrees with the median by >10 cm → belt-plane OBB + `r_in/R_out`.

The crop must use the **identical column window in both eyes**, left-padded by the maximum
disparity, or the right-hand counterpart falls outside the crop. Disparity is invariant to
an equal column shift, so depth stays correct.

**The stereo rig must be rectified.** Both eyes of a pair share one orientation, with the
right eye offset along its X. Aiming each eye separately at the target verges the pair and
breaks `depth = fx·b/disp` — reconstruction came back at 2–5 m instead of 0.63 m. If
`config/calib.json` ever loses its `"rectified": true` flag, the pipeline refuses to load
rather than silently producing garbage.

Measured: dimensions land within ~34 mm of ground truth on the largest edge; the batched CRE
pass costs ~170 ms per item.

---

## 5. Traps that cost real time

Each of these presented as a silent failure, not an error.

- **Copied items arrive kinematic and hidden.** Kinematic bodies ignore gravity and belt
  friction; hidden ones are invisible to the cameras while still simulating. Both must be
  cleared, plus `sleepThreshold = 0` or a settled item is never woken by the belt.
- **`BBoxCache` / `XformCache` return the AUTHORED transform during simulation.** A moving
  item looks frozen. Use `RigidPrim.get_world_poses()`.
- **`timeline.stop()` resets items to their authored poses**, so captures of a finished run
  must be taken while still playing.
- **`rep.orchestrator.step_async()` stops the timeline.** Any live loop that classifies must
  call `timeline.play()` again afterwards or the whole line freezes mid-run.
- **The viewport's active camera may not be Persp.** Setting Persp's pose then does nothing;
  call `vp.set_active_camera("/OmniverseKit_Persp")` first.
- **Camera bodies carry cosmetic `AimRay` cones** that sit right over the inspection point
  and dominate the frame. Hidden on load.
- **Stale composition:** if a stage was opened while a referenced asset was missing, USD
  caches the failure. Dropping the file in later does not fix it — the prim stays valid with
  `localErrors: none` but empty typeName and bbox. Diagnose with `GetPrimStack()`, fix by
  re-opening the stage.
- **`Stage.TraverseAll()` crashes Isaac on this scene.** Use targeted `GetPrimAtPath`, or run
  read-only checks headlessly via `/home/whatevenif/isaacsim/python.sh` (has pxr, no Kit).
- **Cube colliders:** use `size = 2.0` so the scale op equals the half-extent. Any other
  arrangement makes PhysX use the wrong bounds and goods drop through the belt.

---

## 6. Known limitations

- **Classification is weak.** Metrology is sound but the roundness metric does not separate
  classes at this scale: genuinely round items read K ≈ 0.75 against a 0.80 threshold while a
  plain box reads 0.76. `config.ROUND_K` needs recalibration before the vision output should
  drive the pusher for real. Use `--no-vision` to exercise mechanics on ground truth.
- **Consecutive class-D items are missed at the default 700 mm pitch.** Headway is 0.70 s and
  one pusher cycle is 0.62 s; the gate is blind while a cycle runs, so a D item immediately
  behind another D crosses the beam unseen. Verified: `D,C,D,B` sorts both D correctly, `D,D`
  back-to-back loses the second. Fixing it needs a larger pitch, a slower belt or a second
  diverter — not a faster blade, which is already at its stability limit.
- **The plow (`DiverterEnd`) is untouched in `sorter.usd`.** There its geometry was
  restored and its arm returned to rest, but its control logic is left exactly as authored.
  It is `scene/plow_cell.usd` that makes the plow controllable — see below.