Skip to content

feat(control): hybrid-teleop-sim, an xArm7 sim an operator can hold and e-stop - #4788

Merged
mustafab0 merged 2 commits into
mainfrom
mustafa/feat/operator-ui-hold
Oct 31, 2026
Merged

mustafab0 merged 2 commits into
mainfrom
mustafa/feat/operator-ui-hold

Conversation

@mustafab0

@mustafab0 mustafab0 commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

feat(control): hybrid-teleop-sim, an xArm7 sim an operator can hold and e-stop from a Desktop app

Changes

  • OperatorHoldCoordinator (in the blueprint file): operator_estop RPC, two status outputs, hold task status.
  • Blueprint: operator_hold_task() added; MuJoCo cloud on and renamed to scene_pointcloud; distance_threshold=0.05 on ObjectSceneRegistration (objects on the table are 10-17 cm apart); UrdfManager added.
  • The operator hold task and its registry are unchanged from main.
  • all_blueprints.py regenerated; gateway annotation and self-hosted blueprint list entries.

Tests

  • test_hybrid_teleop_sim.py: e-stop order (set_estop, failsafe hold, cancel_trajectory), a failed step does not skip the rest, clear keeps the hold, Bool on both status streams, start sends the e-stop state.
  • dimos/control/test_operator_hold.py and test_operator_hold_task.py unchanged and passing.

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go2 replay realtime (arm64)

Details
Benchmark suite Current: 7c002e0 Previous: d79ab4d Ratio
peak memory 1719.316 MB 1729.973 MB 0.99
peak threads 373 threads 374 threads 1.00
network (transport) 2725.793 MB 2736.274 MB 1.00
disk write 1.707 MB 1.688 MB 1.01
instructions 136.478 G 137.869 G 0.99

This comment was automatically generated by workflow using github-action-benchmark.

@mustafab0 mustafab0 changed the title feat(control): operator hold over streams in a new xarm-operator-sim blueprint feat(control): operator hold over streams Oct 23, 2026
@codecov

codecov Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.07407% with 2 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
.../manipulators/xarm/blueprints/hybrid_teleop_sim.py 95.74% 2 Missing ⚠️
@@            Coverage Diff             @@
##             main    #4788      +/-   ##
==========================================
- Coverage   83.15%   82.92%   -0.23%     
==========================================
  Files        1743     1745       +2     
  Lines      167633   167848     +215     
  Branches    13868    13872       +4     
==========================================
- Hits       139393   139190     -203     
- Misses      24485    24886     +401     
- Partials     3755     3772      +17     
Components Coverage Δ
Tests 96.54% <100.00%> (-0.60%) ⬇️
Flag Coverage Δ
OS-macos-latest 77.31% <99.07%> (+0.03%) ⬆️
OS-ubuntu-24.04-arm 77.09% <99.07%> (+0.02%) ⬆️
OS-ubuntu-latest 77.46% <99.07%> (+0.03%) ⬆️
Py-3.10 77.46% <99.07%> (+0.02%) ⬆️
Py-3.11 77.44% <99.07%> (+0.04%) ⬆️
Py-3.12 77.46% <99.07%> (+0.03%) ⬆️
SelfHosted-Linux 40.35% <32.87%> (-0.23%) ⬇️
SelfHosted-macOS 40.08% <32.87%> (-0.77%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...rol/tasks/operator_hold_task/operator_hold_task.py 98.02% <100.00%> (+2.47%) ⬆️
...asks/operator_hold_task/test_operator_hold_task.py 100.00% <100.00%> (ø)
dimos/control/test_operator_hold.py 99.48% <100.00%> (+0.07%) ⬆️
dimos/robot/all_blueprints.py 100.00% <ø> (ø)
...pulators/xarm/blueprints/test_hybrid_teleop_sim.py 100.00% <100.00%> (ø)
dimos/robot/test_all_blueprints.py 100.00% <ø> (ø)
.../manipulators/xarm/blueprints/hybrid_teleop_sim.py 95.74% <95.74%> (ø)

... and 35 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@greptile-apps

greptile-apps Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] No blocking issues found in the hold-stream implementation; browser use depends on PR #4789.

Summary

Adds hold commands and status streams through OperatorHoldCoordinator, plus the hybrid-teleop-sim blueprint.

  • Operator hold commands and status now travel through coordinator streams.
  • A simulated xArm pick-and-place stack includes operator hold.

No blocking issues found. Full simulator startup and physical hardware operation were not exercised.

Reviews (1) · Last reviewed commit: "feat(control): operator hold over stream..." · Reviewed by Greptile

@mustafab0 mustafab0 added the backport:skip Skip creating a backport to any release branches label Oct 23, 2026
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Oct 23, 2026
…nd e-stop

A new blueprint for a Dimensional Desktop operator page: the
xarm-perception-sim stack plus the operator hold, an e-stop, a point cloud
of the scene and the arm's URDF. The page talks to it through the dimos
gateway only: RPCs and standard LCM streams that Desktop's msgs.js decodes.

- OperatorHoldCoordinator (in the blueprint file) adds an operator_estop(stop)
  RPC. True latches the e-stop, requests the hold (route "failsafe", reason
  "e-stop") and cancels the running trajectory, in that order; a failed step
  is logged and the rest still run. False only clears the latch; the hold
  stays until acknowledged.
- Hold request, acknowledge and status use the existing task_invoke RPC; the
  operator hold task and its registry are unchanged.
- operator_hold_status and operator_estop_status are std_msgs.Bool outputs.
  diagnostic_msgs is not in dimos.msgs, so Desktop cannot decode it; route
  and reason come from task_invoke("operator_hold", "get_status").
- MuJoCo raycast cloud from the wrist camera (80 x 60, 1 cm voxels) on
  scene_pointcloud, kept apart from the object clouds on pointcloud.
- ObjectSceneRegistration distance_threshold 0.05 m: the objects on the table
  are 10 to 17 cm apart and the 0.2 m default merges them.
- UrdfManager serves the URDF and meshes so Desktop draws the arm. Its joint
  tf is off: the planner publishes link_base and link1..link7 on tf at the
  arm's real mount on the 12 cm pedestal, which the URDF's world joint does
  not have.
@mustafab0
mustafab0 force-pushed the mustafa/feat/operator-ui-hold branch from b24ea74 to db63a71 Compare October 30, 2026 19:19
@mustafab0 mustafab0 changed the title feat(control): operator hold over streams feat(control): hybrid-teleop-sim, an xArm7 sim an operator can hold and e-stop Oct 30, 2026
@github-actions github-actions Bot removed the ready-to-merge Required CI checks have passed on this PR label Oct 30, 2026
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Oct 30, 2026
…he coordinator subclass

- OperatorHoldTask gets a set_estop hook, called from the coordinator's
  set_estop. An e-stop turns the hold on (route failsafe, reason e-stop)
  and it stays on until acknowledged. While stopped it commands nothing;
  after the clear it holds each joint at its new reading.
- acknowledge is refused while the e-stop is latched.
- OperatorHoldStatus carries estopped.
- The operator_estop RPC is removed from OperatorHoldCoordinator, which
  now only adds the hold and e-stop on/off streams. The page uses the
  coordinator's existing set_estop.
@github-actions github-actions Bot removed the ready-to-merge Required CI checks have passed on this PR label Oct 30, 2026
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Oct 30, 2026
@TomCC7

TomCC7 commented Oct 31, 2026

Copy link
Copy Markdown
Member

whathappend to this stack lol
image

@mustafab0
mustafab0 enabled auto-merge (squash) October 31, 2026 00:41
auto-merge was automatically disabled October 31, 2026 00:41

Pull request removed from stack

@mustafab0
mustafab0 removed this pull request from stack #4809 October 31, 2026 00:41
@mustafab0

Copy link
Copy Markdown
Contributor Author

@TomCC7 the following 4 PRs were me adding features on websdk using streams, since RPC is not supported on the websdk.

But then moved to using the dimos gateway for teleop. and the remaining 4 are not relevant anymore.

@mustafab0
mustafab0 enabled auto-merge (squash) October 31, 2026 00:42
@mustafab0
mustafab0 merged commit 4953df4 into main Oct 31, 2026
67 of 69 checks passed
@mustafab0
mustafab0 deleted the mustafa/feat/operator-ui-hold branch October 31, 2026 01:25

This branch was previously deployed

1 inactive deployment
cachix — 7c002e0f Deployed Oct 31, 2026 by mustafab0 via cachix-build-macos #15243
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip Skip creating a backport to any release branches ready-to-merge Required CI checks have passed on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants