Skip to content

integration(learning): add MakerMods Metal collection and Dreamlabs setups - #4900

Draft
TomCC7 wants to merge 16 commits into
cc/feat/dreamlabs-dual-yamfrom
cc/feat/metal-learning-integration
Draft

TomCC7 wants to merge 16 commits into
cc/feat/dreamlabs-dual-yamfrom
cc/feat/metal-learning-integration

Conversation

@TomCC7

@TomCC7 TomCC7 commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

MakerMods Metal stations can collect demonstrations and run Dreamlabs EEF policies through the shared learning controls, using one robot model for teleop, planning, policy FK, and execution. This integration includes the Metal implementation and the Dreamlabs backend from #4653; no additional merge of #4865/#4881 is required.

Collection and rollout blueprints cover single Metal with overhead, dual Metal with two wrists, and dual Metal with overhead plus wrists. Live testing confirmed the public dreamlabs/dream-1-beta checkpoint requires all three camera roles; the smaller layouts remain available for collection and compatible custom checkpoints. Metal accepts EEF output only, and Dreamlabs joint output remains restricted to canonical dual YAM. Preview is the default, dispatch is explicit, and Quest grip takeover stops policy without automatic resume.

The shared model now distinguishes the execution grasp_frame (80 mm along gripper_base +Z) from the Dreamlabs protocol dreamlabs_tcp (135 mm along vendor link6 +X, RPY pi/2, 0, pi/2). The protocol frame matches the actual deployed MakerMods playground policy worker. Using grasp_frame directly previously introduced a fixed 20.8 mm and 90-degree discrepancy. Observations use dreamlabs_tcp in each arm's own base frame, and returned protocol poses are transformed back to grasp_frame. Both links are defined in Python, with side prefixes for dual Metal; no separate TCP JSON is required for the standard model. Named or explicit legacy tool-transform overrides remain supported.

Physical Metal arms enable the shared adapter's URDF-based gravity compensation. This stack also includes EEF CBOR RPC serialization, coordinator task-call compatibility fixes, a bounded wait for transient observation skew, the SDK's 15-second cloud request timeout, and detailed trajectory fault reasons. Metal uses a 5 cm final position tolerance. Optional station debugging settings can disable endpoint convergence or observation timing gates; their defaults remain enabled, and hardware limits, finite-value checks, tracking bounds, and deadman supervision remain active.

Validation of the protocol-frame correction: 72 focused regressions and 8 cached-URDF/Pink/mock tests passed, plus scoped mypy and pre-commit. Golden FK vectors were generated offline from the actual deployed playground policy-worker-DxdwLh8C.js, with its URL and SHA256 recorded in the fixture. Tests cover outgoing state and incoming action conversion, single/dual arms, nontrivial base mounts, and existing override compatibility. The deployed worker's FK and the corrected model agree within 1e-7 m in these checks. No new inference or motor commands were issued to validate this correction.

Earlier station testing reached live three-camera inference and hardware EEF execution, and exposed tracking and convergence faults. Sustained task success, physical TCP calibration, measured dual mounting geometry, and real execution after the protocol-frame correction remain unverified. Default dual mounts are a mock example, not measured installation geometry.

This remains an integration draft based on #4653. Station guide: docs/capabilities/manipulation/learning/guides/metal-dreamlabs-setups.md.

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go2 replay realtime (arm64)

Details
Benchmark suite Current: 8d960d6 Previous: 6c5624a Ratio
peak memory 1752.547 MB 1779.605 MB 0.98
peak threads 365 threads 374 threads 0.98
network (transport) 2726.236 MB 2736.27 MB 1.00
disk write 14.324 MB 1.496 MB 9.57
instructions 137.723 G 138.686 G 0.99

This comment was automatically generated by workflow using github-action-benchmark.

@TomCC7
TomCC7 force-pushed the cc/feat/metal-learning-integration branch from 2e1fbc6 to 0322960 Compare October 30, 2026 00:44
@TomCC7
TomCC7 force-pushed the cc/feat/metal-learning-integration branch from 0322960 to f0400fc Compare October 30, 2026 17:26
@TomCC7
TomCC7 force-pushed the cc/feat/metal-learning-integration branch from f0400fc to f45a957 Compare October 30, 2026 23:55
@TomCC7
TomCC7 force-pushed the cc/feat/metal-learning-integration branch from f45a957 to 4b646db Compare October 31, 2026 00:32
@codecov

codecov Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
7653 1 7652 57
View the top 1 failed test(s) by shortest run time
dimos.codebase_checks.test_no_underscore_assign::test_no_underscore_assignment
Stack Traces | 8.09s run time
def test_no_underscore_assignment():
        """Fail if any file assigns to a bare `_`."""
        dimos_dir = DIMOS_PROJECT_ROOT / "dimos"
        hits = find_underscore_assignments()
        if hits:
            listing = "\n".join(f"  - {p.relative_to(dimos_dir)}:{lineno}" for p, lineno in hits)
>           raise AssertionError(
                f"Found assignment(s) to `_` in dimos/:\n{listing}\n\n"
                "Assigning to `_` is not allowed: it hides an unused variable instead "
                "of removing it. Delete the variable. If you only need the "
                "expression's side effect, evaluate it directly with a call "
                "(`obj.method()`, `getattr(obj, 'attr')`) or log it; a bare attribute "
                "access needs `# noqa: B018`. Tuple unpacking (`a, _ = f()`) is fine "
                "and not flagged by this rule."
            )
E           AssertionError: Found assignment(s) to `_` in dimos/:
E             - control/test_task_context.py:70
E             - control/test_task_context.py:113
E             - control/test_task_context.py:128
E           
E           Assigning to `_` is not allowed: it hides an unused variable instead of removing it. Delete the variable. If you only need the expression's side effect, evaluate it directly with a call (`obj.method()`, `getattr(obj, 'attr')`) or log it; a bare attribute access needs `# noqa: B018`. Tuple unpacking (`a, _ = f()`) is fine and not flagged by this rule.

dimos_dir  = PosixPath('.../dimos/dimos/dimos')
hits       = [(PosixPath('.../dimos/dimos/dimos/control/test_task_context.py'), 70), (PosixPath('.../runner/work/...ol/test_task_context.py'), 113), (PosixPath('.../dimos/dimos/dimos/control/test_task_context.py'), 128)]
listing    = '  - control/test_task_context.py:70\n  - control/test_task_context.py:113\n  - control/test_task_context.py:128'

dimos/codebase_checks/test_no_underscore_assign.py:55: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

TomCC7 added a commit that referenced this pull request Nov 2, 2026
Build directly on main with standard connection ports, dedicated control blueprints and the minimal Metal adapter/model configuration. Keep learning and policy changes out of this branch.

Reuse model-backed gripper support from #4939 (27d3753) and the Metal driver/configuration from #4900. Lock only the required can-motor-control update to 0.0.11.
@TomCC7
TomCC7 added this pull request to stack #4898 November 3, 2026 04:49
@TomCC7
TomCC7 force-pushed the cc/feat/metal-learning-integration branch from 9d4ae5e to fdb595c Compare November 3, 2026 08:03
@TomCC7
TomCC7 force-pushed the cc/feat/metal-learning-integration branch from fdb595c to 203dd34 Compare November 3, 2026 22:39

This branch was previously deployed

1 inactive deployment
cachix — 8d960d60 Deployed Nov 4, 2026 by TomCC7 via cachix-build #15705
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant