Franka inference

Dropbear exposes MolmoAct2-DROID inference through dropbear.franka.observe(). You provide camera frames and robot state; Dropbear returns a 15-step absolute action chunk. This guide stops before actuation because the SDK does not ship a Franka safety controller.

Do not send the result below to a robot. A Franka controller must validate joint, velocity, acceleration, workspace, collision, gripper, watchdog, hold, and physical e-stop behavior locally.

1. Install and diagnose

Use the base SDK alongside your existing Franka controller:

uv add "dropbear==0.1.0a2"
uv tool install "dropbear==0.1.0a2"
dropbear login
dropbear doctor
dropbear status --model molmoact2-droid

The status command must show available DROID capacity for the region you intend to use. It does not open a session.

2. Preserve the model ordering

All images must be H×W×3 uint8 RGB arrays. Joint state is seven values in your controller’s Franka order, expressed in radians. The gripper state is normalized from 0 (open) to 1 (closed). If you omit second_exterior_frame, the worker uses the first exterior view for the checkpoint’s second exterior input.

3. Request one chunk

Save this as franka_prediction.py and map the four controller reads to your stack:

import dropbear

from my_franka_stack import controller


exterior_rgb = controller.read_exterior_rgb()
wrist_rgb = controller.read_wrist_rgb()
joint_radians = controller.read_joint_positions()
gripper_position = controller.read_gripper_position()

observation = dropbear.franka.observe(
    exterior_frame=exterior_rgb,
    wrist_frame=wrist_rgb,
    joint_positions=joint_radians,
    gripper=gripper_position,
)

with dropbear.connect(model="molmoact2-droid") as policy:
    result = policy.predict(
        observation,
        instruction="pick up the green block",
    )
    print(f"session={policy.session_id}")
    print(f"region={policy.region}")
    print(f"transport={result.transport_mode}")
    print(f"latency_ms={result.timing.obs_to_action_ms:.1f}")

print(f"shape={len(result.actions)}x{len(result.actions[0])}")
assert len(result.actions) == 15
assert all(len(action) == 8 for action in result.actions)
print("Stopped before actuation.")

Run it:

uv run python franka_prediction.py

Expected result:

shape=15x8
Stopped before actuation.

Each row is:

[joint_0, joint_1, joint_2, joint_3, joint_4, joint_5, joint_6, gripper]

The joint targets are absolute radians; the gripper target is in [0, 1].

4. Build the missing safety layer

Before any motion experiment, your local controller needs:

an explicit mapping between model and controller joint order;
hard position, velocity, acceleration, and jerk limits;
workspace and self/environment collision constraints;
per-action finite-value and timestamp checks;
a disconnect watchdog that commands a tested hold or stop;
a physical e-stop independent of the Dropbear process;
a low-speed, reduced-workspace validation plan with an operator present.

Only after those controls are reviewed and tested should you connect the RobotIO.apply_action() boundary from the Python quickstart. Dropbear’s supported public surface ends at the returned action chunk. Need help validating the contract? Email team@dreamscalelabs.com.

Getting started

Build with Python

Robot guides

Other paths

Reference

Help

1. Install and diagnose

2. Preserve the model ordering

3. Request one chunk

4. Build the missing safety layer

​1. Install and diagnose

​2. Preserve the model ordering

​3. Request one chunk

​4. Build the missing safety layer

1. Install and diagnose

2. Preserve the model ordering

3. Request one chunk

4. Build the missing safety layer