02 iPhone · Apple Watch · human authority
Command R.O.B.
with intent.
A native human command, telemetry, and safety console that turns touch, device motion, voice, and Lidar data into authenticated, bounded robot control—then keeps AI proposals and autonomy visibly under operator authority.
01 Manual command deck
ROBController does not send disconnected button events and hope the robot reconstructs operator intent. Five times per second it emits a complete controller snapshot: independent left and right touch demands, a full device-attitude matrix, yaw/pitch/roll, base and mechanism states, speed, direction, playback state, location, and the current speech transcript.
That snapshot becomes stale as a unit. Cerebro can stop using it without guessing which button-up packet went missing.
Two circular pads track separate touches, clamp each touch to its half of the display, normalize each side to approximately ±1, and return both to an explicit neutral sentinel on release. The operator can drive each tread directly instead of collapsing tank motion into a single generic joystick.
Speed spans 0–100% with ±10 steps, a 10% preset, and Max. Dedicated controls cover forward/reverse, play/pause, tread lock, flipper forward/relax-brake/back/lock, and linear-actuator front/gravity/back. A separate master-control request prevents silent competition between controllers.
Core Motion supplies the 3×3 rotation matrix plus yaw, pitch, and roll at the same 5 Hz cadence. Core Location contributes latitude and longitude when authorized. These values travel with the command state so Cerebro can interpret them in the same freshness window.
Reconnect clears in-flight AI proposal state before establishing a new link. Request Control explicitly asks Cerebro for authority. Shut Up sends a direct speech-stop command. The app disables the idle timer so a live operator console does not disappear mid-session.
02 Cerebro control plane
ROBController becomes connected only after encrypted transport, exact certificate pinning, a fresh reciprocal proof, and application-level session acceptance all succeed. Discovery is convenient; it is never trusted as identity.
Bonjour browsing filters paired robot UUID and protocol metadata. A manual host/port v2 path exists for controlled networks, but both routes still require the same certificate pin and reciprocal proof.
Every RCTL v2 frame carries type, bounded payload length, increasing sequence, and message UUID. Malformed headers, repeated or decreasing sequences, and payloads above 4 MiB are rejected.
The pairing profile is stored in iOS Keychain as AfterFirstUnlockThisDeviceOnly—not UserDefaults—and a secure pairing field handles replacement. Cerebro can revoke this controller without affecting another operator device.
Failed links reconnect with jittered exponential backoff from one to thirty seconds. The retired plaintext transport is available only through an explicit engineering flag and never becomes an automatic downgrade.
03 Human-gated AI actions
ROBController is the visible approval and outcome console for high-level Gemini-originated proposals created by Cerebro. It runs no Gemini model itself, and pressing Approve never calls a motor API. Approval records operator intent; completion, failure, cancellation, rejection, and expiry remain distinct states.
look_at, play_gesture, request_pick, navigate_relative, and stop_motion. Exact argument keys are enforced; raw joint arrays and extra model prose are rejected.
Relative travel is capped at ±1 meter, yaw at ±π, and speed scale at 0.35. Requests expire within 120 seconds; detail text is capped at 2,048 characters and the entire proposal at 64 KiB.
A sender-plus-call-ID ledger replays the last known status, absorbs UDP reordering, keeps cancellation tombstones, admits one open proposal, and stays bounded to 128 entries.
04 Bounded Social Roam
Start Social Roam creates a unique autonomy session centered on the activation pose with a five-meter radius, a 0.20 maximum speed scale, an eight-hour expiry, and an allow-list of talk, look-at-person, idle gesture, roam, and stop behavior. Cerebro owns the fresh-Lidar planner and Gemini conversation; ROBController owns authorization and stop.
05 Live RPLidar views
ROBController consumes both polar scan returns and grayscale occupancy-map frames. The live scan view draws the robot at center, plots each distance/angle return, shows the publisher's pose and IMU yaw/pitch/roll, and gives the operator a 1–1000 zoom scale. Map bytes are converted into an 8-bit image and published on the main queue.
06 R.O.B. on your wrist
The SwiftUI Watch companion never stores a Cerebro credential. It sends versioned, bounded commands through the paired iPhone, and the phone relays drive only while its app is active and its Cerebro connection is fully authenticated.
Touch begin requests master authority; a radial mixer turns the one-finger vector into normalized differential treads. Touch end sends braked neutral, then releases control. Leaving the view, lowering the wrist, losing reachability, backgrounding the phone, deactivating WatchConnectivity, or missing the heartbeat triggers the same stop path. Motion is never queued for replay.
07 Voice and language
Touch-down creates an Apple Speech request in record/measurement mode, streams microphone buffers, and shows the best transcription live. Touch-up cancels. The current transcript rides with the controller state rather than becoming a separate unbounded command channel.
More than fifty locale choices span major English, Spanish, French, Chinese, Portuguese, German, Dutch, Danish, Italian, and many single-locale languages. Choosing one sends an explicit output-language command to Cerebro.
08 Internal architecture
The app combines Objective-C++ orchestration and established storyboards with Swift networking, pairing, action/autonomy protocols, Lidar rendering, and Watch relay logic. The Watch target is pure SwiftUI. Each boundary has a small, versioned wire contract instead of sharing UI state across devices.
Implementation status
iPhone dual-touch/manual control; Core Motion and location snapshots; Apple Speech and locale selection; QUIC/TLS pairing; replay-resistant framing; Gemini-proposal approval and status; bounded Social Roam authorization; polar scan and grayscale Lidar maps; SwiftUI Watch dictation and drive relay.
ROBController does not run Gemini, MLX, Core ML inference, a camera feed, or robot autonomy. It forwards operator intent and shows upstream proposals; Cerebro remains the intelligence, planning, and physical-execution boundary.
Dormant AurioTouch waveform/FFT sources and an unimplemented automation-script message remain in the repository and are not product features. The current iPhone + Watch surfaces are the demonstrated path; the separate iPad storyboard still needs runtime repair before it should be presented as ready.
Orbitus Robotics · built on the physical platform
R.O.B.'s applications are designed around real cameras, real networks, real actuators, and the discipline required to put intelligent behavior near a moving machine.
