Arc Wall
2026 — present · in progressI want to turn my room into a JARVIS-inspired workshop: a projected canvas I can draw on with my hands, a desk for measuring and designing things, and an AI assistant that can see what I'm working on. A few small monitors would show the cameras, system health, and live telemetry.
It's a long-term project I'm building a piece at a time. The first goal is a workspace I actually enjoy using; the rest can grow around it.
what's built so far
The current software has a drawing canvas, single-camera hand controls, four-corner projection mapping, a separate projector window, and browser-local saves. There's also an initial canvas-aware assistant with reviewable scene edits. The physical setup still needs testing; multi-camera tracking, CAD, connected telemetry, and robotics are planned.
where I want to take it
- 01
A usable projected workspace
Get the drawing and gesture controls working reliably on my wall, with a control desk on the laptop and real system metrics on a second screen.
- 02
Multi-camera 3D tracking
Calibrate two cameras first, then add a third. Track a fingertip in a shared coordinate system and use pinch, grab, and rotate gestures to manipulate a 3D model. Measure latency and tracking loss along the way.
- 03
A measurement-to-CAD workbench
Use an overhead camera and a calibrated desk grid to measure flat objects. Turn a confirmed outline into an editable CadQuery model, inspect it in the workspace, and export it for 3D printing. Start with a simple bracket.
- 04
An assistant that understands the workspace
Connect voice, canvas snapshots, and saved designs through Hermes and a small set of MCP tools. Ask questions about a sketch, retrieve an earlier design, or propose annotations and model changes for review.
- 05
Live telemetry and diagnostics
Collect camera FPS, tracking confidence, application latency, and host health. Store the history in TimescaleDB, show it in Grafana and the interface, then add threshold alerts and incident investigation. Start collecting real metrics in the first phase.
- 06
Physical experiments, later
Connect an ESP32, sensors, and a small motor or gimbal. Eventually design and print a physical companion, with independent motor controls and telemetry. This comes after the workspace is useful on its own.
how I'm thinking about it
Keep tracking and rendering responsive on their own, with the assistant handling higher-level requests. Use the homelab for always-on services and history, and a separate computer for heavier vision work when needed. Build each part so it's useful before connecting everything.