In VLM mode, Gemini reads energy, mood, presence, urgency, and gestures, then selects a motion behavior. In tracking mode, MediaPipe maps a person's hand directly to the arm's position and pitch.

The core ideas are semantic perception, embodied interaction, organic motion, and safe real-time control. Both modes share the same Cartesian motion pipeline, filtering, workspace limits, and recovery logic.