In Progress

Desktop Companion Robot

Personal Project

Project Details

Building an expressive animatronic companion by designing custom mechanical gear assemblies in SolidWorks and integrating ESP32/RP2040 microcontrollers for face-tracking and audio processing. With the goal to bring a character to through integrating an AI chatbot and related character animations. 

❋ Mechanical Design

Planned and modelled 9-DOF gear driven internal mechanical structures, keeping spatial constraints to within a 12×12 cm footprint.

❋ Embedded Firmware

Programmed custom servo classes in MicroPython to translate real-time facial tracking and AI logic into physical animations

❋ Hardware Assembly

soldered and wired 9 servos, camera, and microphone directly to custom pcb

❋ Edge AI

Deployed open source Ai agents and vision modules on microcontrollers to drive responsible physical behaviours.

Design Process

Concept sketches

Initial concept sketch and design to see how things could lay out

Mapping out more size constraints and key points of the internal mechanics.

Gear Design

Base gear design used to make sure all the gears mesh

Mechanical design

Started with layout of all gear paths (to fit within the concept sketch constraints)

  1. Began with the base

  2. All the gears + servo motor holders

  3. Mechanical housing components

  4. Then finally outer shell (in progress)

mechanical structure of the head ^

mechanical structure of the body

Final Internal Mechanical Design!

Hardware

Custom PCB with RP2040 & ESP32

Using the RP2040 to get coordinates from the Grove AI module to convert to servo controls for face tracking.

ESP32 with open source AI agent from Xiaozhi, for voice and camera recognition.

The PCB design was sourced from an open-source project and adjusted slightly to fit my needs.

so robot can hear you

MG90S Servos

Microphone + 6 pin wire

for the robot to move

For the face-tracking

Grove AI Vision Module

Speaker

so robot can make noise

System Architecture Diagram

Prototyping Process

Back