How to make an AI Chatbot with face on any ESP32S3

Overview

A concise tutorial for building a voice‑controlled AI chatbot with local speech and an animated face on an ESP32‑S3. The system runs Xiaoqi firmware/assistant on the ESP32‑S3 and uses Tech Talkie’s “face engine” to render an interactive face on a 128×64 OLED. Audio I/O is handled via an I2S MEMS microphone and an I2S amplifier + speaker.

Key features

• Wake‑word voice control and local assistant responses.
• Speech output through an I2S amplifier and small speaker.
• 128×64 OLED displays an interactive animated face.
• All runs on an ESP32‑S3 with Xiaoqi firmware and a separate face engine.

Required hardware

• ESP32‑S3 board with:
  • ≥ 4 MB flash
  • ≥ 2 MB PSRAM
  • Example: Waveshare ESP32‑S3‑Zero
• INMP441 MEMS I2S microphone
• MAX98357 (I2S) amplifier + small speaker
• 128×64 OLED (I2C)
• (Optional) Custom PCB — JLCPCB recommended in the video

Wiring / hardware notes

• Microphone (INMP441):
  • Connect SCK, WS, SD to the ESP32’s I2S pins
  • L and R to GND
  • VDD → 3.3V (do NOT use 5V)
• Amplifier (MAX98357):
  • Connect BCLK, LRC, DIN to the ESP32’s I2S pins
  • VIN → 5V
  • Set gain pin to GND (as shown in the example)
• OLED:
  • SDA and SCL to the ESP32’s I2C pins
• General:
  • Check your board’s pinout to choose supported I2S / I2C pins
  • Ensure PSRAM and flash capacity match the board specifications

Warning: Verify voltage connections carefully—especially microphone VDD (must be 3.3V). Also confirm PSRAM/flash settings match your board.

Software environment & repositories

• Tools:
  • Visual Studio Code with the ESP‑IDF extension
• Repositories to download:
  • Xiaoqi GitHub repo (main assistant firmware)
  • Tech Talkie face engine repo (face UI)
• Important: do not use spaces in the extraction path for these repos

Build & configuration (high level)

1. Open the Xiaoqi folder in VS Code.
2. Select board: ESP32‑S3 → built‑in USB JTAG.
3. Open the ESP‑IDF configuration editor (menuconfig) and set:
   • Serial flasher flash size and a matching partition table (e.g., 4 MB)
   • In Xiaoqi assistant section: board type (e.g., bread_compact_wifi), OLED driver, wake word (e.g., “Jarvis”)
   • PSRAM mode to match your board (quad, etc.)
4. Save configuration, do a full clean, then build the project.
5. Edit main/boards/bread_compact_wifi/config.h to set the GPIO pins for mic, amp, and OLED according to your wiring.
6. Flash: use build + flash + monitor; choose UART if prompted.
7. Wi‑Fi setup:
   • Device starts in AP mode (SSID: “Xiaoqi”)
   • Connect with a phone and open http://192.168.4.1 to configure Wi‑Fi
8. Device registration:
   • The device shows a device code
   • Register/login on the Xiaoqi console and add the device using that code
   • Configure language, voice, role, etc.
9. Optional: enable dark mode in IDF config

Adding the face engine (interactive face)

• Copy two files from Tech Talkie’s face engine repo into the firmware display folder.
• Add the face engine source file to the CMakeLists.txt source list.
• Edit display source code:
  • In oled_display.cc: paste the face engine code after includes and remove the original setup_ui_128x64 function.
  • In oled_display.h: apply the single‑line change required by the face engine.
• Rebuild and flash — the original UI will be replaced with the interactive face.

Demo & capabilities shown

• Voice interaction with wake word (assistant identifies as the chosen name, e.g., “Jarvis”).
• Speech output with adjustable volume.
• Responses include date/time, weather, general knowledge, math, and Q&A (examples: set volume to 100%, simple math, Shakespeare questions).
• Interactive OLED face responds on the 128×64 display.

Other notes

• JLCPCB is recommended for PCB manufacturing and assembly (upload Gerbers, select specs, order/track).
• Double‑check PSRAM and flash capacity and configuration in menuconfig.
• Keep extraction paths free of spaces to avoid build issues.

Primary sources & credits

• Video presenter (project author/demonstrator)
• Xiaoqi GitHub repository (firmware / assistant)
• Tech Talkie face engine repository (OLED face UI)
• Waveshare ESP32‑S3‑Zero (example board)
• JLCPCB (PCB manufacturing / assembly service)