Voice-Interactive Chatbot on Jetson (Multimodal)

Deploy a fully local voice chatbot that listens, thinks, and talks back. Combines NVIDIA Riva for speech recognition and synthesis with Ollama for local LLM inference. Everything runs in Docker containers on Jetson — no cloud required.

Execution model

Run one phase at a time. After each phase:

• Relay all output to the user.
• If output contains [STOP] → stop, consult the failure decision tree.
• If output ends with [OK] → tell the user "Phase N complete" and proceed.

Prerequisites

Phase 1 — Install Jetson Containers and Ollama (~5 min)

git clone https://github.com/dusty-nv/jetson-containers
bash jetson-containers/install.sh

Run Ollama and pull a model:

jetson-containers run --name ollama $(autotag ollama)

Inside the container:

ollama run llama3.2:1b

Type /bye to exit after confirming the model loads.

Verify from host:

curl http://localhost:11434/api/tags

[OK] when curl returns JSON listing the model. [STOP] if Ollama container fails to start — check Docker and NVIDIA runtime.

Phase 2 — Install and configure NGC CLI (~3 min)

mkdir -p ~/ngc_setup && cd ~/ngc_setup
wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/ngc-apps/ngc_cli/versions/3.36.0/files/ngccli_arm64.zip
unzip ngccli_arm64.zip
chmod u+x ngc-cli/ngc
echo "export PATH=\"\$PATH:$(pwd)/ngc-cli\"" >> ~/.bash_profile
source ~/.bash_profile
ngc config set

Enter your NGC API key when prompted.

[OK] when ngc config current shows your org/team. [STOP] if API key is rejected — regenerate at catalog.ngc.nvidia.com.

Phase 3 — Install NVIDIA Riva (~15–30 min)

mkdir -p ~/riva_setup && cd ~/riva_setup
ngc registry resource download-version nvidia/riva/riva_quickstart_arm64:2.16.0
cd riva_quickstart_arm64_v2.16.0

Edit config.sh to disable unused services:

sed -i 's/service_enabled_nlp=.*/service_enabled_nlp=false/' config.sh
sed -i 's/service_enabled_nmt=.*/service_enabled_nmt=false/' config.sh

Configure Docker NVIDIA runtime (if not already set):

sudo tee /etc/docker/daemon.json > /dev/null << 'EOF'
{
  "default-runtime": "nvidia",
  "runtimes": {
    "nvidia": {
      "path": "nvidia-container-runtime",
      "runtimeArgs": []
    }
  }
}
EOF
sudo systemctl restart docker

Initialize and start Riva:

sudo bash riva_init.sh
sudo bash riva_start.sh

[OK] when Riva services start without error (check docker ps for riva containers). [STOP] if init fails — check disk space (Riva models are large) and JetPack compatibility.

Phase 4 — Clone and run the chatbot (~2 min)

git clone https://github.com/kouroshkarimi/local_chatbot_jetson.git
cd local_chatbot_jetson
pip3 install -r requirements.txt

List audio devices:

python3 app.py --list-input-devices
python3 app.py --list-output-devices

Start the chatbot with your audio devices:

python3 app.py --input-device <input_id> --output-device <output_id>

[OK] when the chatbot starts listening and responds to voice input. [STOP] if audio devices not found — check connections and ALSA config.

Failure decision tree

Reference files

• references/source.body.md — Full project documentation with system architecture, Ollama model table, Riva setup details, and usage examples (reference only)