Voice-Interactive Chatbot on Jetson (Physical AI)

Deploy a fully local voice chatbot for physical AI use cases — the system listens, reasons with a local LLM, and speaks back. Uses NVIDIA Riva for speech processing and Ollama for inference, all running in Docker on Jetson hardware.

Execution model

Run one phase at a time. After each phase:

• Relay all output to the user.
• If output contains [STOP] → stop, consult the failure decision tree.
• If output ends with [OK] → tell the user "Phase N complete" and proceed.

Prerequisites

Phase 1 — Install Jetson Containers and Ollama (~5 min)

git clone https://github.com/dusty-nv/jetson-containers
bash jetson-containers/install.sh

Run Ollama and pull a model:

jetson-containers run --name ollama $(autotag ollama)

Inside the container:

ollama run llama3.2:1b

Type /bye to exit after confirming the model loads.

Verify from host:

curl http://localhost:11434/api/tags

[OK] when curl returns JSON listing the model. [STOP] if Ollama container fails to start — check Docker and NVIDIA runtime.

Phase 2 — Install and configure NGC CLI (~3 min)

mkdir -p ~/ngc_setup && cd ~/ngc_setup
wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/ngc-apps/ngc_cli/versions/3.36.0/files/ngccli_arm64.zip
unzip ngccli_arm64.zip
chmod u+x ngc-cli/ngc
echo "export PATH=\"\$PATH:$(pwd)/ngc-cli\"" >> ~/.bash_profile
source ~/.bash_profile
ngc config set

Enter your NGC API key when prompted.

[OK] when ngc config current shows your org/team. [STOP] if API key is rejected — regenerate at catalog.ngc.nvidia.com.

Phase 3 — Install NVIDIA Riva (~15–30 min)

mkdir -p ~/riva_setup && cd ~/riva_setup
ngc registry resource download-version nvidia/riva/riva_quickstart_arm64:2.16.0
cd riva_quickstart_arm64_v2.16.0

Edit config.sh to disable unused services:

sed -i 's/service_enabled_nlp=.*/service_enabled_nlp=false/' config.sh
sed -i 's/service_enabled_nmt=.*/service_enabled_nmt=false/' config.sh

Configure Docker NVIDIA runtime (if not already set):

sudo tee /etc/docker/daemon.json > /dev/null << 'EOF'
{
  "default-runtime": "nvidia",
  "runtimes": {
    "nvidia": {
      "path": "nvidia-container-runtime",
      "runtimeArgs": []
    }
  }
}
EOF
sudo systemctl restart docker

Initialize and start Riva:

sudo bash riva_init.sh
sudo bash riva_start.sh

[OK] when Riva services start without error (check docker ps for riva containers). [STOP] if init fails — check disk space (Riva models are large) and JetPack compatibility.

Phase 4 — Clone and run the chatbot (~2 min)

git clone https://github.com/kouroshkarimi/local_chatbot_jetson.git
cd local_chatbot_jetson
pip3 install -r requirements.txt

List audio devices:

python3 app.py --list-input-devices
python3 app.py --list-output-devices

Start the chatbot with your audio devices:

python3 app.py --input-device <input_id> --output-device <output_id>

[OK] when the chatbot starts listening and responds to voice input. [STOP] if audio devices not found — check connections and ALSA config.

Failure decision tree

Reference files

• references/source.body.md — Full project documentation with system architecture, Ollama model table, Riva setup details, and usage examples (reference only)