Agent Instructions for python-openai-demos

This document provides comprehensive instructions for coding agents working on this repository. Following these instructions will help you work more efficiently and avoid common pitfalls.

Overview

This repository contains a collection of Python scripts that demonstrate how to use the OpenAI Responses API (and compatible APIs like Azure OpenAI and Ollama). The repository includes examples of:

• Basic responses (streaming, async, history)
• Function calling (basic to advanced multi-function scenarios)
• Structured outputs using Pydantic models
• Retrieval-Augmented Generation (RAG) with various complexity levels
• Prompt engineering and safety features

The scripts are designed to be educational and can run with multiple LLM providers: Azure OpenAI (preferred), OpenAI.com, or local Ollama models.

Code Layout

Python Scripts (Root Directory)

All example scripts are located in the root directory. They follow a consistent pattern of setting up an OpenAI client based on environment variables, then demonstrating specific API features.

Chat Scripts:

• chat.py - Simple response example
• chat_stream.py - Streaming responses
• chat_async.py - Async responses with asyncio.gather examples
• chat_history.py - Multi-turn chat with message history
• chat_history_stream.py - Multi-turn chat with streaming
• chat_safety.py - Content safety filter exception handling

Function Calling Scripts:

• function_calling_basic.py - Single function declaration, prints tool calls (no execution)
• function_calling_call.py - Executes the function once if the model requests it
• function_calling_extended.py - Full round-trip: executes, returns tool output, gets final answer
• function_calling_errors.py - Same as extended but with robust error handling (malformed JSON args, missing tool, tool exceptions, JSON serialization)
• function_calling_parallel.py - Shows model requesting multiple tools in one response
• function_calling_while_loop.py - Conversation loop that keeps executing sequential tool calls until the model produces a final natural language answer (with error handling)

Structured Outputs Scripts:

• structured_outputs_basic.py - Basic Pydantic model extraction
• structured_outputs_description.py - Using field descriptions
• structured_outputs_enum.py - Restricting values with enums
• structured_outputs_function_calling.py - Combining function calling with Pydantic
• structured_outputs_nested.py - Nested Pydantic models

RAG Scripts (require requirements-rag.txt):

• rag_csv.py - RAG with CSV file retrieval
• rag_multiturn.py - RAG with multi-turn chat interface
• rag_queryrewrite.py - RAG with query rewriting step
• rag_documents_ingestion.py - PDF ingestion (uses pymupdf, langchain, creates JSON)
• rag_documents_flow.py - RAG flow using ingested documents
• rag_documents_hybrid.py - Hybrid retrieval with vector + keyword search + RRF + reranking

Other Scripts:

• prompt_engineering.py - Prompt engineering examples
• reasoning.py - Reasoning examples
• chained_calls.py - Chained API calls
• retrieval_augmented_generation.py - Alternative RAG example

Spanish Directory

The spanish/ directory contains Spanish translations of most example scripts and a Spanish README. The Python scripts are functionally equivalent to their English counterparts.

Data Directory

The data/ directory contains PDF files used by the RAG document ingestion examples:

• Aphideater_hoverfly.pdf
• California_carpenter_bee.pdf
• Centris_pallida.pdf
• Western_honey_bee.pdf

Infrastructure Files (infra/)

Bicep Infrastructure as Code:

• infra/main.bicep - Defines Azure OpenAI resource provisioning with GPT-4o and embedding models
• infra/main.parameters.json - Parameters for Bicep deployment

Environment Setup Scripts:

• infra/write_dot_env.sh - Shell script to create .env from azd environment (Linux/Mac)
• infra/write_dot_env.ps1 - PowerShell script to create .env from azd environment (Windows)

These scripts are automatically run by azd provision via the azure.yaml postprovision hooks.

Configuration Files

Python Configuration:

• pyproject.toml - Ruff and Black linter/formatter configuration (line-length: 120, target: py311)
• requirements.txt - Core dependencies: azure-identity, openai>=1.108.1, python-dotenv, langchain-text-splitters
• requirements-rag.txt - RAG-specific dependencies: pymupdf4llm, lunr, sentence-transformers
• requirements-dev.txt - Development dependencies: includes requirements.txt + requirements-rag.txt + pre-commit, ruff, black

Pre-commit Configuration:

• .pre-commit-config.yaml - Pre-commit hooks for check-yaml, end-of-file-fixer, trailing-whitespace, ruff, and black

Azure Developer CLI:

• azure.yaml - Defines azd hooks (postprovision scripts)

Environment Variables:

• .env.sample - Example .env file showing all possible configurations
• .env.sample.azure - Azure-specific example
• .env.sample.ollama - Ollama example
• .env.sample.openai - OpenAI.com example

GitHub Workflows (.github/workflows/)

python.yaml - Linting and Formatting Check:

• Runs on: push to main, pull requests to main
• Uses: uv for virtual environment and dependency installation
• Runs: uv run ruff check . and uv run black . --check --verbose
• Important: The CI uses uv but local development typically uses standard pip

Dev Container Files (.devcontainer/)

• .devcontainer/devcontainer.json - Default dev container (Azure OpenAI setup with azd)
• .devcontainer/Dockerfile - Base Python 3.12 image, installs all requirements-dev.txt
• .devcontainer/ollama/ - Ollama variant
• .devcontainer/openai/ - OpenAI.com variant

All dev containers install all dependencies from requirements-dev.txt which includes base, RAG, and dev tools.

Running the Code

Environment Setup

ALWAYS follow these steps in order:

1. Create a Python virtual environment (if not in a Dev Container or Codespaces):

   python -m venv .venv
   source .venv/bin/activate  # On Windows: .venv\Scripts\activate

2. Install base dependencies:

   python -m pip install -r requirements.txt

3. For RAG examples, ALSO install RAG dependencies:

   python -m pip install -r requirements-rag.txt

4. For development (linting, formatting), install dev dependencies:

   python -m pip install -r requirements-dev.txt

   Note: requirements-dev.txt includes both requirements.txt and requirements-rag.txt via -r directives.

Configuring LLM Provider

The scripts read environment variables from a .env file. Create one based on your provider:

Option 1: Azure OpenAI (recommended)

For agents: Check if Azure OpenAI environment variables are already configured:

if [ -n "$AZURE_OPENAI_ENDPOINT" ] && [ -n "$AZURE_OPENAI_CHAT_DEPLOYMENT" ]; then
    echo "Azure OpenAI available - required environment variables are set"
else
    echo "Azure OpenAI not available - missing AZURE_OPENAI_ENDPOINT or AZURE_OPENAI_CHAT_DEPLOYMENT"
fi

If Azure OpenAI variables are not available, an administrator would need to provision Azure resources using:

azd auth login
azd provision

This creates real Azure resources that incur costs. The .env file would be created automatically with all needed variables after provisioning.

Option 2: OpenAI.com (requires API key and costs)

For agents: Check if OpenAI.com API key is available:

if [ -n "$OPENAI_API_KEY" ]; then
    echo "OpenAI.com available - OPENAI_API_KEY is set"
else
    echo "OpenAI.com not available - OPENAI_API_KEY not found"
fi

If OPENAI_API_KEY is available, ensure API_HOST=openai and OPENAI_MODEL are also set (e.g., gpt-4o-mini).

Option 3: Ollama (requires local Ollama installation)

For agents: Check if Ollama is installed and running:

if command -v ollama &> /dev/null; then
    echo "Ollama command found"
    ollama list
    if [ $? -eq 0 ]; then
        echo "Ollama is running and available"
    else
        echo "Ollama installed but not running"
    fi
else
    echo "Ollama not installed"
fi

If Ollama is available, configure the environment:

# API_HOST=ollama
# OLLAMA_ENDPOINT=http://localhost:11434/v1
# OLLAMA_MODEL=llama3.1

Important: If running in a Dev Container, use http://host.docker.internal:11434/v1 instead of localhost.

Running Scripts

After environment setup and .env configuration, run any script:

python chat.py
python function_calling_basic.py
python rag_csv.py  # Requires requirements-rag.txt to be installed

For Spanish versions:

python spanish/chat.py

Running Tests

Linting with Ruff

# After installing requirements-dev.txt
ruff check .

To auto-fix issues:

ruff check . --fix

Formatting with Black

Check formatting:

black . --check --verbose

Auto-format:

black .

Pre-commit Hooks

Install pre-commit hooks to automatically run checks before commits:

pre-commit install

Run manually:

pre-commit run --all-files

Integration Tests

The repository has limited automated testing via GitHub Actions. Changes to scripts should be manually verified by running them:

python chat.py
python spanish/chat.py

Note: Most scripts are demonstration scripts, not unit-tested. Changes to scripts should be manually verified by running them.

Important Notes and Gotchas

Environment Variables

• All scripts default to API_HOST=azure if no .env file is present and no environment variable is set.
• Scripts use load_dotenv(override=True) which means .env values override environment variables.

Model Compatibility

• Function calling scripts require models that support tools. Not all models support this:
  • ✅ Supported: gpt-4o, gpt-4o-mini, gpt-5.4, and many others
  • ❌ Not supported: Older models, some local Ollama models
• If a script fails with a function calling error, check if your model supports the tools parameter.

RAG Scripts Dependencies

• RAG scripts require requirements-rag.txt to be installed. They will fail with import errors if these dependencies are missing.
• The rag_documents_ingestion.py script creates rag_ingested_chunks.json (large file, ~6MB) which is used by rag_documents_flow.py and rag_documents_hybrid.py.
• The ingestion process requires the PDF files in the data/ directory.

Dev Container Behavior

• Dev containers automatically install all dependencies from requirements-dev.txt during build.
• Azure-focused dev container includes azure-cli and azd features.
• The default dev container expects Azure OpenAI configuration.

Linting and Formatting

• Line length is 120 characters (configured in pyproject.toml).
• Target Python version is 3.11 (though 3.12 is used in dev containers).
• Ruff is configured to check: E (errors), F (pyflakes), I (isort), UP (pyupgrade).
• Black and Ruff should both pass for CI to succeed.

CI/CD Workflow Differences

• CI uses uv for faster installation: uv venv .venv && uv pip install -r requirements-dev.txt
• Local development typically uses pip: python -m venv .venv && python -m pip install -r requirements-dev.txt
• Both approaches work, but uv is faster for CI.

Azure Provisioning

• Running azd provision creates real Azure resources that incur costs.
• The infrastructure creates:
  • Azure OpenAI account with gpt-4o deployment (capacity: 30)
  • Text embedding deployment text-embedding-3-small (capacity: 30)
• The postprovision hook automatically creates the .env file.
• Always run azd down when done to avoid ongoing charges.

File Organization

• Spanish translations are in spanish/ directory with their own README and scripts.
• Spanish scripts are functionally identical to English versions, just translated.
• Both English and Spanish directories have their own rag_ingested_chunks.json files.

Common Errors and Solutions

Error: ImportError: No module named 'pymupdf4llm'

• Solution: Install RAG dependencies: python -m pip install -r requirements-rag.txt

Error: KeyError: 'AZURE_OPENAI_ENDPOINT'

• Solution: Your .env file is missing required Azure variables, or API_HOST is set to azure but you haven't configured Azure. Run azd provision or configure Azure properly.

Error: openai.APIError: content_filter

• This is expected behavior for chat_safety.py - it's demonstrating content filtering.
• The script catches this error and prints a message.

Error: Function calling not supported

• Solution: Use a model that supports tools, such as gpt-4o, gpt-4o-mini, or gpt-5.4.

Error: azd command not found

• Solution: Install Azure Developer CLI: https://aka.ms/install-azd

Agent Workflow Recommendations

When making code changes:

1. Always install dependencies first: python -m pip install -r requirements-dev.txt
2. Run linters before making changes to understand baseline: ruff check . and black . --check
3. Make minimal, surgical changes to the relevant scripts.
4. Run linters again after changes: ruff check . and black . (auto-fix)
5. Manually test the changed script:

   python your_modified_script.py

6. Check that Spanish translations are updated if applicable.

Trust These Instructions

These instructions have been validated by:

• Reviewing all documentation (README.md, CONTRIBUTING.md)
• Examining all configuration files (pyproject.toml, .pre-commit-config.yaml, azure.yaml)
• Analyzing all CI/CD workflows (.github/workflows/)
• Inspecting all infrastructure files (infra/)
• Testing environment setup and linting commands
• Verifying dependency installation processes

If you encounter discrepancies between these instructions and the actual repository state, the repository may have been updated. In that case:

1. Re-examine the relevant documentation or configuration files
2. Update these instructions if needed
3. Otherwise, trust these instructions and only explore further if they are incomplete or proven incorrect