ollama-local
Skill1 starsChanged 3 months agoMITView on GitHub
Use when the user wants to run an LLM locally without cloud API costs or data leaving their machine — installing Ollama, pulling/managing models, calling the REST API (generate, chat, embeddings, streaming), picking a model for their RAM budget, or wiring Ollama into Hermes as a provider.
- Pipes a download into a shell
- Installs packages
In this repository58 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- api-test-suiteSkill
- ascii-artSkill
- caddy-reverse-proxySkill
- changelog-generatorSkill
- color-palette-generatorSkill
- cron-taskSkill
- csv-toolkitSkill
- discord-bot-buildSkill
- docker-umbrellaSkill
- dotfiles-manageSkill
- email-sendSkill
- env-config-managerSkill
- excalidraw-diagramSkill
- file-organizerSkill
- forgejo-self-hostSkill
- frontend-design-toolkitSkill
- generate-dockerfileSkill
- gif-searchSkill
- git-backupSkill
- github-actions-ciSkill
- hallmark-readmeSkill
- http-api-testerSkill
- invoice-generatorSkill
- json-formatterSkill
- log-analyzerSkill
- markdown-linterSkill
- markdown-to-pdfSkill
- markdown-to-slidesSkill
- mcp-server-buildSkill
- media-id-mappingSkill
- movie-catalogue-siteSkill
- movie-night-calendarSkill
- ntfy-notifierSkill
- ocr-documentsSkill
- openapi-generatorSkill
- password-generatorSkill
- pdf-extractSkill
- portfolio-upkeepSkill
- qr-code-generatorSkill
- regex-testerSkill
- resume-builderSkill
- rss-monitorSkill
- searxng-self-hostSkill
- skill-publishSkill
- skill-registry-catalogSkill
- skills-portfolio-scaffoldSkill
- snippet-managerSkill
- sqlite-dashboardSkill
- streaming-provider-embedsSkill
- tailscale-deploySkill
- telegram-bot-buildSkill
- tmdb-metadataSkill
- uptime-kuma-self-hostSkill
- watchlist-syncSkill
- webhook-receiverSkill
- youtube-transcriptSkill
The file
We split this skill into the rules it gives an agent
1 rule. About 0 are common, 0 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈0
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 204
Port conflict on 11434: If another service already binds Ollama's default port, the server fails to start silently in some setups.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
ollama-local
Overview
Set up and use Ollama for running large language models locally. Ollama runs models on your machine — no API keys, no cloud, no per-token costs. The agent installs Ollama, pulls models, and shows you how to use them via the REST API or command line.
When to Use
- The user wants to run an LLM locally without paying for API access.
- The user wants privacy — no data leaves their machine.
- The user wants to use a local model with their agent or application.
- The user says "set up Ollama", "run a local LLM", or "I want offline AI".
Installation
Linux
curl -fsSL https://ollama.com/install.sh | shmacOS
# Via Homebrew
brew install ollama
# Or download from https://ollama.com/downloadWindows
Download from https://ollama.com/download and run the installer. Ollama runs as a background service on Windows.
Verify installation
ollama --version
# ollama version is 0.x.xModel Management
Pull a model
# Small, fast model (good for testing)
ollama pull llama3.2:3b
# Medium model (good balance of speed and quality)
ollama pull llama3.1:8b
# Large model (best quality, needs 16GB+ RAM)
ollama pull llama3.1:70b
# Coding-focused model
ollama pull qwen2.5-coder:7b
# Embedding model
ollama pull nomic-embed-textList installed models
ollama listRun a model (interactive chat)
ollama run llama3.1:8b
>>> Tell me about quantum computingRemove a model
ollama rm llama3.2:3bAPI Usage
Ollama exposes a REST API at http://localhost:11434:
Generate a response
curl http://localhost:11434/api/generate -d '{
"model": "llama3.1:8b",
"prompt": "Explain recursion in one sentence.",
"stream": false
}'Chat (multi-turn)
curl http://localhost:11434/api/chat -d '{
"model": "llama3.1:8b",
"messages": [
{"role": "user", "content": "What is 2+2?"},
{"role": "assistant", "content": "4"},
{"role": "user", "content": "What about 3+5?"}
],
"stream": false
}'Generate embeddings
curl http://localhost:11434/api/embeddings -d '{
"model": "nomic-embed-text",
"prompt": "The quick brown fox jumps over the lazy dog."
}'Python client
import requests
response = requests.post('http://localhost:11434/api/generate', json={
'model': 'llama3.1:8b',
'prompt': 'Write a haiku about the ocean.',
'stream': False
})
print(response.json()['response'])Streaming responses
import requests
response = requests.post('http://localhost:11434/api/generate', json={
'model': 'llama3.1:8b',
'prompt': 'Tell me a story.',
'stream': True
}, stream=True)
for line in response.iter_lines():
if line:
import json
chunk = json.loads(line)
print(chunk.get('response', ''), end='', flush=True)Integration with Hermes
Configure Hermes to use the local Ollama instance as a provider:
# Set Ollama as a custom provider
hermes config set model.provider custom
hermes config set model.base_url http://localhost:11434/v1
hermes config set model.api_key ollama # Ollama doesn't require a real key
hermes config set model.default llama3.1:8bOr use Ollama for specific tasks (like auxiliary/compression) while keeping a cloud model for main reasoning:
hermes config set auxiliary.compression.provider custom
hermes config set auxiliary.compression.base_url http://localhost:11434/v1
hermes config set auxiliary.compression.model llama3.2:3bModel Selection Guide
| Model | Size | RAM needed | Best for |
llama3.2:3b | 2 GB | 4 GB | Fast responses, simple tasks |
llama3.1:8b | 5 GB | 8 GB | General purpose, good balance |
qwen2.5-coder:7b | 5 GB | 8 GB | Code generation, debugging |
llama3.1:70b | 40 GB | 64 GB | High quality, complex reasoning |
nomic-embed-text | 0.3 GB | 1 GB | Embeddings for RAG/search |
Performance Tips
- Use GPU if available — Ollama auto-detects NVIDIA/AMD GPUs and Apple Silicon. GPU inference is substantially faster than CPU; measure on your own hardware, since the gap depends on the model, quantisation, and VRAM.
- Match model size to your RAM — A model that doesn't fit in RAM will spill to disk and become extremely slow. Check
ollama psto see if the model is fully in memory. - Use smaller models for simple tasks — Don't use a 70B model for a one-sentence answer. Use 3B or 8B for quick tasks.
- Keep models loaded — Ollama keeps models in memory for 5 minutes after last use by default. Increase this with
OLLAMA_KEEP_ALIVEenv var if you're making frequent requests. - Quantization — Ollama uses 4-bit quantization by default, which reduces memory usage by ~70% with minimal quality loss. No configuration needed.
Common Pitfalls
- Model larger than available RAM. Ollama will fall back to disk swap and performance becomes unusable rather than failing outright — check
ollama psto confirm the model is fully resident in memory, and use a smaller model or add RAM if not. - First pull looks "stuck". The first
ollama pulldownloads the full model file (multiple GB); this can take minutes on a slow connection. Subsequent runs use the cached model and start instantly — don't kill the process assuming it's hung. - Port conflict on 11434. If another service already binds Ollama's default port, the server fails to start silently in some setups. Set
OLLAMA_HOST=0.0.0.0:11435before starting and update client URLs to match. - GPU not detected. On Linux, missing NVIDIA drivers/CUDA toolkit means Ollama silently falls back to CPU (much slower) instead of erroring. Verify with
nvidia-smibefore assuming GPU is in use. - Prompts exceed local context comfortably. Local models advertise large context windows (e.g., 128k for Llama 3.1) but running at full context requires far more RAM than the base model size suggests. Keep prompts under ~8k tokens for 8B-class models in practice.
- Requests queue instead of running in parallel. Ollama processes requests sequentially by default, so concurrent callers block each other. Set
OLLAMA_NUM_PARALLELif concurrency is needed.
Verification Checklist
ollama --versionsucceeds andollama listshows the pulled modelollama psconfirms the model is loaded and shows a reasonable memory footprint (not swapping)- A test
curl http://localhost:11434/api/generatecall returns a non-emptyresponsefield - If GPU acceleration was expected,
ollama psor system GPU monitor (nvidia-smi) confirms it's actually being used - If wired into Hermes as a provider,
hermes config get model.base_urlreflects the correct local URL and a real Hermes call round-trips successfully
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

