truefoundry-integrate-gateway
Skill14 starsChanged 6 months agoMITView on GitHub
Integrates a codebase with TrueFoundry AI Gateway. Scans for all LLM calls, MCP configs, and credentials, diffs against existing gateway config, generates a migration plan, applies code changes, and verifies routing end-to-end. Invoked from within the customer's codebase.
- Reads credentials
In this repository8 other files it gives its agents.
The file
Show its rules and text
We split this skill into the rules it gives an agent
18 rules. About 0 are common, 8 are distinctive, and 10 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈8
Distinctive · estimate
This author's own choices. Worth a look.
10
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 20
Ensure every model in use is configured in the gateway
- Very specific: line 25
No TFY account yet -> onboard skill first
- Distinctive: line 37
Read actual source files to understand how clients are instantiated
- Very specific: line 41
Check if .env is gitignored before any key operations
- Distinctive: line 44
Read existing CI/CD configs to understand deployment flow
- Distinctive: line 68
Before touching anything, verify safety:
- Distinctive: line 96
Run the scanner script for structured output:
- Distinctive: line 201
Never handle raw keys. Tell the user:
- Very specific: line 222
If user chose "keep existing model names", create virtual model routing config.
- Very specific: line 234
Safety: Keep original .env backed up.
- Distinctive: line 251
Test each model found in the codebase:
- Distinctive: line 294
All models verified working through gateway
- Very specific: line 326
STOP. Fix this before any key operations: echo '.env' >> .gitignore && git add .gitignore
- Very specific: line 329
Use onboard skill. Do not proceed with integration.
- Very specific: line 332
Check: correct project root?
- Very specific: line 335
Verify: integration has correct model_types, collaborators include team:everyone, provider account status is active.
- Very specific: line 338
Check: TFY_API_KEY set to valid PAT/VAT, OPENAI_BASE_URL points to ${TFY_BASE_URL}/api/llm (not hardcoded gateway URL).
- Very specific: line 344
Ask which services to scan.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
<objective>Integrate Gateway
Integrate the current codebase with TrueFoundry AI Gateway. This skill runs inside the customer's repo and has full filesystem access — use it for deep analysis, not surface scanning.
When to Use
- Customer has a TFY account and wants to route AI traffic through the gateway
- Audit which LLM calls bypass the gateway
- Migrate hardcoded credentials to TFY Secrets
- Ensure every model in use is configured in the gateway
- Customer says "integrate", "connect to gateway", "route through TrueFoundry", "audit my AI calls"
When NOT to Use
- No TFY account yet ->
onboardskill first - Just wants to call a model ->
gatewayskill - Wants to configure routing/guardrails without migration ->
gatewayskill - Wants to deploy a model -> Enterprise feature
</objective><context>What This Skill Does That Others Don't
This skill has the codebase. It can:
- Read actual source files to understand how clients are instantiated
- Trace env var references to
.envfiles and docker-compose configs - Detect whether code uses env vars (easy migration) or hardcodes values (needs edits)
- Identify the framework stack (bare SDK vs LangChain vs LlamaIndex vs Vercel AI)
- Check if
.envis gitignored before any key operations - Detect monorepo structure and scope appropriately
- Find model names used across the codebase for virtual model planning
- Read existing CI/CD configs to understand deployment flow
Integration Flow
1. PREFLIGHT 2. SCAN 3. DIFF 4. PLAN 5. APPLY 6. VERIFY
Check login → Find all AI → Compare with → Present → Make changes → Smoke test
& .env safety call sites gateway state migration (with conf) each modelGateway Model ID Format
Models in the gateway use {provider_account_name}/{model_name}:
openai-main/gpt-4o(notgpt-4o)anthropic-main/claude-sonnet-4-20250514(notclaude-sonnet-4-20250514)
Virtual models allow keeping original names — gateway handles the mapping.
</context><instructions>Phase 1: Preflight
Before touching anything, verify safety:
# 1. Check TFY login exists
python3 -c "
import json; from pathlib import Path
d = json.loads((Path.home()/'.truefoundry'/'credentials.json').read_text())
host = d.get('host') or d.get('base_url') or ''
token = d.get('access_token') or d.get('refresh_token') or ''
print(f'tfy login: {\"ok (\" + host + \")\" if host and token else \"missing\"}')" 2>/dev/null || echo "tfy login: missing"
# 2. Check .env is gitignored (CRITICAL — before any key operations)
if [ -f .env ]; then
git check-ignore .env >/dev/null 2>&1 && echo ".env: gitignored (safe)" || echo ".env: NOT GITIGNORED — fix this first"
fi
# 3. Check TFY env vars
echo "TFY_BASE_URL: ${TFY_BASE_URL:-(not set)}"
echo "TFY_API_KEY: ${TFY_API_KEY:+(set)}${TFY_API_KEY:-(not set)}"Stop if:
tfy login: missing-> useonboardskill.env: NOT GITIGNORED-> fix.gitignorebefore proceedingTFY_API_KEY: (not set)-> user needs a PAT from dashboard
Phase 2: Deep Scan
Run the scanner script for structured output:
scripts/scan.sh . --skip-testsThen go deeper — read the actual files to understand patterns:
2.1 Understand Client Instantiation
For each LLM call site found, read the surrounding code to classify:
| Pattern | Effort | Migration |
OpenAI() — no args, reads env | None | Just update env vars |
OpenAI(api_key=os.environ["X"]) — key from env, no base_url | Low | Add OPENAI_BASE_URL to env |
OpenAI(api_key="sk-...", base_url="...") — hardcoded | Medium | Rewrite to use env vars |
ChatOpenAI(model="gpt-4o") — framework with params | Medium | Add base_url or set env |
| Multiple providers with different keys | High | Consolidate to single TFY PAT |
2.2 Trace the Dependency Chain
# Where do env vars get set?
rg -rn 'OPENAI_API_KEY|OPENAI_BASE_URL' --include='*.env*' --include='docker-compose*' --include='*.yaml' --include='*.yml' .
# CI/CD pipeline env injection?
find . -name '*.yml' -path '*/.github/*' -exec grep -l 'OPENAI\|ANTHROPIC\|LLM' {} \;
find . -name 'Dockerfile*' -exec grep -l 'OPENAI\|ANTHROPIC\|API_KEY' {} \;2.3 Detect Framework & Architecture
# Package manager files reveal the stack
cat package.json 2>/dev/null | python3 -c "import sys,json; d=json.load(sys.stdin).get('dependencies',{}); [print(k) for k in d if 'openai' in k or 'anthropic' in k or 'ai-sdk' in k or 'langchain' in k]" 2>/dev/null
cat requirements.txt 2>/dev/null | grep -i 'openai\|anthropic\|langchain\|llama.index\|litellm' 2>/dev/null
cat pyproject.toml 2>/dev/null | grep -i 'openai\|anthropic\|langchain\|llama.index\|litellm' 2>/dev/nullPhase 3: Diff Against Gateway
Compare what the codebase uses against what's already configured:
TFY_API_SH="$(find ~/.claude/skills ~/.cursor/skills ~/.codex/skills -name tfy-api.sh 2>/dev/null | head -1)"
# What's already in the gateway?
$TFY_API_SH GET /api/svc/v1/provider-accounts
# What models are callable?
curl -s "${TFY_BASE_URL}/api/llm/models" -H "Authorization: Bearer ${TFY_API_KEY}" | python3 -c "import sys,json; [print(m['id']) for m in json.load(sys.stdin).get('data',[])]"Produce a gap analysis:
- Models in code but NOT in gateway -> need provider account or integration
- Models in gateway but NOT in code -> already configured, no action
- Provider keys in
.envbut NOT in TFY Secrets -> need secret creation
Phase 4: Present Migration Plan
Present findings as an actionable report. Format:
## Integration Summary
| Metric | Count |
|--------|-------|
| LLM call sites found | X |
| Already routed through TFY | Y |
| Need migration | Z |
| Hardcoded credentials (CRITICAL) | N |
| Models missing from gateway | M |
## Call Sites (by migration effort)
### No changes needed (env-var driven)
| File | Line | SDK | Why it works already |
...
### Low effort (add env var)
| File | Line | SDK | What to add |
...
### Medium effort (code change)
| File | Line | SDK | Current → Required change |
...
## Gateway Gaps
| Model | Provider | Status | Action |
...
## Recommended approach:
- [ ] Strategy A or B for model names? (ask user)
- [ ] N env files to update
- [ ] M provider accounts to create
- [ ] Estimated: X minutesAsk for confirmation before proceeding to Phase 5.
Phase 5: Apply Changes
5.1 Store Provider Keys in TFY Secrets
Never handle raw keys. Tell the user:
"Set your provider keys as environment variables, then I'll store them in TFY Secrets:
! export OPENAI_API_KEY=<your-key>"
Then create secret group:
$TFY_API_SH POST /api/svc/v1/secret-groups "$payload"5.2 Create Missing Provider Accounts
For templates, see references/provider-manifests.md.
tfy apply -f provider-account.yaml --dry-run --show-diff
tfy apply -f provider-account.yaml5.3 Create Virtual Models (if Strategy B)
If user chose "keep existing model names", create virtual model routing config. See gateway skill [references/calling-models.md] for gateway-load-balancing-config manifest.
5.4 Update Environment Files
Replace provider keys with TFY PAT and add base URL:
# .env — always use TFY_BASE_URL, never hardcode gateway.truefoundry.ai
OPENAI_API_KEY=<tfy-pat>
OPENAI_BASE_URL=${TFY_BASE_URL}/api/llmSafety: Keep original .env backed up. Original provider keys are now in TFY Secrets — confirm they're accessible before removing from .env.
5.5 Apply Code Changes (if needed)
For complete before/after patterns per framework, see references/migration-code-patterns.md.
Always ask for confirmation before modifying source files.
5.6 Update .gitignore
grep -q '^\.env$' .gitignore 2>/dev/null || echo '.env' >> .gitignore
grep -q '^\.env\.local$' .gitignore 2>/dev/null || echo '.env.local' >> .gitignorePhase 6: Verify
Test each model found in the codebase:
scripts/verify-gateway.sh "openai-main/gpt-4o-mini"Or if using virtual models (Strategy B):
scripts/verify-gateway.sh "gpt-4o-mini"Present final status:
## Integration Complete
| Model | Status | Latency |
|-------|--------|---------|
| openai-main/gpt-4o | OK | 1.2s |
| anthropic-main/claude-sonnet | OK | 0.8s |
What's now active:
- Cost tracking in TFY dashboard → AI Gateway → Observability
- Request traces with latency, tokens, cost per call
- Budget controls and rate limiting available
Next steps:
- [ ] Set up budget alerts (gateway skill)
- [ ] Add guardrails for content safety
- [ ] Create production VAT for deployed services
- [ ] Configure CI/CD to use TFY_API_KEY secret</instructions><success_criteria>
- Preflight confirms login, .env safety, and API key availability
- Deep scan identifies all LLM call sites with file/line/pattern classification
- Gap analysis shows exactly what's missing from gateway config
- Migration plan presented with effort estimates before any changes
- User confirms before any file modifications
- Provider keys stored in TFY Secrets (never in conversation)
- All models verified working through gateway
- Original provider keys confirmed accessible in TFY Secrets before removal from .env
</success_criteria>
<references>References
- migration-code-patterns.md — Before/after code patterns per framework, verification scripts
- migration-patterns.md — Full migration pattern reference with decision matrix
- provider-manifests.md — Provider account YAML templates
- cli-reference.md — CLI commands (
tfy apply,tfy login) - api-endpoints.md — REST API reference
Scripts
scripts/scan.sh [dir] [--skip-tests]— Structured codebase scan (run first)scripts/verify-gateway.sh [model_id]— Smoke test a model through the gateway
Composability
- Gateway config:
gatewayskill for routing, rate limits, guardrails, virtual models - Secrets:
platformskill (Secrets section) for advanced secret management - MCP servers:
mcp-serversskill to register discovered MCP configs in the registry - Onboarding:
onboardskill if customer has no TFY account
</references><troubleshooting>.env Not Gitignored
STOP. Fix this before any key operations: echo '.env' >> .gitignore && git add .gitignore
tfy login Missing
Use onboard skill. Do not proceed with integration.
No LLM Calls Found
Check: correct project root? Code in subdirectory? Try scripts/scan.sh ./src. Ask if framework isn't covered.
Model Not Found After Provider Account Creation
Verify: integration has correct model_types, collaborators include team:everyone, provider account status is active.
401 After Migration
Check: TFY_API_KEY set to valid PAT/VAT, OPENAI_BASE_URL points to ${TFY_BASE_URL}/api/llm (not hardcoded gateway URL).
Original Keys Lost
Keys should remain in TFY Secrets. Check: $TFY_API_SH GET /api/svc/v1/secret-groups and verify the group exists.
Monorepo — Scan Too Broad
Ask which services to scan. Scope: scripts/scan.sh ./services/api --skip-tests
LiteLLM Already Used as Router
Options: (1) replace with TFY Gateway, (2) point LiteLLM at gateway for observability, (3) keep LiteLLM for dev only.
</troubleshooting>Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

