agentleFS
truefoundrytruefoundry/skills/skills/integrate-gateway/SKILL.md

truefoundry-integrate-gateway

Skill14 starsChanged 6 months agoMITView on GitHub

Integrates a codebase with TrueFoundry AI Gateway. Scans for all LLM calls, MCP configs, and credentials, diffs against existing gateway config, generates a migration plan, applies code changes, and verifies routing end-to-end. Invoked from within the customer's codebase.

  • Reads credentials

The file

Show its rules and text

We split this skill into the rules it gives an agent

18 rules. About 0 are common, 8 are distinctive, and 10 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈8

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 10

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Ensure every model in use is configured in the gateway

    line 20
  2. Very specific:

    No TFY account yet -> onboard skill first

    line 25
  3. Distinctive:

    Read actual source files to understand how clients are instantiated

    line 37
  4. Very specific:

    Check if .env is gitignored before any key operations

    line 41
  5. Distinctive:

    Read existing CI/CD configs to understand deployment flow

    line 44
  6. Distinctive:

    Before touching anything, verify safety:

    line 68
  7. Distinctive:

    Run the scanner script for structured output:

    line 96
  8. Distinctive:

    Never handle raw keys. Tell the user:

    line 201
  9. Very specific:

    If user chose "keep existing model names", create virtual model routing config.

    line 222
  10. Very specific:

    Safety: Keep original .env backed up.

    line 234
  11. Distinctive:

    Test each model found in the codebase:

    line 251
  12. Distinctive:

    All models verified working through gateway

    line 294
  13. Very specific:

    STOP. Fix this before any key operations: echo '.env' >> .gitignore && git add .gitignore

    line 326
  14. Very specific:

    Use onboard skill. Do not proceed with integration.

    line 329
  15. Very specific:

    Check: correct project root?

    line 332
  16. Very specific:

    Verify: integration has correct model_types, collaborators include team:everyone, provider account status is active.

    line 335
  17. Very specific:

    Check: TFY_API_KEY set to valid PAT/VAT, OPENAI_BASE_URL points to ${TFY_BASE_URL}/api/llm (not hardcoded gateway URL).

    line 338
  18. Very specific:

    Ask which services to scan.

    line 344

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md
<objective>

Integrate Gateway

Integrate the current codebase with TrueFoundry AI Gateway. This skill runs inside the customer's repo and has full filesystem access — use it for deep analysis, not surface scanning.

When to Use

  • Customer has a TFY account and wants to route AI traffic through the gateway
  • Audit which LLM calls bypass the gateway
  • Migrate hardcoded credentials to TFY Secrets
  • Ensure every model in use is configured in the gateway
  • Customer says "integrate", "connect to gateway", "route through TrueFoundry", "audit my AI calls"

When NOT to Use

  • No TFY account yet -> onboard skill first
  • Just wants to call a model -> gateway skill
  • Wants to configure routing/guardrails without migration -> gateway skill
  • Wants to deploy a model -> Enterprise feature
</objective><context>

What This Skill Does That Others Don't

This skill has the codebase. It can:

  • Read actual source files to understand how clients are instantiated
  • Trace env var references to .env files and docker-compose configs
  • Detect whether code uses env vars (easy migration) or hardcodes values (needs edits)
  • Identify the framework stack (bare SDK vs LangChain vs LlamaIndex vs Vercel AI)
  • Check if .env is gitignored before any key operations
  • Detect monorepo structure and scope appropriately
  • Find model names used across the codebase for virtual model planning
  • Read existing CI/CD configs to understand deployment flow

Integration Flow

1. PREFLIGHT     2. SCAN         3. DIFF          4. PLAN         5. APPLY        6. VERIFY
Check login   →  Find all AI  →  Compare with →  Present     →  Make changes → Smoke test
& .env safety    call sites      gateway state    migration      (with conf)    each model

Gateway Model ID Format

Models in the gateway use {provider_account_name}/{model_name}:

  • openai-main/gpt-4o (not gpt-4o)
  • anthropic-main/claude-sonnet-4-20250514 (not claude-sonnet-4-20250514)

Virtual models allow keeping original names — gateway handles the mapping.

</context><instructions>

Phase 1: Preflight

Before touching anything, verify safety:

# 1. Check TFY login exists
python3 -c "
import json; from pathlib import Path
d = json.loads((Path.home()/'.truefoundry'/'credentials.json').read_text())
host = d.get('host') or d.get('base_url') or ''
token = d.get('access_token') or d.get('refresh_token') or ''
print(f'tfy login: {\"ok (\" + host + \")\" if host and token else \"missing\"}')" 2>/dev/null || echo "tfy login: missing"

# 2. Check .env is gitignored (CRITICAL — before any key operations)
if [ -f .env ]; then
  git check-ignore .env >/dev/null 2>&1 && echo ".env: gitignored (safe)" || echo ".env: NOT GITIGNORED — fix this first"
fi

# 3. Check TFY env vars
echo "TFY_BASE_URL: ${TFY_BASE_URL:-(not set)}"
echo "TFY_API_KEY: ${TFY_API_KEY:+(set)}${TFY_API_KEY:-(not set)}"

Stop if:

  • tfy login: missing -> use onboard skill
  • .env: NOT GITIGNORED -> fix .gitignore before proceeding
  • TFY_API_KEY: (not set) -> user needs a PAT from dashboard

Phase 2: Deep Scan

Run the scanner script for structured output:

scripts/scan.sh . --skip-tests

Then go deeper — read the actual files to understand patterns:

2.1 Understand Client Instantiation

For each LLM call site found, read the surrounding code to classify:

PatternEffortMigration
OpenAI() — no args, reads envNoneJust update env vars
OpenAI(api_key=os.environ["X"]) — key from env, no base_urlLowAdd OPENAI_BASE_URL to env
OpenAI(api_key="sk-...", base_url="...") — hardcodedMediumRewrite to use env vars
ChatOpenAI(model="gpt-4o") — framework with paramsMediumAdd base_url or set env
Multiple providers with different keysHighConsolidate to single TFY PAT

2.2 Trace the Dependency Chain

# Where do env vars get set?
rg -rn 'OPENAI_API_KEY|OPENAI_BASE_URL' --include='*.env*' --include='docker-compose*' --include='*.yaml' --include='*.yml' .

# CI/CD pipeline env injection?
find . -name '*.yml' -path '*/.github/*' -exec grep -l 'OPENAI\|ANTHROPIC\|LLM' {} \;
find . -name 'Dockerfile*' -exec grep -l 'OPENAI\|ANTHROPIC\|API_KEY' {} \;

2.3 Detect Framework & Architecture

# Package manager files reveal the stack
cat package.json 2>/dev/null | python3 -c "import sys,json; d=json.load(sys.stdin).get('dependencies',{}); [print(k) for k in d if 'openai' in k or 'anthropic' in k or 'ai-sdk' in k or 'langchain' in k]" 2>/dev/null
cat requirements.txt 2>/dev/null | grep -i 'openai\|anthropic\|langchain\|llama.index\|litellm' 2>/dev/null
cat pyproject.toml 2>/dev/null | grep -i 'openai\|anthropic\|langchain\|llama.index\|litellm' 2>/dev/null

Phase 3: Diff Against Gateway

Compare what the codebase uses against what's already configured:

TFY_API_SH="$(find ~/.claude/skills ~/.cursor/skills ~/.codex/skills -name tfy-api.sh 2>/dev/null | head -1)"

# What's already in the gateway?
$TFY_API_SH GET /api/svc/v1/provider-accounts

# What models are callable?
curl -s "${TFY_BASE_URL}/api/llm/models" -H "Authorization: Bearer ${TFY_API_KEY}" | python3 -c "import sys,json; [print(m['id']) for m in json.load(sys.stdin).get('data',[])]"

Produce a gap analysis:

  • Models in code but NOT in gateway -> need provider account or integration
  • Models in gateway but NOT in code -> already configured, no action
  • Provider keys in .env but NOT in TFY Secrets -> need secret creation

Phase 4: Present Migration Plan

Present findings as an actionable report. Format:

## Integration Summary

| Metric | Count |
|--------|-------|
| LLM call sites found | X |
| Already routed through TFY | Y |
| Need migration | Z |
| Hardcoded credentials (CRITICAL) | N |
| Models missing from gateway | M |

## Call Sites (by migration effort)

### No changes needed (env-var driven)
| File | Line | SDK | Why it works already |
...

### Low effort (add env var)
| File | Line | SDK | What to add |
...

### Medium effort (code change)
| File | Line | SDK | Current → Required change |
...

## Gateway Gaps
| Model | Provider | Status | Action |
...

## Recommended approach:
- [ ] Strategy A or B for model names? (ask user)
- [ ] N env files to update
- [ ] M provider accounts to create
- [ ] Estimated: X minutes

Ask for confirmation before proceeding to Phase 5.

Phase 5: Apply Changes

5.1 Store Provider Keys in TFY Secrets

Never handle raw keys. Tell the user:

"Set your provider keys as environment variables, then I'll store them in TFY Secrets: ! export OPENAI_API_KEY=<your-key>"

Then create secret group:

$TFY_API_SH POST /api/svc/v1/secret-groups "$payload"

5.2 Create Missing Provider Accounts

For templates, see references/provider-manifests.md.

tfy apply -f provider-account.yaml --dry-run --show-diff
tfy apply -f provider-account.yaml

5.3 Create Virtual Models (if Strategy B)

If user chose "keep existing model names", create virtual model routing config. See gateway skill [references/calling-models.md] for gateway-load-balancing-config manifest.

5.4 Update Environment Files

Replace provider keys with TFY PAT and add base URL:

# .env — always use TFY_BASE_URL, never hardcode gateway.truefoundry.ai
OPENAI_API_KEY=<tfy-pat>
OPENAI_BASE_URL=${TFY_BASE_URL}/api/llm

Safety: Keep original .env backed up. Original provider keys are now in TFY Secrets — confirm they're accessible before removing from .env.

5.5 Apply Code Changes (if needed)

For complete before/after patterns per framework, see references/migration-code-patterns.md.

Always ask for confirmation before modifying source files.

5.6 Update .gitignore

grep -q '^\.env$' .gitignore 2>/dev/null || echo '.env' >> .gitignore
grep -q '^\.env\.local$' .gitignore 2>/dev/null || echo '.env.local' >> .gitignore

Phase 6: Verify

Test each model found in the codebase:

scripts/verify-gateway.sh "openai-main/gpt-4o-mini"

Or if using virtual models (Strategy B):

scripts/verify-gateway.sh "gpt-4o-mini"

Present final status:

## Integration Complete

| Model | Status | Latency |
|-------|--------|---------|
| openai-main/gpt-4o | OK | 1.2s |
| anthropic-main/claude-sonnet | OK | 0.8s |

What's now active:
- Cost tracking in TFY dashboard → AI Gateway → Observability
- Request traces with latency, tokens, cost per call
- Budget controls and rate limiting available

Next steps:
- [ ] Set up budget alerts (gateway skill)
- [ ] Add guardrails for content safety
- [ ] Create production VAT for deployed services
- [ ] Configure CI/CD to use TFY_API_KEY secret
</instructions>

<success_criteria>

  • Preflight confirms login, .env safety, and API key availability
  • Deep scan identifies all LLM call sites with file/line/pattern classification
  • Gap analysis shows exactly what's missing from gateway config
  • Migration plan presented with effort estimates before any changes
  • User confirms before any file modifications
  • Provider keys stored in TFY Secrets (never in conversation)
  • All models verified working through gateway
  • Original provider keys confirmed accessible in TFY Secrets before removal from .env

</success_criteria>

<references>

References

  • migration-code-patterns.md — Before/after code patterns per framework, verification scripts
  • migration-patterns.md — Full migration pattern reference with decision matrix
  • provider-manifests.md — Provider account YAML templates
  • cli-reference.md — CLI commands (tfy apply, tfy login)
  • api-endpoints.md — REST API reference

Scripts

  • scripts/scan.sh [dir] [--skip-tests] — Structured codebase scan (run first)
  • scripts/verify-gateway.sh [model_id] — Smoke test a model through the gateway

Composability

  • Gateway config: gateway skill for routing, rate limits, guardrails, virtual models
  • Secrets: platform skill (Secrets section) for advanced secret management
  • MCP servers: mcp-servers skill to register discovered MCP configs in the registry
  • Onboarding: onboard skill if customer has no TFY account
</references><troubleshooting>

.env Not Gitignored

STOP. Fix this before any key operations: echo '.env' >> .gitignore && git add .gitignore

tfy login Missing

Use onboard skill. Do not proceed with integration.

No LLM Calls Found

Check: correct project root? Code in subdirectory? Try scripts/scan.sh ./src. Ask if framework isn't covered.

Model Not Found After Provider Account Creation

Verify: integration has correct model_types, collaborators include team:everyone, provider account status is active.

401 After Migration

Check: TFY_API_KEY set to valid PAT/VAT, OPENAI_BASE_URL points to ${TFY_BASE_URL}/api/llm (not hardcoded gateway URL).

Original Keys Lost

Keys should remain in TFY Secrets. Check: $TFY_API_SH GET /api/svc/v1/secret-groups and verify the group exists.

Monorepo — Scan Too Broad

Ask which services to scan. Scope: scripts/scan.sh ./services/api --skip-tests

LiteLLM Already Used as Router

Options: (1) replace with TFY Gateway, (2) point LiteLLM at gateway for observability, (3) keep LiteLLM for dev only.

</troubleshooting>

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.