agentleFS
MicrosoftKnown publishermicrosoft/bobsql/examples/sqlserver2025/zavahealth/zavahospital/nvidianim/SKILL.md

nvidianim

Skill1 starsChanged 12 days agoMITView on GitHub

NVIDIA NIM on AKS with GPU — vector embeddings and chat completions from SQL Server 2025 T-SQL using GPU-accelerated inference. Use when: deploying NIM containers on AKS, creating AKS GPU node pools, testing NIM endpoints, creating external models for NIM embeddings, Azure Local edge AI pattern, working in nvidianim/.

  • Reads credentials

The file

We split this skill into the rules it gives an agent

11 rules. About 0 are common, 4 are distinctive, and 7 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈4

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 7

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Create AKS cluster: .\create-aks-cluster.ps1

    line 24
  2. Distinctive:

    Wait for pods + ingress ready

    line 26
  3. Very specific:

    Test endpoints: .\test-embedding.ps1, .\test-chat.ps1

    line 28
  4. Very specific:

    Run SQL scripts: sql\00_setup.sql through sql\04_test_chat_completion.sql

    line 29
  5. Very specific:

    If pods stuck in ImagePullBackOff → check NGC_API_KEY is correct and K8s secret is created

    line 83
  6. Very specific:

    If pods stuck in Pending → GPU node pool may not have scaled up yet (wait ~5-10 min)

    line 84
  7. Distinctive:

    If ingress has no IP → check Web Application Routing add-on is enabled

    line 85
  8. Distinctive:

    If SQL returns TLS errors → ensure cert is in Windows Trusted Root and hosts file has correct IP

    line 86
  9. Very specific:

    If NIM returns "Unsupported media type" → ingress.yaml needs proxy_set_header Content-Type annotation

    line 87
  10. Distinctive:

    If chat model gives bad answers → add grounding facts in the system prompt (3B model needs context)

    line 88
  11. Very specific:

    If 8B chat model OOMs on T4 → reduce NIM_MAX_MODEL_LEN or use 3B model with system prompt grounding

    line 89

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

NVIDIA NIM + AKS + SQL Server 2025 Demo

Architecture

  • AKS cluster in westus2 with 2x NC4as_T4_v3 GPU nodes (1x NVIDIA T4 16GB each)
  • NIM chat: meta/llama-3.2-3b-instruct on dedicated T4
  • NIM embeddings: nvidia/nv-embedqa-e5-v5 (1024 dims) on dedicated T4
  • SQL Server 2025 localhost, Windows auth, FoundryLocalTest database
  • TLS: Self-signed cert (CN=nim-aks.local) via AKS Web Application Routing (managed NGINX ingress)
  • Hostname: nim-aks.local via Windows hosts file → AKS ingress IP (get with kubectl get ingress -n nim)
  • Content-Type fix: NGINX annotation proxy_set_header Content-Type "application/json" strips charset suffix that NIM rejects
  • NGC API key required to pull NIM containers from nvcr.io

Flow

SQL Server 2025 → HTTPS → AKS Ingress (TLS termination) → HTTP → NIM pods (port 8000)

Setup Sequence

  1. Create AKS cluster: .\create-aks-cluster.ps1
  2. Deploy NIM containers: .\deploy-nim.ps1 (requires $env:NGC_API_KEY)
  3. Wait for pods + ingress ready
  4. Switch to local hostname: .\switch-hostname.ps1 -NewHostname nim-aks.local (generates TLS cert, updates ingress, hosts file, SQL objects)
  5. Test endpoints: .\test-embedding.ps1, .\test-chat.ps1
  6. Run SQL scripts: sql\00_setup.sql through sql\04_test_chat_completion.sql

PowerShell Scripts

ScriptPurpose
create-aks-cluster.ps1Create AKS cluster + GPU node pool (2x NC4as_T4_v3)
deploy-nim.ps1Deploy NIM containers + ingress to AKSswitch-hostname.ps1TLS cert generation, K8s secrets, ingress update, hosts file, SQL cert/modeltest-chat.ps1Test chat completion endpoint
test-embedding.ps1Test embedding endpoint
cleanup.ps1Delete AKS cluster and resource group

K8s Manifests

FilePurpose
k8s\nim-chat.yamlDeployment + Service for llama-3.2-3b-instruct NIM
k8s\nim-embedding.yamlDeployment + Service for nv-embedqa-e5-v5 NIM
k8s\ingress.yamlWeb App Routing ingress with TLS for nim-aks.local
k8s\tls.*Self-signed TLS cert files (.crt, .key, .pfx, .cer)

SQL Scripts (run in order)

ScriptPurpose
sql\00_setup.sqlCreate FoundryLocalTest database (skip if exists from foundrylocal demo)
sql\01_create_external_model.sqlCREATE EXTERNAL MODEL → AKS NIM embedding endpoint
sql\02_test_embedding.sqlAI_GENERATE_EMBEDDINGS smoke test (1024-dim vector)
sql\03_enable_rest_endpoint.sqlsp_configure + GRANT (skip if already enabled)
sql\04_test_chat_completion.sqlsp_invoke_external_rest_endpoint → NIM chat

Azure Details (defaults — override on the command line)

  • Subscription: pass with -Subscription <your-subscription-guid> to create-aks-cluster.ps1
  • Region: westus2
  • GPU: 8 vCPUs NCASv3_T4 quota (2x NC4as_T4_v3 nodes)
  • Resource group: rg-nvidianim-westus2
  • Cluster name: aks-nvidianim

Key Facts

  • NIM containers require NGC API key set as $env:NGC_API_KEY
  • NIM images pulled from nvcr.io/nim/ registry
  • Both NIM endpoints are OpenAI API compatible (/v1/chat/completions, /v1/embeddings)
  • Same T-SQL patterns as Foundry Local demo — only URLs change
  • nv-embedqa-e5-v5 produces 1024-dim embeddings (same as qwen3-embedding-0.6b)
  • NIM rejects application/json;charset=utf-8 — NGINX annotation fixes this
  • Llama 3.1 8B OOMs on T4 (16GB); Llama 3.2 3B fits but needs system prompt grounding for quality answers
  • Chat system prompt should include key facts from https://learn.microsoft.com/sql/ to avoid hallucination
  • switch-hostname.ps1 handles full hostname migration (cert, K8s, hosts file, SQL objects)
  • NVIDIA NeMo Guardrails can add content safety, jailbreak prevention, and topic control to chat completions — see https://developer.nvidia.com/nemo-guardrails
  • Same containers and K8s manifests work on Azure Local for edge deployment

Demo Narrative

This is the "cloud version" that proves the Azure Local edge story:

  • Hospital runs SQL Server 2025 + NIM on Azure Local in their data center
  • Patient data never leaves the building — zero cloud API calls
  • Same code, same containers, same T-SQL — different infrastructure underneath

Troubleshooting

  • If pods stuck in ImagePullBackOff → check NGC_API_KEY is correct and K8s secret is created
  • If pods stuck in Pending → GPU node pool may not have scaled up yet (wait ~5-10 min)
  • If ingress has no IP → check Web Application Routing add-on is enabled
  • If SQL returns TLS errors → ensure cert is in Windows Trusted Root and hosts file has correct IP
  • If NIM returns "Unsupported media type" → ingress.yaml needs proxy_set_header Content-Type annotation
  • If chat model gives bad answers → add grounding facts in the system prompt (3B model needs context)
  • If 8B chat model OOMs on T4 → reduce NIM_MAX_MODEL_LEN or use 3B model with system prompt grounding

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.