Cross-Resource Model Invocation via Foundry AI Gateway

Status — verified live on 2026-04-23 with Foundry account xtest-foundry-mr5kfi / project xtest-proj-mr5kfi (Sweden Central) calling deployment gpt-4o-mini v2024-07-18 hosted on a different Azure OpenAI account (acme-aoai-shared) through APIM acme-ai-apim. Both ApiKey and ProjectManagedIdentity auth paths returned PONG on all three invocation patterns. See "Verified working configuration" at the end.

Default in worked examples below: gpt-5.4-mini (current Foundry-routable chat-mini family, May 2026). The original verification was on gpt-4o-mini; the gateway routes by deployment-name-in-path regardless of model family, so the recipe is mechanically identical for any deployment your APIM backend actually carries. The exact API version string in the connection metadata below is illustrative — use whatever your backend deployment is registered with (check az cognitiveservices account deployment show).

1. What this skill solves

You have a Foundry project ("consumer") that needs to invoke models deployed on a different Azure OpenAI / AI Services account ("backend"), fronted by an Azure API Management instance acting as the AI Gateway. You want to address those models via the Foundry-native string connectionName/deploymentName so the application code is identical to a local-deployment call.

┌────────────────────────┐                ┌──────────────────────────┐                ┌─────────────────────────────┐
│  Consumer Foundry      │                │  APIM AI Gateway         │                │  Backend AI/OpenAI account  │
│  project               │  Foundry MI    │  acme-ai-apim        │   APIM MI →    │  acme-aoai-shared       │
│  xtest-proj-mr5kfi     ├───or────────► │   /xtest-aoai            ├──Bearer───────►│  gpt-5.4-mini deployment    │
│  (Sweden Central)      │  ApiKey        │   /xtest-aoai-pmi        │  (msi token)   │  (East US 2)                │
│                        │                │                          │                │                             │
│  ApiManagement         │                │  validate token /        │                │  Cognitive Services User RBAC│
│  connection            │                │  enforce subscription    │                │  granted to APIM MI         │
│  category=ApiManagement│                │  set-backend-service     │                │                             │
│  target=APIM API URL   │                │  authentication-managed- │                │                             │
│  authType=ApiKey | PMI │                │   identity → backend     │                │                             │
└────────────────────────┘                └──────────────────────────┘                └─────────────────────────────┘

The consumer project never holds the backend's API key or RBAC. APIM is the trust boundary.

2. The 100% reliable checklist

Before any agent call, ALL of these must be true. Cross them off in order; the official validator script test_apim_connection.py plus the field-tested probes in §8 below verify each step end-to-end.

If step 4 doesn't return 200 outside Foundry, no Foundry call will work. Always smoke-test the gateway directly first.

3. Decision tree

                          ┌──────────────────────────────────────┐
                          │ What backend does APIM forward to?   │
                          └─────────────┬────────────────────────┘
                                        │
              ┌─────────────────────────┴────────────────────────┐
              │                                                  │
   AOAI / AI Services on /openai                       OpenAI v1 (/v1/...)  
   (most common — Azure OpenAI)                        or Anthropic etc.    
              │                                                  │
              ▼                                                  ▼
   metadata.deploymentInPath = "true"           metadata.deploymentInPath = "false"
   metadata.inferenceAPIVersion = "2024-10-21"  metadata.inferenceAPIVersion = ""  (or omit)
   models[].properties.model.format = "OpenAI"  models[].properties.model.format = "OpenAI"|"Anthropic"|"NonOpenAI"
                                                modelDiscovery.listModelsEndpoint = "/models"
                                                modelDiscovery.deploymentProvider = "AzureOpenAI"|"OpenAI"|"Anthropic"|"NonOpenAI"

                          ┌──────────────────────────────────────┐
                          │ How do callers prove identity to APIM?│
                          └─────────────┬────────────────────────┘
                                        │
              ┌─────────────────────────┼────────────────────────┐
              │                         │                        │
        APIM subscription key   Project Managed Identity    Both (dual-auth)
              │                  (no static secrets)               │
              ▼                         ▼                        ▼
   authType: "ApiKey"           authType: "ProjectManagedIdentity"  Two connections OR
   credentials.key: <key>       credentials: {}                     APIM <choose><when>
   APIM api: subscription req   audience:                           policy that branches
   APIM policy: pass-through      "https://cognitiveservices.azure.com"
                                APIM policy: validate-azure-ad-token
                                  with <client-application-ids>
                                  containing project MI clientId
                                APIM api: subscriptionRequired=false

4. APIM-side configuration

4.1 Service URL on the API

The Foundry connection target is https://<apim>.azure-api.net/<apiPath>. Inside the API's inbound policy you set-backend-service to the backend account. Foundry then appends /deployments/{dep}/chat/completions?api-version=... (or /v1/responses etc.) to the gateway URL.

Operations on the API (i.e., the routes Foundry will call) must therefore match the AOAI / OpenAI surface. The simplest setup is catch-all with one operation per HTTP verb on the wildcard path /{*path}. For Azure OpenAI backends, all of the following must reach the inbound policy and 200:

POST /deployments/{deployment-id}/chat/completions?api-version=2024-10-21
POST /deployments/{deployment-id}/embeddings?api-version=2024-10-21
POST /v1/responses?api-version=preview          (Responses API)
GET  /models?api-version=2024-10-21             (model catalogue, used for dynamic discovery)

4.2 ApiKey-only inbound policy (verified working — xtest-aoai)

<policies>
    <inbound>
        <base />
        <set-backend-service base-url="https://acme-aoai-shared.openai.azure.com/openai" />
        <authentication-managed-identity
            resource="https://cognitiveservices.azure.com"
            output-token-variable-name="msi-access-token"
            ignore-error="false" />
        <set-header name="Authorization" exists-action="override">
            <value>@("Bearer " + (string)context.Variables["msi-access-token"])</value>
        </set-header>
    </inbound>
    <backend><base /></backend>
    <outbound><base /></outbound>
    <on-error><base /></on-error>
</policies>

Subscription enforcement is at the API/product level (set subscriptionRequired: true and configure subscriptionKeyParameterNames.header to api-key so the caller sends api-key: <key>, not Ocp-Apim-Subscription-Key). Foundry sends the api-key header.

4.3 PMI-only inbound policy (verified working — xtest-aoai-pmi)

<policies>
    <inbound>
        <base />
        <validate-azure-ad-token
            tenant-id="<your-tenant-id>"
            header-name="Authorization"
            failed-validation-httpcode="401"
            failed-validation-error-message="Unauthorized: token did not match expected audience or application">
            <client-application-ids>
                <application-id><consumer-project-MI-clientId></application-id>
            </client-application-ids>
            <audiences>
                <audience>https://cognitiveservices.azure.com</audience>
                <audience>https://cognitiveservices.azure.com/</audience>
            </audiences>
        </validate-azure-ad-token>
        <set-backend-service base-url="https://acme-aoai-shared.openai.azure.com/openai" />
        <authentication-managed-identity
            resource="https://cognitiveservices.azure.com"
            output-token-variable-name="msi-access-token"
            ignore-error="false" />
        <set-header name="Authorization" exists-action="override">
            <value>@("Bearer " + (string)context.Variables["msi-access-token"])</value>
        </set-header>
    </inbound>
    <backend><base /></backend>
    <outbound><base /></outbound>
    <on-error><base /></on-error>
</policies>

PMI API also needs subscriptionRequired: false on the API resource (otherwise APIM rejects with 401 before the policy runs).

Why <client-application-ids> and not <required-claims> with xms_mirid? Both mechanisms work in principle, but the xms_mirid value Foundry's project MI puts in its token is not the project ARM ID — it varies by resource provider and current APIs. The MI's appid claim is universally present and stable, so <client-application-ids> is the documented and most reliable check. (Verified 2026-04-23: a policy requiring xms_mirid = <project ARM ID> returned 401; switching to <client-application-ids> returned 200.)

4.4 Dual-auth inbound policy (ApiKey OR PMI)

<policies>
    <inbound>
        <base />
        <choose>
            <when condition="@(context.Subscription == null)">
                <validate-azure-ad-token
                    tenant-id="<your-tenant-id>"
                    header-name="Authorization"
                    failed-validation-httpcode="401">
                    <client-application-ids>
                        <application-id><consumer-project-MI-clientId></application-id>
                    </client-application-ids>
                    <audiences>
                        <audience>https://cognitiveservices.azure.com</audience>
                        <audience>https://cognitiveservices.azure.com/</audience>
                    </audiences>
                </validate-azure-ad-token>
            </when>
        </choose>
        <set-backend-service base-url="https://<backend>.openai.azure.com/openai" />
        <authentication-managed-identity
            resource="https://cognitiveservices.azure.com"
            output-token-variable-name="msi-access-token" />
        <set-header name="Authorization" exists-action="override">
            <value>@("Bearer " + (string)context.Variables["msi-access-token"])</value>
        </set-header>
    </inbound>
</policies>

API must have subscriptionRequired: false (so PMI calls aren't rejected up front); the <choose> branch enforces token validation only when no subscription was used.

5. Connection schema (verified live)

ARM type: Microsoft.CognitiveServices/accounts/projects/connections@2025-04-01-preview. Reachable also via the Foundry data-plane endpoint https://<account>.services.ai.azure.com/api/projects/<project>/connections/<name>?api-version=v1.

⚠ The metadata.models and metadata.modelDiscovery fields are JSON-encoded strings, not real JSON objects. This is because Azure connection metadata is a flat string→string dictionary. Pass the JSON as a string value or your PUT will be silently ignored / rejected.

5.1 ApiKey connection — verified working PUT body

{
  "properties": {
    "category": "ApiManagement",
    "target": "https://acme-ai-apim.azure-api.net/xtest-aoai",
    "authType": "ApiKey",
    "credentials": { "key": "<APIM subscription primary key>" },
    "isSharedToAll": true,
    "metadata": {
      "deploymentInPath": "true",
      "inferenceAPIVersion": "2024-10-21",
      "models": "[{\"name\":\"gpt-5.4-mini\",\"properties\":{\"model\":{\"name\":\"gpt-5.4-mini\",\"format\":\"OpenAI\",\"version\":\"2026-04-30\",\"publisher\":\"Microsoft\"}}}]"
    }
  }
}

PUT URL:

https://management.azure.com/subscriptions/<sub>/resourceGroups/<rg>/providers/
Microsoft.CognitiveServices/accounts/<consumer-acct>/projects/<consumer-proj>/
connections/<connectionName>?api-version=2025-04-01-preview

Or via the data plane:

PUT https://<consumer-acct>.services.ai.azure.com/api/projects/<consumer-proj>/connections/<name>?api-version=v1

5.2 PMI connection — verified working PUT body

{
  "properties": {
    "category": "ApiManagement",
    "target": "https://acme-ai-apim.azure-api.net/xtest-aoai-pmi",
    "authType": "ProjectManagedIdentity",
    "credentials": {},
    "audience": "https://cognitiveservices.azure.com",
    "isSharedToAll": true,
    "metadata": {
      "deploymentInPath": "true",
      "inferenceAPIVersion": "2024-10-21",
      "models": "[{\"name\":\"gpt-5.4-mini\",\"properties\":{\"model\":{\"name\":\"gpt-5.4-mini\",\"format\":\"OpenAI\",\"version\":\"2026-04-30\",\"publisher\":\"Microsoft\"}}}]"
    }
  }
}

audience lives at properties.audience, NOT in metadata. The data-plane GET strips it from the response, but it is required on PUT and is what Foundry uses when requesting the MI token.

5.3 Metadata reference (full)

models[].properties.model.format accepts OpenAI, Anthropic, NonOpenAI. For Azure OpenAI deployments use OpenAI.

5.4 What does NOT work (verified failures)

authType decision tree (May 2026):

• Responses API, public endpoint (oai.responses.create(), Patterns A/B/C) → use ProjectManagedIdentity (preferred) or ApiKey
• Responses API, private VNet standard agents → use ApiKey with APIM subscription key (AAD is internally mapped to PMI which is rejected in private VNet standard-agent setups)
• Standard Agents / Assistants v1 (/assistants, /threads/<id>/runs) → use AAD (PMI is rejected); note the runtime itself fails server-side with APIM connections regardless of metadata correctness — this is a platform limitation, v1 retires May 22 2026

6. Setup paths

6.1 Bicep (recommended for production)

Use the official module from microsoft-foundry/foundry-samples/infrastructure/infrastructure-setup-bicep/01-connections/apim/. Files of interest:

• connection-apim.bicep — top-level module
• modules/apim-connection-common.bicep — actual connections resource declaration
• samples/parameters-static-models.json — ApiKey + static models
• samples/parameters-dynamic-discovery.json — dynamic via /models
• samples/parameters-custom-headers.json
• samples/parameters-custom-auth-config.json

When using Bicep, you write models as a real Bicep array; the module JSON-stringifies it before assigning to metadata.models. Check apim-connection-common.bicep to confirm — do NOT hand-stringify if you go through Bicep, only if you call ARM/REST directly.

6.2 PowerShell REST (operational scripts)

$tenant = '<tenant-id>'
$proj   = 'https://<consumer-acct>.services.ai.azure.com/api/projects/<consumer-proj>'
$apim   = 'https://acme-ai-apim.azure-api.net/xtest-aoai'

# Token for Foundry data plane (use the AI resource audience)
$tok = az account get-access-token --resource 'https://ai.azure.com' --query accessToken -o tsv

$models = @(@{
    name = 'gpt-5.4-mini'
    properties = @{
      model = @{ name='gpt-5.4-mini'; format='OpenAI'; version='2026-04-30'; publisher='Microsoft' }
    }
}) | ConvertTo-Json -Depth 6 -Compress

$body = @{
  properties = @{
    category      = 'ApiManagement'
    target        = $apim
    authType      = 'ApiKey'
    credentials   = @{ key = '<APIM subscription primary key>' }
    isSharedToAll = $true
    metadata      = @{
      deploymentInPath    = 'true'
      inferenceAPIVersion = '2024-10-21'
      models              = $models    # already a JSON string thanks to ConvertTo-Json
    }
  }
} | ConvertTo-Json -Depth 8

Invoke-RestMethod -Method Put `
  -Uri "$proj/connections/aigw-strmeta?api-version=v1" `
  -Headers @{ Authorization = "Bearer $tok"; 'Content-Type' = 'application/json' } `
  -Body $body

For PMI: replace authType with 'ProjectManagedIdentity', set credentials = @{}, and add audience = 'https://cognitiveservices.azure.com' inside properties.

6.3 Foundry Portal (manual / quick check)

Operate → Admin console → All projects → click parent AI Services account → Admin-connected models tab → Add → choose API Management → wizard fills:

• Display name (becomes connection name)
• Source = APIM service (drop-down lists APIMs in the same tenant)
• API = drop-down of APIs on that APIM
• Auth = ApiKey or PMI
• Discovery = Static models or Auto-discover (calls listModelsEndpoint)
• For Static: enter model name(s), pick format

Portal also writes the connection with stringified metadata under the hood.

7. Invocation patterns (Python — azure-ai-projects >= 2.0.0, openai >= 2.0.0)

All three patterns succeed against both ApiKey and PMI connections (verified 2026-04-23). The model string is always <connectionName>/<deploymentName>, where <deploymentName> is the name field inside metadata.models[] (NOT the backend's actual deployment id, unless they happen to match).

7.1 Pattern A — Direct Responses API (PRIMARY)

from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

project = AIProjectClient(
    endpoint="https://<consumer-acct>.services.ai.azure.com/api/projects/<consumer-proj>",
    credential=DefaultAzureCredential(),
)
oai = project.get_openai_client()

resp = oai.responses.create(
    model="aigw-strmeta/gpt-5.4-mini",       # connection-name/model-alias
    input="Say PONG.",
    max_output_tokens=20,                    # MUST be >= 16 to fit any non-trivial reply
)
print(resp.output_text)

7.2 Pattern B — Prompt agent + agent_reference

from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import PromptAgentDefinition
from azure.identity import DefaultAzureCredential

project = AIProjectClient(
    endpoint="https://<consumer-acct>.services.ai.azure.com/api/projects/<consumer-proj>",
    credential=DefaultAzureCredential(),
)
oai = project.get_openai_client()

agent = project.agents.create_version(
    agent_name="cross-resource-helper",
    definition=PromptAgentDefinition(
        model="aigw-strmeta/gpt-5.4-mini",
        instructions="You only ever reply with the word PONG.",
    ),
)

conv = oai.conversations.create()
resp = oai.responses.create(
    input="Say it.",
    conversation=conv.id,
    extra_body={"agent_reference": {"name": agent.name, "type": "agent_reference"}},
)
print(resp.output_text)

7.3 Pattern C — Refreshed-preview hosted-agent client

from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

project = AIProjectClient(
    endpoint="https://<consumer-acct>.services.ai.azure.com/api/projects/<consumer-proj>",
    credential=DefaultAzureCredential(),
    allow_preview=True,            # MUST be on the constructor, NOT on get_openai_client
)
oai = project.get_openai_client(agent_name="cross-resource-helper")
resp = oai.responses.create(input="Reply with PONG only.")
print(resp.output_text)

REST equivalent (refreshed preview): include header Foundry-Features: HostedAgents=V1Preview on the call, per learn.microsoft.com/azure/foundry/agents/how-to/migrate-hosted-agent-preview.

7.4 Negative patterns — DO NOT work

# 404 DeploymentNotFound — Foundry only routes the conn/dep alias on /v1/responses,
# never on the legacy /chat/completions endpoint.
oai.chat.completions.create(model="aigw-strmeta/gpt-5.4-mini", messages=[...])

If you have legacy code that must use chat.completions, point it at the backend resource directly (with proper RBAC) — not at the Foundry project's OpenAI surface.

Assistants v1 runtime (/threads/<id>/runs) also fails. Creating an assistant with model: "conn/dep" succeeds (the string is stored), but POST /threads/<id>/runs returns server_error with prompt_tokens: 0 and no upstream APIM traffic. The standard-agent runtime does not resolve <connection>/<deployment> model strings through APIM. Reproduced 2026-05-19 on a private VNet topology (Switzerland North + Sweden Central) with correct connection metadata (models, deploymentInPath, inferenceAPIVersion all present). This is a platform-side limitation, not a configuration issue. If you need the Assistants v1 API shape with cross-resource models, file a Microsoft support case.

7.5 Hosted Agents container (MODEL_DEPLOYMENT_NAME)

For Foundry Hosted Agents (MAF + ResponsesHostServer, or GHCP SDK InvocationAgentServerHost), set MODEL_DEPLOYMENT_NAME in the agent's container env to the same connectionName/deploymentName string. Remove any config.deployments / model-deployment block from azure.yaml — the runtime dispatches via the connection. See foundry-hosted-agents and ghcp-hosted-agents skills.

8. Validation

8.1 Pre-Foundry: smoke-test APIM directly

$apim = 'https://acme-ai-apim.azure-api.net/xtest-aoai'
$key  = '<subscription primary key>'

Invoke-RestMethod -Method Post `
  -Uri "$apim/deployments/gpt-5.4-mini/chat/completions?api-version=2024-10-21" `
  -Headers @{ 'api-key' = $key; 'Content-Type' = 'application/json' } `
  -Body (@{ messages=@(@{role='user';content='Say PONG.'}); max_tokens=20 } | ConvertTo-Json -Depth 5)

Expected: HTTP 200 with choices[0].message.content. If this fails, fix APIM before touching Foundry.

8.2 Foundry probe: test_apim_connection.py

microsoft-foundry/foundry-samples/.../apim/test_apim_connection.py is the official validator — it issues a probe /models (or /deployments) request and surfaces auth / discovery / target reachability problems before agent calls.

8.3 Field-tested probes

Stand up two minimal probe scripts under tests/ in your consumer project — one per auth mode — that exercise all three invocation patterns plus the negative chat.completions case for ApiKey. Pattern shape:

# tests/probe_apim_apikey.py  (and probe_apim_pmi.py — same shape, different connection)
import os
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

CONNECTION = os.environ["APIM_CONNECTION_NAME"]   # aigw-strmeta or aigw-pmi
DEPLOYMENT = os.environ["APIM_DEPLOYMENT_NAME"]   # e.g. gpt-5.4-mini

project = AIProjectClient(
    endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
    credential=DefaultAzureCredential(),
    allow_preview=True,
)
oai = project.inference.get_openai_client(api_version="v1")

# Pattern A — Responses
r = oai.responses.create(model=f"{CONNECTION}/{DEPLOYMENT}",
                         input="Say PONG", max_output_tokens=16)
assert r.output_text, "Pattern A returned empty output"

# Pattern B/C — see §6 for full hosted-agent invocation shapes

Run them as a regression gate when you provision new APIM/Foundry combinations.

9. Troubleshooting matrix (reproduced root causes)

10. Network isolation

For private VNet deployments (private endpoint on AI Services + private endpoint on APIM + DNS), see template 16-private-network-standard-agent-apim-setup-preview. Conceptually identical to the public path — the connection target becomes https://<apim-private-host>/<api-path>; Foundry resolves it through the project's outbound private DNS zone.

Agent subnet CIDR restriction (verified 2026-05-19). In some regions (observed in Switzerland North; may apply elsewhere), the Foundry RP rejects networkInjections with 10.x.x.x subnets: "Provided subnet must be of the proper address space. Please provide a subnet which has address space in the range of 172 or 192." Use 172.16.x.x/xx or 192.168.x.x/xx CIDRs for the agent VNet and its subnets in those regions. West Europe and East US 2 accept 10.x ranges.

11. Verified working configuration shape

This section captures the resource shape used in live verification of this skill (subscriptions, principal IDs, and resource names redacted — the recipe is reproducible from the placeholders).

Model currency note. Verification ran on gpt-4o-mini (April 2026). Worked examples in this document default to gpt-5.4-mini (current Foundry-routable chat-mini family, May 2026) for currency, NOT because gpt-5.4-mini was independently re-verified end-to-end against this APIM topology — only that the gateway routes by deployment-name-in-path so the recipe is mechanically identical across model families. When you adopt this skill for a new pilot, verify with the actual deployment name + version in your APIM backend (az cognitiveservices account deployment show); the version field shown in the connection-metadata JSON is illustrative. gpt-4o family is legacy and reaches end-of-support per the Azure OpenAI lifecycle calendar — see foundry-doc-vision-speech § "GPT-4o is LEGACY".

SDK versions used: azure-ai-projects 2.1.0, openai 2.36.0, azure-identity 1.x. Compatible with azure-ai-projects >= 2.0.0.

12. References

Authoritative sources backing every claim in this skill:

• MS Learn — AI Gateway connection (Foundry agents)
• MS Learn — Refreshed hosted agent preview
• GitHub microsoft-foundry/foundry-samples —
  • connection-apim.bicep
  • apim-connection-common.bicep
  • APIM-Connection-Objects.md
  • apim-setup-guide-for-agents.md
  • troubleshooting-guide.md
  • test_apim_connection.py
  • 16-private-network-standard-agent-apim-setup-preview/
• ARM — Microsoft.CognitiveServices/accounts/projects/connections@2025-04-01-preview
• APIM — validate-azure-ad-token policy reference
• APIM — authentication-managed-identity policy reference
• Related skills — foundry-hosted-agents, ghcp-hosted-agents, azure-tenant-isolation, azd-patterns, foundry-mcp-aca