agentleFS
JinLee794JinLee794/agent-framework-skills/.cursor/skills/ai-search/SKILL.md

ai-search

Skill2 starsChanged 4 months agoMITView on GitHub

Build and debug code-first Azure AI Search indexing for Foundry agents: load local files, chunk text, generate embeddings through the Foundry project endpoint, create vector and semantic indexes, upload idempotently, and inspect the live service. Use for RAG ingestion, reindexing, schema mismatches, empty indexes, and retrieval checks. NOT for runtime retrieval wiring - load maf-foundry-agent.

  • Reads credentials

5 other projects use a version of this.

Adopted by

Most recently taken up by

The file

Show its rules and text

We split this skill into the rules it gives an agent

15 rules. 0 are common, 9 are distinctive, and 6 are very specific to this project.

  • 0

    Common

    Most of the 3 edited versions we read keep these.

  • 9

    Distinctive

    This author's own choices. Worth a look.

  • 6

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Do not add Blob Storage, ADLS, Cosmos DB, indexers, skillsets, Content Understanding, project connections, managed identities, or a second …

    line 28
  2. Very specific:

    Read deployment coordinates from the repository-root .env.

    line 46
  3. Very specific:

    Do not reuse the chat deployment for embeddings unless it actually is an embedding model.

    line 58
  4. Very specific:

    Call load_dotenv() once at the process entry point, before settings are read.

    line 59
  5. Distinctive:

    Do not delete or rebuild a live index just because a desired schema differs.

    line 69
  6. Very specific:

    Create one embedding for a short non-empty string and derive the dimension from len(response.data[0].embedding).

    line 82
  7. Distinctive:

    Use one Azure AI Search document per chunk.

    line 91
  8. Very specific:

    Use HNSW with cosine distance for Azure OpenAI embeddings.

    line 103
  9. Distinctive:

    every input is non-empty and below the model's per-input token limit,

    line 131
  10. Distinctive:

    no request contains more than the endpoint's input-count limit,

    line 132
  11. Distinctive:

    After Search reports the upload complete:

    line 141
  12. Very specific:

    Only after those checks pass should runtime retrieval be wired through maf-foundry-agent.

    line 149
  13. Distinctive:

    Every uploaded vector length matches the live vector field dimension.

    line 157
  14. Distinctive:

    Every upload result is checked, not merely submitted.

    line 158
  15. Distinctive:

    No credential value or document body appears in diagnostics.

    line 160

“Common” is measured: kept by at least 6 in 10 of the 3 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Azure AI Search Indexing for Foundry Agents

Build a repeatable push pipeline that reads content in Python, chunks it in Python, creates embeddings with the existing Foundry resource, and writes vectors plus readable text directly to the existing Azure AI Search index.

The resource boundary is deliberate:

local or application-readable files
  -> Python loader and token-aware chunker
  -> Foundry embedding deployment
  -> Azure AI Search push API

Do not add Blob Storage, ADLS, Cosmos DB, indexers, skillsets, Content Understanding, project connections, managed identities, or a second model resource. The local MAF agent consumes the finished index directly; this workflow ends at a live, verified search index.

Runtime retrieval belongs to maf-foundry-agent. Agent and voice behaviour belongs in YAML and is owned by maf-agent-config.

Load only the depth you need

TaskReference
Establish packages, .env, credentials, and first connectionreferences/setup.md
Implement chunking, stable keys, embeddings, schema, and uploadsreferences/indexing-pipeline.md
Inspect and query the live index without exposing secretsreferences/live-checks.md

Environment contract

Read deployment coordinates from the repository-root .env. Never print key values.

VariablePurpose
FOUNDRY_PROJECT_ENDPOINTExisting project endpoint, ending in /api/projects/<project>
FOUNDRY_MODELChat deployment used by the reasoning and VoiceLive paths
FOUNDRY_EMBEDDING_MODELEmbedding deployment name, not the model family guessed from it
AZURE_SEARCH_ENDPOINTExisting Azure AI Search service endpoint
AZURE_SEARCH_API_KEYAdmin key in the indexing process; query key in read-only runtime processes
AZURE_SEARCH_INDEX_NAMEThe exact index to inspect, create, populate, and query
AZURE_SEARCH_API_VERSIONPinned Search API version used by the installed SDK

Do not reuse the chat deployment for embeddings unless it actually is an embedding model. Call load_dotenv() once at the process entry point, before settings are read.

Procedure

1. Inspect before mutating

Load .env, construct SearchIndexClient, and call get_index() plus get_index_statistics() for AZURE_SEARCH_INDEX_NAME. Record names, field types, vector dimensions, vector profile, semantic configuration, document count, and vector index size.

Do not delete or rebuild a live index just because a desired schema differs. Classify the difference first:

Live stateAction
Index absentCreate it after the embedding probe
Schema compatibleKeep it and upload
New optional field onlyApply an additive update deliberately
Vector dimensions, key, field type, or analyzer differsStop; create a versioned index and switch intentionally
Existing documents use an unknown key strategyStop; determine ownership before writing

2. Probe the embedding deployment

Create one embedding for a short non-empty string and derive the dimension from len(response.data[0].embedding). The response is the source of truth. Never infer dimensions from a deployment name.

The vector field's vector_search_dimensions must equal that value exactly. The ingestion and query paths must use the same embedding deployment and dimension setting.

3. Define the minimum useful schema

Use one Azure AI Search document per chunk. Keep both readable content and its vector:

FieldShapeWhy it exists
idkey stringStable chunk identity and upserts
parent_idfilterable stringGroups chunks from one source and supports stale-chunk cleanup
titlesearchable stringResult display and semantic title
contentsearchable stringGrounding text, keyword search, and semantic ranking
source_urifilterable stringCitation and source traceability
chunk_indexfilterable integerDeterministic ordering and diagnostics
content_vectorCollection(Edm.Single)Vector similarity, using the probed dimensions

Use HNSW with cosine distance for Azure OpenAI embeddings. Add a semantic configuration whose content field is content and whose title field is title. Keep content_vector out of normal result payloads.

4. Normalize and chunk in code

Give each source a canonical source_uri, title, and normalized text. Preserve paragraph and heading boundaries where possible, then enforce the embedding model's token limit with a token-aware splitter. Make chunk size and overlap checked-in ingestion settings, not secret environment values.

Reject empty chunks. Store enough metadata to trace every chunk back to its source. Do not use character count as a claim about token count.

5. Generate deterministic keys

Derive parent_id from the canonical source identity and id from parent_id plus the chunk ordinal. Use a URL-safe hash. The same corpus and settings must produce the same keys on every run.

An upsert alone does not remove chunks left behind when a document becomes shorter. Compare the expected key set with the live key set for each parent_id, then delete only stale keys.

6. Embed and upload in bounded batches

Batch chunk texts in source order. Preserve the embedding response's item indexes when joining vectors back to chunks. Enforce all three limits before each request:

  • every input is non-empty and below the model's per-input token limit,
  • no request contains more than the endpoint's input-count limit,
  • aggregate tokens stay below the request limit and deployment quota.

Upload with SearchClient.upload_documents(). Treat any returned IndexingResult whose succeeded value is false as a failed run; report its key and error message without dumping document content.

7. Verify the live index

After Search reports the upload complete:

  1. get_index_statistics() reports the expected document count and positive vector storage.
  2. An empty or keyword query returns readable title, content, and source_uri fields.
  3. A vector query built with the same embedding deployment returns a known relevant chunk.
  4. A hybrid plus semantic query returns useful ranking for a corpus-specific question.
  5. A negative query does not produce a misleadingly strong known-answer result.

Only after those checks pass should runtime retrieval be wired through maf-foundry-agent.

Completion criteria

  • The pipeline runs from a clean environment using only .env deployment values and checked-in ingestion settings.
  • Rerunning unchanged input produces the same keys and document count.
  • Removing or shortening a source removes stale chunks from the index.
  • Every uploaded vector length matches the live vector field dimension.
  • Every upload result is checked, not merely submitted.
  • Live keyword, vector, and hybrid checks return readable source metadata.
  • No credential value or document body appears in diagnostics.

Failure signatures

SymptomFirst check
Embedding call returns 404FOUNDRY_EMBEDDING_MODEL is a deployment in FOUNDRY_PROJECT_ENDPOINT
Upload rejects content_vectorCompare the probed vector length with the live field dimensions
Upload returns partial successInspect every IndexingResult; do not rely on the HTTP status alone
Document count grows on every runChunk keys are random or source identities are not canonical
Old passages remain searchableStale keys were not deleted for the affected parent_id
Keyword works but vector search is emptyVectors are missing, dimensions differ, or the vector profile is not attached
Vector results have no usable citationssource_uri, title, or readable content was omitted
Portal works but code failsThe code loaded a different .env, endpoint, index name, or API version

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.