agentleFS
Sign inStart free
google-ai-edgegoogle-ai-edge/LiteRT/tensor/AGENTS.md

LiteRT / tensor

AGENTS.md3.6k starsChanged 8 days agoApache-2.0View on GitHub

This directory contains the C++ LiteRT Tensor API: a lightweight, backend-agnostic graph construction and execution framework for on-device generative AI and ML models across XNNPACK (CPU), ML Drift (OpenCL/Metal/WebGPU GPU), TFLite FlatBuffer export, and multi-backend ModelChain runners. - GPU tests on Forge (ML Drift over OpenCL): Tag cc_test targets that require an OpenCL GPU with tags = ["requires-gpu-sm90"]. - Large checkpoint tests: Do not check multi-GB .safetensors or .gguf weights into Piper. Keep unit tests in TAP fast and hermetic…

Trending

Quiet since September.about the whole repository

Dec 2025: 1 mentionJan 2026: 5 mentionsFeb 2026: 0 mentionsMar 2026: 1 mentionApr 2026: 1 mentionMay 2026: 0 mentionsJun 2026: 7 mentionsJul 2026: 10 mentionsAug 2026: 0 mentionsSep 2026: 1 mentionOct 2026: 0 mentionsNov 2026: 0 mentions
Dec 2025Jun 2026Nov 2026
Mentions a month of the google-ai-edge/LiteRT repository that our weekly crawl found. There is no adoption data for this file yet.

Who's talking about google-ai-edge/LiteRT

The biggest 5 of 27 mentions linking to the repository that our crawl found. Each line is their own headline.

See all 27

Talked about on GitHub, Hacker News and Bluesky (27 times).

On GitHub · 25

20 more on GitHub

On Hacker News · 1

  • Where files are headingNew AGENTS.md files have more rules about keeping secrets out of the code, and fewer about code style.

    Of the AGENTS.md files we read, the 31,700 first seen in the last six months, against the older ones: the kind of rule that gained most share, and the kind that lost most. About files like this in general, not this one, and an estimate, since a kind is a pattern in the words.

The file

We split this AGENTS.md into the rules it gives an agent

1 rule. About 0 are common, 0 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈0

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    When bringing up a new Hugging Face or GGUF model family under models/<family>/, follow the 5-stage gated workflow and load the …

    line 37

≈ “Common” is an estimate: at least 1 in 5 of the 40,246 AGENTS.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
AGENTS.md

Agent Instructions for //tensor/

<!-- disableFinding(LINE_OVER_80) -->

This directory contains the C++ LiteRT Tensor API: a lightweight, backend-agnostic graph construction and execution framework for on-device generative AI and ML models across XNNPACK (CPU), ML Drift (OpenCL/Metal/WebGPU GPU), TFLite FlatBuffer export, and multi-backend ModelChain runners.

1. Repository Layout

PathPurpose
tensor.h, arithmetic.h, datatypes.h, buffer.hCore Tensor API graph primitives (TensorHandle, Shape, Type, Quantization, arithmetic ops).
ops/Core operator definitions (FullyConnected, BatchMatMul, DepthwiseConv2D, Concatenation, Slice, Reshape, Transpose, Softmax, Select, etc.).
examples/ops/transformer/Tier-1 reusable transformer building blocks (RmsNorm, RoPE, ScaledDotProductAttention, GatedMlp, etc.) templated on backend mixins.
examples/utils/Checkpoint and config helpers (SafetensorLoader, LazyTensorMapping, TensorLoadingHooks, minijson).
examples/gguf/GGUF parser and loader (GgufParser, GgufLoader) for multi-format quantized GGUF checkpoints (Q4_0, Q4_K_M, K-quants, I-quants, Q8_0).
backends/xnnpack/, runners/xnnpack/Direct XNNPACK CPU backend and model runner (XnnpackModelRunner).
backends/ml_drift/, runners/ml_drift/Direct ML Drift GPU backend and compiled OpenCL & Apple Metal model runners (CompiledMlDriftModelRunner, MlDriftClBackend, MlDriftMetalBackend, CompiledMlDriftMetalModelRunner).
backends/tflite/, backends/multi_backend/.tflite FlatBuffer serializer (ModelFactory) and heterogeneous CompositeModel / ModelChain partitioner.
examples/gemma3/, examples/gemma4/Reference Gemma 3 and Gemma 4 C++ graphs and runtimes.
models/common/Model-agnostic bring-up scaffold: checkpoint weight hooks and load-time quantization, declarative WeightMap, HfConfigReader, DecoderLayout / BuildDecoderForwardGraph / RuntimeBackend, the generic TextDecoderRuntime<ModelTraits> (checkpoint discovery, tokenizer, prefill/decode/generate, Benchmark()), XnnpackRuntimeBackend<ModelTraits>, and the reusable TYPED_TEST_P suites in common/testing/.
models/<family>/End-to-end model bring-up packages (e.g., models/qwen3_5/) built on models/common/: config, weight declaration + name predicates, templated C++ graph, DecoderLayoutOf + ModelTraits, RuntimeTraits, GPU backends, HF reference goldens, and the V0–V3 + benchmark test instantiations.
_agents/skills/Modular agent skills for end-to-end model bring-up, verification, quantization, and CPU/GPU performance tuning.

2. Build & Test Conventions

  • Always use blaze-for-agents:
    /google/bin/releases/arca9-local-blaze-cli/blaze-for-agents test -c opt //tensor/...
  • GPU tests on Forge (ML Drift over OpenCL): Tag cc_test targets that require an OpenCL GPU with tags = ["requires-gpu-sm90"].
  • Large checkpoint tests: Do not check multi-GB .safetensors or .gguf weights into Piper. Keep unit tests in TAP fast and hermetic using deterministic LCG synthetic models (test_model.h, CreateSynthetic, <model>_perf_smoke_test) and checked-in small golden files (testdata/*.txt, testdata/*.json). Mark full-checkpoint tests tags = ["manual", "local"] (CPU) or tags = ["manual", "notap", "requires-gpu-sm90"] (GPU via local staging genrules).

3. End-to-End Model Bring-Up Lifecycle (V0 -> V1 -> V2 -> V3 -> Perf)

When bringing up a new Hugging Face or GGUF model family under models/<family>/, follow the 5-stage gated workflow and load the corresponding skill from _agents/skills/ before writing code:

StageGate & ObjectiveSkill to Read First
Stage 1: Intake & Architecture CardParse config.json & checkpoint tensor headers, document the architecture card and weight table in models/<family>/README.md, audit Tier-1 ops vs backend capabilities, and select target precision classes._agents/skills/hf_intake_and_arch_card/SKILL.md
Stage 2: C++ Graph & Runtime AuthoringAuthor <model>_config (HfConfigReader), <model>_weights (DeclareWeights + name predicates over the shared hooks), <model>_graph.h (template <typename BackendMixin>), <model>_forward_pass (DecoderLayoutOf + ModelTraits over BuildDecoderForwardGraph), and <model>_runtime (RuntimeTraits; the runtime itself is common::TextDecoderRuntime)._agents/skills/cpp_graph_authoring/SKILL.md
Stage 3: V0–V2 Golden VerificationVerify V0 per-op/stage parity (<model>_op_parity_ml_drift_test), V1 synthetic LCG + real-weight FP32 parity against transformers (`maxΔ<= 1e-4), and V2 decode-step state handoff (prefill(T) + decode(1) == prefill(T+1)`) on XNNPACK and ML Drift._agents/skills/hf_golden_verification/SKILL.md
Stage 4: V3 Quantization & GGUFEnable compressed-tensors W8A8, load-time int8_channel (--weights=int8) and q4_0 (--weights=q4_0), per-row INT8 tied lm_head (--int8_head), and multi-format GGUF (GgufLoader), verified against dequantized-weight HF goldens._agents/skills/quantize_weights_and_gguf/SKILL.md
Stage 5: CPU & GPU Runner Tuning & BenchmarkingOptimize peak RSS (--low_memory), XNNPACK qd8/qb4w threading, ML Drift zero-copy GPU state binding (ExternalTensorsInfo::immutable_tensors), async I/O, and kF32F16 precision; benchmark pp64/tg32 against llama.cpp, instantiate TextDecoderPerfSmokeTest as <model>_perf_smoke_test, and register the model in the shared harness (agent/scripts/benchmark_tensor_api.py, models/docs/BASELINE.md)._agents/skills/tune_cpu_and_gpu_runners/SKILL.md

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free