add-onnx-op-lowering
Skill1.1k starsChanged 12 days agoApache-2.0View on GitHub
Use when writing or changing the ONNXToKrnl lowering of an ONNX op that is already in the dialect: the conversion pattern class, registration in ConvertONNXToKrnl.cpp, IndexExpr shape helpers, SIMD/parallel gating, and lit tests.
Trending
Quiet since December.about the whole repository
Who's talking about onnx/onnx-mlir
- elixir-nx/nx“Import of MLIR modules”GitHub · Mar 5, 2024 · 4 reactions
- PyTorchKR/pytorchcore-kr“더 나은 질의 응답 구조를 위한 Discussion 활용 방안”GitHub · Jan 20, 2025 · 2 reactions
- llvm/torch-mlir“[RFC] Support for an embedded build of Torch-MLIR”GitHub · Sep 23, 2022 · 2 reactions
- Samsung/ONE“[circle-mlir] Update onnx-mlir to official release”GitHub · Jun 8, 2025 · 1 reactions
- Xilinx/onnx-mlir“ONNX to TOSA Legalization Upstream tracker”GitHub · Apr 20, 2023 · 1 reactions
The biggest 5 of 22 mentions linking to the repository that our crawl found. Each line is their own headline.
See all 22
Talked about on GitHub (22 times).
On GitHub · 22
- Mar 5, 2024Import of MLIR modules — elixir-nx/nx4 reactions · 2 comments
- Jan 20, 2025더 나은 질의 응답 구조를 위한 Discussion 활용 방안 — PyTorchKR/pytorchcore-kr2 reactions
- Sep 23, 2022[RFC] Support for an embedded build of Torch-MLIR — llvm/torch-mlir2 reactions · 4 comments
- Jun 8, 2025[circle-mlir] Update onnx-mlir to official release — Samsung/ONE1 reaction · 4 comments
- Apr 20, 2023ONNX to TOSA Legalization Upstream tracker — Xilinx/onnx-mlir1 reaction
17 more on GitHub
- Dec 16, 2025[ttl] Automate elementwise op generation from specifications — tenstorrent/tt-lang0 reactions · 1 comment
- Jul 29, 2025Missing Symbols due to visibility setting change — onnx/onnx0 reactions · 2 comments
- Nov 19, 2024Add fancier CI badges showing number of tests and/or test coverage — iree-org/iree-test-suites0 reactions · 2 comments
- Aug 6, 2024Standardize rewriter tests; testing facilities — microsoft/onnxscriptsays it tested it0 reactions · 1 comment
- May 8, 2024Fix macos build by pinning version — Xilinx/onnx-mlir0 reactions
- Mar 30, 2024Include the ".onnxtext" extension in supported serialization format — onnx/onnx0 reactions · 1 comment
- Dec 20, 2023add filecheck test for add.onnx — xdslproject/xdsl0 reactions · 7 comments
- Dec 1, 2023Failed to build with the use of --cpp-implementation — protocolbuffers/protobuf0 reactions
- Oct 27, 2023Bring in the fix for empty slice in canonicalization — Xilinx/onnx-mlir0 reactions
- Oct 25, 2023Cherry-picked upstream PR: Add conversion for DivOp ONNX to TOSA (#2207) — Xilinx/onnx-mlir0 reactions
- Aug 26, 2023Build error on M1 MacOS machine — onnx/onnx0 reactions · 3 comments
- Oct 18, 2022ONNX MLIR exporter — nod-ai/onnx-mlir-exporter0 reactions · 1 comment
- Oct 9, 2022[Feature request] Machine Readable definition of Operations (Schema file instead of Markdown or cc file) — onnx/onnx0 reactions · 14 comments
- Jun 28, 2022Can we have newer opset of bidaf-9 model (including opset 9 or newer onnx.Scan op)? — onnx/models0 reactions · 1 comment
- Jun 5, 2022Broken Links — onnx/onnx.github.io0 reactions · 3 comments
- May 12, 2022Update the onnx-mlir version — DependableSystemsLab/LLTFI0 reactions · 2 comments
- Jun 30, 2020Error when trying to lower alex net onnx model to MLIR — onnx/models0 reactions · 1 comment
In this repository14 other files it gives its agents.
- AGENTS.mdAGENTS.md
- add-fusion-patternSkill
- add-new-onnx-operationSkill
- add-onnx-op-to-dialectSkill
- add-onnx-transform-passSkill
- coding-conventionsSkill
- debug-compilationSkill
- debug-type-catalogueSkill
- ground-lit-testsSkill
- nnpa-internalsSkill
- nnpa-performanceSkill
- onnx-mlir-internalsSkill
- upgrade-llvm-mlirSkill
- using-torch-onnxmlir-with-huggingfaceSkill
The file
We split this skill into the rules it gives an agent
9 rules. About 0 are common, 1 is distinctive, and 8 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
8
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 27
Check that {{OP_NAME}} appears in ONNXOps.td.inc.
- Distinctive: line 33
If absent, the op is not in the dialect.
- Very specific: line 39
Create src/Conversion/ONNXToKrnl/{{CATEGORY}}/{{OP_FILE}}.
- Very specific: line 95
If the op has no shape helper yet, add one in src/Dialect/ONNX/ONNXOps/ShapeHelper.hpp first (Col2Im, for example, uses using …
- Very specific: line 111
Use the collapse-eligible constructor KrnlParallelPlan(loopDef, enableCollapse, ...) only for a plain nest of original loops with zero …
- Very specific: line 122
Call it from populateONNXToKrnlConversionPattern() in ConvertONNXToKrnl.cpp:
- Very specific: line 126
Add the source file to src/Conversion/ONNXToKrnl/CMakeLists.txt (e.g.
- Very specific: line 179
Use ONNXBroadcastOpShapeHelper (ShapeHelper.hpp): computeShapeAndAssertOnFailure() gives the output dims, and inside the loop …
- Very specific: line 195
How to test: Run make check-onnx-mlir or use RunONNXModel.py --verify ort
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
<!-- Generated from the onnx-mlir knowledge base (skills/add-onnx-op-lowering.md), verified against onnx-mlir@b505d64b. Do not edit here: see .agents/skills/README.md. -->Add an ONNXToKrnl Lowering
Recipe for adding a new ONNX op lowering to the ONNXToKrnl conversion pass.
Parameters
| Parameter | Description | Example |
{{OP_NAME}} | ONNX op name (CamelCase) | Elu |
{{CATEGORY}} | Conversion category directory | Math |
{{OP_FILE}} | Source file for the lowering | Elementwise.cpp |
{{OPSET_VER}} | Opset version to support | 6 |
Step 1: Verify the op is in the ONNX dialect
Check that {{OP_NAME}} appears in ONNXOps.td.inc. If not, add it via gen_onnx_mlir.py first (see add-onnx-op-to-dialect.md).
grep "ONNX{{OP_NAME}}Op" src/Dialect/ONNX/ONNXOps.td.inc | head -5If absent, the op is not in the dialect. Add it before continuing.
Step 2: Add the lowering class
Create src/Conversion/ONNXToKrnl/{{CATEGORY}}/{{OP_FILE}}. Current lowerings are typed OpConversionPattern<ONNX{{OP_NAME}}Op> with a generated adaptor, take output dims from the op's shape helper, and allocate with MemRefBuilder::alignedAlloc. The smallest real template is Tensor/EyeLike.cpp:
#include "src/Conversion/ONNXToKrnl/ONNXToKrnlCommon.hpp"
#include "src/Dialect/ONNX/ONNXOps/ShapeHelper.hpp"
using namespace mlir;
namespace onnx_mlir {
struct ONNX{{OP_NAME}}OpLowering : public OpConversionPattern<ONNX{{OP_NAME}}Op> {
ONNX{{OP_NAME}}OpLowering(TypeConverter &typeConverter, MLIRContext *ctx)
: OpConversionPattern(typeConverter, ctx) {}
LogicalResult matchAndRewrite(ONNX{{OP_NAME}}Op onnxOp,
ONNX{{OP_NAME}}OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const final {
Operation *op = onnxOp.getOperation();
Location loc = ONNXLoc<ONNX{{OP_NAME}}Op>(op);
MultiDialectBuilder<KrnlBuilder, IndexExprBuilderForKrnl, MemRefBuilder,
MathBuilder> create(rewriter, loc);
// 1. Output type and dims (shape helper computes them as IndexExprs).
MemRefType memRefType = mlir::cast<MemRefType>(
typeConverter->convertType(*op->result_type_begin()));
ONNX{{OP_NAME}}OpShapeHelper shapeHelper(op, adaptor.getOperands(), &create.krnlIE);
shapeHelper.computeShapeAndAssertOnFailure();
DimsExpr &outputDims = shapeHelper.getOutputDims();
// 2. Allocate output.
Value alloc = create.mem.alignedAlloc(memRefType, outputDims);
// 3. Loop nest over the output.
int64_t rank = memRefType.getRank();
ValueRange loopDef = create.krnl.defineLoops(rank);
SmallVector<IndexExpr, 4> lbs(rank, LitIE(0));
create.krnl.iterateIE(loopDef, loopDef, lbs, outputDims,
[&](const KrnlBuilder &createKrnl, ValueRange loopInd) {
MultiDialectBuilder<KrnlBuilder, MathBuilder> create(createKrnl);
Value x = create.krnl.load(adaptor.getInput(), loopInd); // adjust per op
Value result = /* computation using create.math.* */;
create.krnl.store(result, alloc, loopInd);
});
rewriter.replaceOp(op, alloc);
return success();
}
};
void populateLoweringONNX{{OP_NAME}}OpPattern(RewritePatternSet &patterns,
TypeConverter &typeConverter, MLIRContext *ctx) {
patterns.insert<ONNX{{OP_NAME}}OpLowering>(typeConverter, ctx);
}
} // namespace onnx_mlirIf the op has no shape helper yet, add one in src/Dialect/ONNX/ONNXOps/ShapeHelper.hpp first (Col2Im, for example, uses using ONNXCol2ImOpShapeHelper = ONNXNonSpecificOpShapeHelper<mlir::ONNXCol2ImOp>;).
An op can also be lowered by expanding it into other ONNX ops with MultiDialectBuilder<OnnxBuilder>. The conversion legalizes those ops in the same run, so no Krnl code is needed. Math/Attention.cpp works this way: its pattern only calls lowerONNXAttentionOp() in src/Dialect/ONNX/Transforms/AttentionToONNXOps.cpp, which takes a plain PatternRewriter so that the NNPA path can reuse the same expansion on tensors (interfaces/onnx-to-krnl.md). Put a shared expansion there too if another backend needs it, and add OMONNXRewrite to the OMONNXToKrnl link libraries, as Attention does. When FileCheck should see the expansion before it is lowered, add the emitted ops to the emit-intermediate-ir legal list in ConvertONNXToKrnl.cpp.
Optional: SIMD and parallel
To support enableSIMD/enableParallel, add the bools to the constructor and the populate function. Gate parallelism per op with OnnxToKrnlLoweringConfiguration::enableSpecificParallelOps.isEnabled(ONNX{{OP_NAME}}Op::getOperationName()), as NonZero, Col2Im, QLinearMatMul, Det and TensorScatter do. Emit the parallel loop through a KrnlParallelPlan, and pass plan.optimizedLoopDef() (not loopDef) as the iterate's optimized loops:
auto plan = KrnlParallelPlan::noCollapse(loopDef, /*parFirstInclusiveDim=*/0,
/*parLastExclusiveDim=*/2, {.minTripCountForParallel = 16});
if (enableParallel)
plan.tryCreateParallel(create.krnl, op, "my op outer loop", lbs, outputDims);
create.krnl.iterateIE(loopDef, plan.optimizedLoopDef(), lbs, outputDims, ...);Use the collapse-eligible constructor KrnlParallelPlan(loopDef, enableCollapse, ...) only for a plain nest of original loops with zero lower bounds and no iterArgs. In that case also thread enableCollapse through the populate function. See interfaces/onnx-to-krnl.md.
Step 3: Declare and register the pattern
- Declare the populate function in
ONNXToKrnlCommon.hppnext to its category:void populateLoweringONNX{{OP_NAME}}OpPattern( mlir::RewritePatternSet &, mlir::TypeConverter &, mlir::MLIRContext *); - Call it from
populateONNXToKrnlConversionPattern()inConvertONNXToKrnl.cpp:populateLoweringONNX{{OP_NAME}}OpPattern(patterns, typeConverter, ctx); - Add the source file to
src/Conversion/ONNXToKrnl/CMakeLists.txt(e.g.Tensor/EyeLike.cpp).
Step 4: Conversion target (usually nothing to do)
Do not add target.addIllegalOp<ONNX{{OP_NAME}}Op>(). The pass uses applyPartialConversion, and the ONNX dialect is not declared illegal (addIllegalDialect<ONNXOpsDialect> is commented out). Ops with a matching pattern are converted, and ops without one are left in place and fail later in the pipeline. If the op was listed in src/Dialect/ONNX/ONNXUnsupportedOps.hpp, remove it from there (the Col2Im and Det lowerings did).
Step 5: Build and test
# Build
cd build && make -j$(nproc) onnx-mlir onnx-mlir-opt
# Test with onnx-mlir-opt
echo '
func.func @test(%arg0: tensor<4xf32>) -> tensor<4xf32> {
%0 = "onnx.{{OP_NAME}}"(%arg0) : (tensor<4xf32>) -> tensor<4xf32>
return %0 : tensor<4xf32>
}
' > /tmp/test.mlir
onnx-mlir-opt --shape-inference --convert-onnx-to-krnl /tmp/test.mlir
# With parallel (and collapse, which defaults on with it):
onnx-mlir-opt -O3 --shape-inference --convert-onnx-to-krnl=enable-parallel /tmp/test.mlir
# End-to-end test
python3 utils/RunONNXModel.py --model test_{{OP_NAME}}.onnx --verify ortThe onnx-mlir-opt invocation above confirms the lowering is reachable, not that it is correct — reading a lowered loop nest is where correctness bugs slip through. Before writing any FileCheck test for this lowering, ground it numerically: see ground-lit-tests.md. Flag mode (-r=--O0 -t=--O3) is almost always available for ONNX-to-Krnl work, since the input is ONNX dialect and compiles end to end. Compose the test's RUN line before grounding it, and keep it minimal — for a lowering with nothing architecture-specific about it that means no -march/--mtriple at all, plus --canonicalize after the Krnl lowering: ground-lit-tests.md#step-1b-keep-the-run-line-minimal.
RunONNXModel.py checks one model you built yourself. The ONNX backend tests for the op still need enabling in test/backend/inference_backend.py, with the doc regenerated — see add-new-onnx-operation.md step 6. When the lowering extends an existing op (a new attribute, type, or shape case) rather than adding one, recheck which disabled backend tests it now covers: Extending an existing op.
Common Patterns
Elementwise unary op (input and output same shape)
// x → f(x) for each element
create.krnl.iterate(loops, loops, lbs, ubs,
[&](const KrnlBuilder &b, ValueRange idx) {
Value x = b.load(input, idx);
Value result = create.math.exp(x); // or other math op
b.store(result, output, idx);
});Elementwise binary op with NumPy broadcasting
Use ONNXBroadcastOpShapeHelper (ShapeHelper.hpp): computeShapeAndAssertOnFailure() gives the output dims, and inside the loop getAccessExprs(operand, i, outputAccess, access) gives each input's broadcast-adjusted indices. The final output loop in Math/QLinearMatMul.cpp is a compact example.
Op with attributes
double alpha = onnxOp.getAlpha().convertToDouble(); // e.g. Celu in Elementwise.cpp
Value alphaVal = create.math.constant(elemType, alpha);Verification
- Grounded against:
repo:onnx-mlir@b505d64:src/Conversion/ONNXToKrnl/ - Files read:
ConvertONNXToKrnl.cpp,ONNXToKrnlCommon.hpp,KrnlParallelPlan.hpp,CMakeLists.txt,Tensor/EyeLike.cpp,Tensor/Col2Im.cpp,Tensor/TensorScatter.cpp,Math/Det.cpp,Math/QLinearMatMul.cpp,Math/Attention.cpp,Math/Elementwise.cpp,src/Dialect/ONNX/Transforms/AttentionToONNXOps.{hpp,cpp},src/Dialect/ONNX/ONNXUnsupportedOps.hpp - Re-checked at b505d64: the EyeLike template, the populate/registration steps (Det and TensorScatter, added in #3646 and #3657, follow them exactly, including the
CMakeLists.txtentry and per-op parallel gating), and the Attention expansion, which moved into the sharedlowerONNXAttentionOp()(#3673) - How to test: Run
make check-onnx-mliror useRunONNXModel.py --verify ort
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

