asim-parser-create-parser

---
name: asim-parser-create-parser
description: This starts the process of creating a new ASIM schema parser by generating the initial version of the parser based on the requirements gathered. Use this skill when you have gathered all necessary information for the new ASIM parser and are ready to create the initial version of the parser.
requiredSkills:
  - log-analytics-workspace-queryer
---

# Prerequisites

Before creating the initial version of a new ASIM parser, ensure you have gathered the following information:

1. The schema of the source data type.
2. The source table name.
3. The target ASIM schema.

## Step 1: Data sampling

Query from the source table to get a sample of the data. Use the `log-analytics-workspace-queryer` skill for all KQL queries in this step.

1. Determine the source table schema with this KQL query: `<tableName> | getschema`. This will give you the column names and data types of the source data, which you can use to map to the ASIM schema.
2. Determine how many rows of data there are with this KQL query: `<tableName> | count`.
3. From the number of rows available in the source table, run this KQL query: `<tableName> | take <minimum of rows found or 2000>`. Take as many rows as possible to get a representative sample. Analyze the rows to identify unique values in important columns that will need to be mapped to the ASIM schema. This will help you understand the transformations needed in the parser.

## Step 2: Build the initial version of the ASIM parser

Use the target ASIM schema to build the ASIM parser.

Determine what fields are **Mandatory**, **Recommended**, or **Optional** using this CSV: https://raw.githubusercontent.com/Azure/Azure-Sentinel/refs/heads/master/ASIM/dev/ASimTester/ASimTester.csv

For details about the different fields, use the Learn Microsoft documentation link to the schema. You do not need to map to every field in the ASIM schema, but you should try to map every column in the source data to an ASIM field.

For more information about developing parsers: https://learn.microsoft.com/en-us/azure/sentinel/normalization-develop-parsers

Do not use existing parsers as a reference. Each parser should be built from the ground up based on the source data and the target ASIM schema. This ensures parsers follow best practices and are optimized for performance.

## Step 3: Parser development guidelines

Parsers are KQL functions that follow a clear flow: **Filter → Parse → Map**.

- Use indexes so only relevant extents are scanned.
- Filter early on native columns before parsing to improve performance.
- Use high-performance parsing operators (`split`, `parse-kv`, `parse`) and avoid regular expressions for string parsing.
- Normalize values with `iff`, `case`, or query-local lookup tables rather than copying source values directly.
- Use `project-rename` for mapping, `extend` for calculated or normalized fields.
- Try to map as many fields as possible, including optional ones. This increases usefulness and future-proofs the parser for schema changes.
- Do not use `project-away` to remove unmapped columns. Use `project` instead, as `project-away` does not protect the parser from schema changes in the source data.

### Prohibited patterns

ASIM parsers must read event records from one declared source table. Query-local static mappings created with `datatable` and applied with `lookup` are allowed because they do not read or correlate additional event records. Each lookup key must be unique so the mapping cannot fan out a source record. Do not use:

- **Same-table enrichment** — including a second read of the source table, a self-join, or a tabular subquery that correlates event records.
- **Cross-table enrichment** — including reading event records from another workspace table, a watchlist, `externaldata`, or another external tabular source.
- **One-to-many fan-out** — each source record must produce at most one normalized record. Do not expand one record into multiple normalized records; this indicates that the connector or source event shape must be corrected.
- **Any `mv-*` operator** — including `mv-expand` and `mv-apply`.
- **Event-record aggregation, reaggregation, correlation, or deduplication** — including `summarize`, `distinct`, `arg_min`, `arg_max`, or any other operation that combines source records or collapses expanded rows.

Use values available on the current source row, constants, and query-local static lookup mappings while preserving event-record cardinality. Prefer scalar expressions, scalar dynamic-value access, and direct parsing. If a field cannot be mapped without prohibited event enrichment, fan-out, `mv-*`, or event-record aggregation, leave it unmapped.

### Parsing operators by performance ranking

| Rank | Operator                   | Description                                                                |
| ---: | -------------------------- | -------------------------------------------------------------------------- |
|    1 | `split`                    | Parse a string of values delimited by a delimiter                          |
|    2 | `parse-kv`                 | Extracts structured information from a string expression in key/value form |
|    3 | `parse_csv`                | Parse a string of values formatted as a CSV line                           |
|    4 | `parse`, `parse-where`     | Parse multiple values from an arbitrary string using a pattern             |
|    5 | `extract_all`              | Parse single values from an arbitrary string using a regular expression    |
|    6 | `extract`                  | Extract a single value from an arbitrary string using a regular expression |
|    7 | `parse_json` (`todynamic`) | Parse values in a string formatted as JSON                                 |
|    8 | `parse_xml`                | Parse the values in a string formatted as XML                              |

### Required columns

- Include the column `Type` in the output of the ASIM parser. This column indicates the source table name.
- Determine the placeholder entity fields applicable to the parser from the target schema and `Common` rows in `ASimTester.csv`. Include every defined column whose name ends in `EntityKey` or `AdditionalIds`, as well as `AdditionalEntities` when it is defined for the schema.
- Preserve the exact field names and casing from `ASimTester.csv`. Entity fields are role-specific (for example, `ActorUserEntityKey`, `SrcSystemEntityKey`, and `ActorUserAdditionalIds`); do not replace them with generic names such as `entityKey` or `AdditionalIds`.
- Until mappings for these fields are defined, set every `*EntityKey` field to an empty string and every `*AdditionalIds` field and `AdditionalEntities` to an empty dynamic array. For example:

  ```kusto
  ActorUserEntityKey = "",
  SrcSystemEntityKey = "",
  ActorUserAdditionalIds = dynamic([]),
  AdditionalEntities = dynamic([])
  ```

### Required parameters

Even though this is the parameter-less version, include the following parameters in the KQL function arguments:

- **`disabled: bool = false`** — Allows the parser to be disabled. After calling the table name in the KQL query, include a filter `| where not(disabled)` to ensure the parser can be effectively disabled when needed.

- **`pack: bool = false`** (conditional) — Include this parameter only if the KQL function uses `AdditionalFields`. It allows the user to choose whether to include values in `AdditionalFields` or return it as an empty dynamic. This improves performance for users who do not need the extra information.

## Step 4: Finalize and save

1. **Save** the KQL query in a `.kql` file. The file name must be prefixed with `ASim`, followed by the schema name, event vendor, and event product. For example: `ASimNetworkSessionCiscoASA.kql`. This naming convention is a strict requirement.

2. **Verify** the KQL query runs without syntax errors by using the `log-analytics-workspace-queryer` skill to execute it in the Log Analytics workspace. This step is crucial to validate that the parser is correctly formed and will function as expected when deployed.