agentleFS
terrene-foundationterrene-foundation/pact/.claude/skills/34-kailash-ml/SKILL.md

kailash-ml

Skill0 starsChanged 8 months agoApache-2.0View on GitHub

Kailash ML — classical + deep learning lifecycle with polars-native engines, AutoML, ONNX serving. Use for feature stores, model training, drift monitoring, inference.

  • Installs packages

The file

We split this skill into the rules it gives an agent

3 rules. About 0 are common, 2 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈2

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Every engine accepts and returns polars.DataFrame.

    line 132
  2. Distinctive:

    All engines are polars-native — no pandas/numpy in pipeline code

    line 189
  3. Distinctive:

    All agents follow LLM-first rule — tools are dumb data endpoints

    line 194

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Kailash ML - Classical & Deep Learning Lifecycle

Production ML lifecycle framework built on Kailash Core SDK — polars-native engines, schema-driven pipelines, agent-augmented AutoML, and cross-language ONNX serving.

Install

pip install kailash-ml            # Core (polars, numpy, scipy, sklearn, lightgbm, onnx)
pip install kailash-ml[dl]        # + PyTorch, Lightning, transformers
pip install kailash-ml[dl-gpu]    # + onnxruntime-gpu
pip install kailash-ml[rl]        # + Stable-Baselines3, Gymnasium
pip install kailash-ml[agents]    # + kailash-kaizen (agent integration)
pip install kailash-ml[xgb]       # + XGBoost
pip install kailash-ml[catboost]  # + CatBoost
pip install kailash-ml[full]      # Everything

5 Core Engines

EnginePurposeKey Pattern
FeatureStoreFeature ingestion + point-in-time queriesConnectionManager-backed, polars-native
ModelRegistryModel versioning + lifecyclestaging → shadow → production → archived
TrainingPipelineSchema-driven training + evalFeatureStore + ModelRegistry integration
InferenceServerModel serving + batch inferenceNexus HTTP exposure, ONNX runtime, caching
DriftMonitorStatistical drift detectionKS, chi-squared, PSI, Jensen-Shannon divergence

Quick Start

Feature Ingestion

from kailash.db.connection import ConnectionManager
from kailash_ml import FeatureStore
from kailash_ml.types import FeatureSchema, FeatureField
import polars as pl

conn = ConnectionManager("sqlite:///ml.db")
await conn.initialize()

schema = FeatureSchema(
    name="user_churn",
    features=[
        FeatureField(name="age", dtype="float"),
        FeatureField(name="tenure_months", dtype="float"),
    ],
    target=FeatureField(name="churned", dtype="int"),
)

fs = FeatureStore(conn, table_prefix="kml_feat_")
await fs.initialize()

df = pl.read_csv("data.csv")
await fs.ingest("user_features", schema, df)

Training

from kailash_ml import TrainingPipeline, ModelRegistry, ModelSpec, EvalSpec
from kailash_ml.engines import LocalFileArtifactStore

registry = ModelRegistry(conn, artifact_store=LocalFileArtifactStore("./artifacts"))
await registry.initialize()

pipeline = TrainingPipeline(feature_store=fs, model_registry=registry)
result = await pipeline.train(
    schema=schema,
    model_spec=ModelSpec(model_class="sklearn.ensemble.RandomForestClassifier"),
    eval_spec=EvalSpec(metrics=["accuracy", "f1"]),
)

Drift Monitoring

from kailash_ml import DriftMonitor

monitor = DriftMonitor(conn)
await monitor.initialize()
await monitor.set_reference("model_v1", reference_df)
report = await monitor.check_drift("model_v1", current_df)
# report.overall_drift, report.feature_results, report.recommendations

Decision Tree: kailash-ml vs kailash-align vs kailash-kaizen

You Want To...Use
Train sklearn/LightGBM/XGBoost modelskailash-ml
Manage feature pipelineskailash-ml
Monitor model driftkailash-ml
Export models to ONNXkailash-ml
Fine-tune an LLM (LoRA, DPO, RLHF)kailash-align
Serve a fine-tuned LLM via Ollamakailash-align
Build an AI agent with toolskailash-kaizen
Add agent intelligence to ML engineskailash-ml[agents] (uses Kaizen under the hood)
Train RL policies (Gymnasium)kailash-ml[rl]

Architecture

kailash-ml/
  engines/
    _shared.py              <- Numeric dtypes, model class validation
    _feature_sql.py         <- ALL raw SQL (zero SQL in engine files)
    _guardrails.py          <- AgentGuardrailMixin (5 mandatory guardrails)
    feature_store.py        <- FeatureStore (ConnectionManager, polars-native)
    model_registry.py       <- ModelRegistry (lifecycle, SHA256 integrity)
    training_pipeline.py    <- TrainingPipeline (schema-driven)
    inference_server.py     <- InferenceServer (Nexus, ONNX, caching)
    drift_monitor.py        <- DriftMonitor (KS/chi2/PSI/JS)
    experiment_tracker.py   <- MLflow-compatible run tracking
    hyperparameter_search.py <- Grid/random/bayesian/successive halving
    automl_engine.py        <- Agent-infused AutoML
    ensemble.py             <- Blend/stack/bag/boost
    preprocessing.py        <- Auto-setup from FeatureSchema
  agents/                   <- 6 Kaizen agents (requires kailash-ml[agents])
    tools.py                <- Dumb data endpoints (LLM-first)
  rl/                       <- RLTrainer, EnvironmentRegistry, PolicyRegistry
  interop.py                <- SOLE conversion point (polars <-> sklearn/pandas/arrow)
  bridge/                   <- OnnxBridge (export + verification)

Polars-Native Rule (ABSOLUTE)

Every engine accepts and returns polars.DataFrame. Conversion to numpy/pandas/LightGBM Dataset happens ONLY in interop.py at sklearn/framework boundaries.

# DO: Work in polars throughout
df = pl.read_csv("data.csv")
await fs.ingest("features", schema, df)

# DO NOT: Convert to pandas first
df_pd = pd.read_csv("data.csv")  # WRONG -- polars is the native format

6 ML Agents (kailash-ml[agents])

Agents require both agent=True AND the agents extra installed. All follow LLM-first rule.

AgentPurpose
DataScientistAgentData profiling recommendations
FeatureEngineerAgentFeature generation guidance
ModelSelectorAgentModel selection reasoning
ExperimentInterpreterAgentTrial result analysis
DriftAnalystAgentDrift report interpretation
RetrainingDecisionAgentRetrain/rollback decisions

See ml-agent-guardrails for the 5 mandatory guardrails.

RL Module (kailash-ml[rl])

from kailash_ml.rl import RLTrainer, EnvironmentRegistry, PolicyRegistry

env_reg = EnvironmentRegistry()
env_reg.register("CartPole-v1")

trainer = RLTrainer(env_registry=env_reg, policy_registry=PolicyRegistry())
result = await trainer.train(env_id="CartPole-v1", algorithm="PPO", total_timesteps=100_000)

Security

  • SQL safety: _feature_sql.py is the sole SQL touchpoint. _validate_identifier() on all interpolated identifiers.
  • Model class allowlist: validate_model_class() restricts imports to: sklearn., lightgbm., xgboost., catboost., kailash_ml., torch., lightning.
  • Financial validation: math.isfinite() on all budget/cost fields (NaN bypasses comparisons, Inf defeats bounds).
  • Bounded collections: deque(maxlen=N) for audit trails, cost logs, trial history.

Skill Files

  • ml-feature-pipelines — FeatureStore, polars-only engineering, schema-driven ingestion
  • ml-model-registry — ModelRegistry CRUD, lifecycle stages, MLflow compatibility
  • ml-training-pipeline — TrainingPipeline, hyperparameter search, experiment tracking
  • ml-inference-server — InferenceServer, Nexus exposure, ONNX serving, batch inference
  • ml-agent-guardrails — 5 mandatory guardrails, AutoML, agent integration
  • ml-onnx-export — PyTorch/sklearn to ONNX, verification, cross-language serving
  • ml-drift-monitoring — DriftMonitor, statistical tests, alert thresholds, retraining triggers

Critical Rules

  • All engines are polars-native — no pandas/numpy in pipeline code
  • sklearn interop only at boundary via interop.py
  • FeatureStore uses ConnectionManager, not Express (needs window functions)
  • Zero raw SQL outside _feature_sql.py
  • Agent-augmented engines require double opt-in (agent=True + extras installed)
  • All agents follow LLM-first rule — tools are dumb data endpoints

Related Skills

  • 01-core-sdk — Core workflow patterns
  • 02-dataflow — Database integration (ConnectionManager)
  • 03-nexus — Multi-channel deployment (InferenceServer)
  • 04-kaizen — AI agent framework (ML agents)
  • 35-kailash-align — LLM fine-tuning and alignment

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.