kailash-ml
Skill0 starsChanged 8 months agoApache-2.0View on GitHub
Kailash ML — classical + deep learning lifecycle with polars-native engines, AutoML, ONNX serving. Use for feature stores, model training, drift monitoring, inference.
- Installs packages
In this repository37 other files it gives its agents.
- .claude/rules/agents.mdAGENTS.md
- .claude/guides/claude-code/CLAUDE.mdCLAUDE.md
- CLAUDE.mdCLAUDE.md
- core-sdkSkill
- dataflowSkill
- nexusSkill
- kaizenSkill
- kailash-mcpSkill
- cheatsheetsSkill
- development-guidesSkill
- nodes-referenceSkill
- workflow-patternsSkill
- deployment-gitSkill
- frontend-integrationSkill
- testing-strategiesSkill
- architecture-decisionsSkill
- code-templatesSkill
- enterprise-infrastructureSkill
- validation-patternsSkill
- gold-standardsSkill
- security-patternsSkill
- flutter-patternsSkill
- interactive-widgetsSkill
- enterprise-ai-uxSkill
- conversation-uxSkill
- uiux-design-principlesSkill
- value-auditSkill
- ai-interaction-patternsSkill
- eatp-referenceSkill
- coc-referenceSkill
- 29-pactSkill
- claude-code-patternsSkill
- error-troubleshootingSkill
- kailash-alignSkill
- co-referenceSkill
- managementSkill
- projectSkill
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 2 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 132
Every engine accepts and returns polars.DataFrame.
- Distinctive: line 189
All engines are polars-native — no pandas/numpy in pipeline code
- Distinctive: line 194
All agents follow LLM-first rule — tools are dumb data endpoints
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Kailash ML - Classical & Deep Learning Lifecycle
Production ML lifecycle framework built on Kailash Core SDK — polars-native engines, schema-driven pipelines, agent-augmented AutoML, and cross-language ONNX serving.
Install
pip install kailash-ml # Core (polars, numpy, scipy, sklearn, lightgbm, onnx)
pip install kailash-ml[dl] # + PyTorch, Lightning, transformers
pip install kailash-ml[dl-gpu] # + onnxruntime-gpu
pip install kailash-ml[rl] # + Stable-Baselines3, Gymnasium
pip install kailash-ml[agents] # + kailash-kaizen (agent integration)
pip install kailash-ml[xgb] # + XGBoost
pip install kailash-ml[catboost] # + CatBoost
pip install kailash-ml[full] # Everything5 Core Engines
| Engine | Purpose | Key Pattern |
| FeatureStore | Feature ingestion + point-in-time queries | ConnectionManager-backed, polars-native |
| ModelRegistry | Model versioning + lifecycle | staging → shadow → production → archived |
| TrainingPipeline | Schema-driven training + eval | FeatureStore + ModelRegistry integration |
| InferenceServer | Model serving + batch inference | Nexus HTTP exposure, ONNX runtime, caching |
| DriftMonitor | Statistical drift detection | KS, chi-squared, PSI, Jensen-Shannon divergence |
Quick Start
Feature Ingestion
from kailash.db.connection import ConnectionManager
from kailash_ml import FeatureStore
from kailash_ml.types import FeatureSchema, FeatureField
import polars as pl
conn = ConnectionManager("sqlite:///ml.db")
await conn.initialize()
schema = FeatureSchema(
name="user_churn",
features=[
FeatureField(name="age", dtype="float"),
FeatureField(name="tenure_months", dtype="float"),
],
target=FeatureField(name="churned", dtype="int"),
)
fs = FeatureStore(conn, table_prefix="kml_feat_")
await fs.initialize()
df = pl.read_csv("data.csv")
await fs.ingest("user_features", schema, df)Training
from kailash_ml import TrainingPipeline, ModelRegistry, ModelSpec, EvalSpec
from kailash_ml.engines import LocalFileArtifactStore
registry = ModelRegistry(conn, artifact_store=LocalFileArtifactStore("./artifacts"))
await registry.initialize()
pipeline = TrainingPipeline(feature_store=fs, model_registry=registry)
result = await pipeline.train(
schema=schema,
model_spec=ModelSpec(model_class="sklearn.ensemble.RandomForestClassifier"),
eval_spec=EvalSpec(metrics=["accuracy", "f1"]),
)Drift Monitoring
from kailash_ml import DriftMonitor
monitor = DriftMonitor(conn)
await monitor.initialize()
await monitor.set_reference("model_v1", reference_df)
report = await monitor.check_drift("model_v1", current_df)
# report.overall_drift, report.feature_results, report.recommendationsDecision Tree: kailash-ml vs kailash-align vs kailash-kaizen
| You Want To... | Use |
| Train sklearn/LightGBM/XGBoost models | kailash-ml |
| Manage feature pipelines | kailash-ml |
| Monitor model drift | kailash-ml |
| Export models to ONNX | kailash-ml |
| Fine-tune an LLM (LoRA, DPO, RLHF) | kailash-align |
| Serve a fine-tuned LLM via Ollama | kailash-align |
| Build an AI agent with tools | kailash-kaizen |
| Add agent intelligence to ML engines | kailash-ml[agents] (uses Kaizen under the hood) |
| Train RL policies (Gymnasium) | kailash-ml[rl] |
Architecture
kailash-ml/
engines/
_shared.py <- Numeric dtypes, model class validation
_feature_sql.py <- ALL raw SQL (zero SQL in engine files)
_guardrails.py <- AgentGuardrailMixin (5 mandatory guardrails)
feature_store.py <- FeatureStore (ConnectionManager, polars-native)
model_registry.py <- ModelRegistry (lifecycle, SHA256 integrity)
training_pipeline.py <- TrainingPipeline (schema-driven)
inference_server.py <- InferenceServer (Nexus, ONNX, caching)
drift_monitor.py <- DriftMonitor (KS/chi2/PSI/JS)
experiment_tracker.py <- MLflow-compatible run tracking
hyperparameter_search.py <- Grid/random/bayesian/successive halving
automl_engine.py <- Agent-infused AutoML
ensemble.py <- Blend/stack/bag/boost
preprocessing.py <- Auto-setup from FeatureSchema
agents/ <- 6 Kaizen agents (requires kailash-ml[agents])
tools.py <- Dumb data endpoints (LLM-first)
rl/ <- RLTrainer, EnvironmentRegistry, PolicyRegistry
interop.py <- SOLE conversion point (polars <-> sklearn/pandas/arrow)
bridge/ <- OnnxBridge (export + verification)Polars-Native Rule (ABSOLUTE)
Every engine accepts and returns polars.DataFrame. Conversion to numpy/pandas/LightGBM Dataset happens ONLY in interop.py at sklearn/framework boundaries.
# DO: Work in polars throughout
df = pl.read_csv("data.csv")
await fs.ingest("features", schema, df)
# DO NOT: Convert to pandas first
df_pd = pd.read_csv("data.csv") # WRONG -- polars is the native format6 ML Agents (kailash-ml[agents])
Agents require both agent=True AND the agents extra installed. All follow LLM-first rule.
| Agent | Purpose |
| DataScientistAgent | Data profiling recommendations |
| FeatureEngineerAgent | Feature generation guidance |
| ModelSelectorAgent | Model selection reasoning |
| ExperimentInterpreterAgent | Trial result analysis |
| DriftAnalystAgent | Drift report interpretation |
| RetrainingDecisionAgent | Retrain/rollback decisions |
See ml-agent-guardrails for the 5 mandatory guardrails.
RL Module (kailash-ml[rl])
from kailash_ml.rl import RLTrainer, EnvironmentRegistry, PolicyRegistry
env_reg = EnvironmentRegistry()
env_reg.register("CartPole-v1")
trainer = RLTrainer(env_registry=env_reg, policy_registry=PolicyRegistry())
result = await trainer.train(env_id="CartPole-v1", algorithm="PPO", total_timesteps=100_000)Security
- SQL safety:
_feature_sql.pyis the sole SQL touchpoint._validate_identifier()on all interpolated identifiers. - Model class allowlist:
validate_model_class()restricts imports to:sklearn.,lightgbm.,xgboost.,catboost.,kailash_ml.,torch.,lightning. - Financial validation:
math.isfinite()on all budget/cost fields (NaN bypasses comparisons, Inf defeats bounds). - Bounded collections:
deque(maxlen=N)for audit trails, cost logs, trial history.
Skill Files
- ml-feature-pipelines — FeatureStore, polars-only engineering, schema-driven ingestion
- ml-model-registry — ModelRegistry CRUD, lifecycle stages, MLflow compatibility
- ml-training-pipeline — TrainingPipeline, hyperparameter search, experiment tracking
- ml-inference-server — InferenceServer, Nexus exposure, ONNX serving, batch inference
- ml-agent-guardrails — 5 mandatory guardrails, AutoML, agent integration
- ml-onnx-export — PyTorch/sklearn to ONNX, verification, cross-language serving
- ml-drift-monitoring — DriftMonitor, statistical tests, alert thresholds, retraining triggers
Critical Rules
- All engines are polars-native — no pandas/numpy in pipeline code
- sklearn interop only at boundary via
interop.py - FeatureStore uses ConnectionManager, not Express (needs window functions)
- Zero raw SQL outside
_feature_sql.py - Agent-augmented engines require double opt-in (
agent=True+ extras installed) - All agents follow LLM-first rule — tools are dumb data endpoints
Related Skills
- 01-core-sdk — Core workflow patterns
- 02-dataflow — Database integration (ConnectionManager)
- 03-nexus — Multi-channel deployment (InferenceServer)
- 04-kaizen — AI agent framework (ML agents)
- 35-kailash-align — LLM fine-tuning and alignment
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

