fine-tuning-expert
Skill0 starsChanged 11 months agoMITView on GitHub
Use when fine-tuning LLMs, training custom models, or adapting foundation models for specific tasks. Invoke for configuring LoRA/QLoRA adapters, preparing JSONL training datasets, setting hyperparameters for fine-tuning runs, adapter training, transfer learning, finetuning with Hugging Face PEFT, OpenAI fine-tuning, instruction tuning, RLHF, DPO, or quantizing and deploying fine-tuned models. Trigger terms include: LoRA, QLoRA, PEFT, finetuning, fine-tuning, adapter tuning, LLM training, model training, custom model.
9 other projects use a version of this, 6 of them edited.
Adopted by
Most recently taken up by
- lamb92009/claude-skillsedited, first seen Nov 16, 2026
- lsyamsiar/claude-skillsword for word, first seen Nov 1, 2026
- codeApe-7/ai-agent-workflowGroupedited, first seen Nov 1, 2026
- hu465/claude-skills-word for word, first seen Oct 25, 2026
- zacklecon/claude-skillsedited, first seen Oct 13, 2026
- PIXARTSeu/Synapseedited, first seen Sep 14, 2026
Used with
Projects with it also use ui-ux-pro-max.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
StrayDragonSkillui-ux-pro-max
UI/UX design intelligence. 50 styles, 21 palettes, 50 font pairings, 20 charts, 8 stacks (React, Next.js, Vue, Svelte, SwiftUI, React Native, Flutter, Tailwind). Actions: plan, build, create, design, implement, review, fix, improve, optimize, enhance, refactor, check UI/UX code. Projects: website, landing page, dashboard, admin panel, e-commerce, SaaS, portfolio, blog, mobile app, .html, .tsx, .vue, .svelte. Elements: button, modal, navbar, sidebar, card, table, form, chart. Styles: glassmorphism, claymorphism, minimalism, brutalism, neumorphism, bento grid, dark mode, responsive, skeuomorphism, flat design. Topics: color palette, accessibility, animation, layout, typography, font pairing, spacing, hover, shadow, gradient.
Also in 3 projects that use this
Other versions9 projects use a version of this.
Word for word · 3
- Jeffallan/claude-skills★ 12k100% same
- hu465/claude-skills-★ 0100% same
- lsyamsiar/claude-skills★ 0100% same
Edited · 6
- codeApe-7/ai-agent-workflowGroup★ 18100% same
- PIXARTSeu/Synapse★ 9100% same
- zacklecon/claude-skills★ 338% same
- xiangu152/my_claude_skills★ 247% same
- lamb92009/claude-skills★ 038% same
- usmanmubarak1/ClaudeSkilledAgents★ 034% same
Wrote their own version · 0
None found.
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository67 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- angular-architectSkill
- api-designerSkill
- architecture-designerSkill
- atlassian-mcpSkill
- chaos-engineerSkill
- cli-developerSkill
- cloud-architectSkill
- code-documenterSkill
- code-reviewerSkill
- cpp-proSkill
- csharp-developerSkill
- database-optimizerSkill
- debugging-wizardSkill
- devops-engineerSkill
- django-expertSkill
- django-storages-s3Skill
- dotnet-core-expertSkill
- embedded-systemsSkill
- fastapi-expertSkill
- feature-forgeSkill
- flutter-expertSkill
- fullstack-guardianSkill
- game-developerSkill
- golang-proSkill
- graphql-architectSkill
- java-architectSkill
- javascript-proSkill
- kotlin-specialistSkill
- kubernetes-specialistSkill
- laravel-specialistSkill
- legacy-modernizerSkill
- mcp-developerSkill
- microservices-architectSkill
- ml-pipelineSkill
- monitoring-expertSkill
- nestjs-expertSkill
- nextjs-developerSkill
- pandas-proSkill
- php-proSkill
- playwright-expertSkill
- postgres-proSkill
- prompt-engineerSkill
- python-proSkill
- rag-architectSkill
- rails-expertSkill
- react-expertSkill
- react-native-expertSkill
- rust-engineerSkill
- salesforce-developerSkill
- secure-code-guardianSkill
- security-reviewerSkill
- shopify-expertSkill
- spark-engineerSkill
- spec-minerSkill
- spring-boot-engineerSkill
- sql-proSkill
- sre-engineerSkill
- swift-expertSkill
- terraform-engineerSkill
- The first 60 shown.
The file
We split this skill into the rules it gives an agent
5 rules. 3 are common, 2 are distinctive, and 0 are very specific to this project.
3
Common
Most of the 5 edited versions we read keep these.
2
Distinctive
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 26
Use LoRA for most tasks; QLoRA (4-bit) when GPU memory is constrained.
- Common: line 145
Use parameter-efficient methods for large models (>7B)
- Common: line 147
Document hyperparameters and training config
- Distinctive: line 149
Always include a learning rate warmup
- Common: line 159
When implementing fine-tuning, always provide:
“Common” is measured: kept by at least 6 in 10 of the 5 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Fine-Tuning Expert
Senior ML engineer specializing in LLM fine-tuning, parameter-efficient methods, and production model optimization.
Core Workflow
- Dataset preparation — Validate and format data; run quality checks before training starts
- Checkpoint:
python validate_dataset.py --input data.jsonl— fix all errors before proceeding
- Checkpoint:
- Method selection — Choose PEFT technique based on GPU memory and task requirements
- Use LoRA for most tasks; QLoRA (4-bit) when GPU memory is constrained; full fine-tune only for small models
- Training — Configure hyperparameters, monitor loss curves, checkpoint regularly
- Checkpoint: validation loss must decrease; plateau or increase signals overfitting
- Evaluation — Benchmark against the base model; test on held-out set and edge cases
- Checkpoint: collect perplexity, task-specific metrics (BLEU/ROUGE), and latency numbers
- Deployment — Merge adapter weights, quantize, measure inference throughput before serving
Reference Guide
Load detailed guidance based on context:
| Topic | Reference | Load When |
| LoRA/PEFT | references/lora-peft.md | Parameter-efficient fine-tuning, adapters |
| Dataset Prep | references/dataset-preparation.md | Training data formatting, quality checks |
| Hyperparameters | references/hyperparameter-tuning.md | Learning rates, batch sizes, schedulers |
| Evaluation | references/evaluation-metrics.md | Benchmarking, metrics, model comparison |
| Deployment | references/deployment-optimization.md | Model merging, quantization, serving |
Minimal Working Example — LoRA Fine-Tuning with Hugging Face PEFT
from datasets import load_dataset
from transformers import AutoTokenizer, AutoModelForCausalLM, TrainingArguments
from peft import LoraConfig, get_peft_model, TaskType
from trl import SFTTrainer
import torch
# 1. Load base model and tokenizer
model_id = "meta-llama/Llama-3-8B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.pad_token = tokenizer.eos_token
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
# 2. Configure LoRA adapter
lora_config = LoraConfig(
task_type=TaskType.CAUSAL_LM,
r=16, # rank — increase for more capacity, decrease to save memory
lora_alpha=32, # scaling factor; typically 2× rank
target_modules=["q_proj", "v_proj"],
lora_dropout=0.05,
bias="none",
)
model = get_peft_model(model, lora_config)
model.print_trainable_parameters() # verify: should be ~0.1–1% of total params
# 3. Load and format dataset (Alpaca-style JSONL)
dataset = load_dataset("json", data_files={"train": "train.jsonl", "test": "test.jsonl"})
def format_prompt(example):
return {"text": f"### Instruction:\n{example['instruction']}\n\n### Response:\n{example['output']}"}
dataset = dataset.map(format_prompt)
# 4. Training arguments
training_args = TrainingArguments(
output_dir="./checkpoints",
num_train_epochs=3,
per_device_train_batch_size=4,
gradient_accumulation_steps=4, # effective batch size = 16
learning_rate=2e-4,
lr_scheduler_type="cosine",
warmup_ratio=0.03, # always use warmup
fp16=False,
bf16=True,
logging_steps=10,
eval_strategy="steps",
eval_steps=100,
save_steps=200,
load_best_model_at_end=True,
)
# 5. Train
trainer = SFTTrainer(
model=model,
args=training_args,
train_dataset=dataset["train"],
eval_dataset=dataset["test"],
dataset_text_field="text",
max_seq_length=2048,
)
trainer.train()
# 6. Save adapter weights only
model.save_pretrained("./lora-adapter")
tokenizer.save_pretrained("./lora-adapter")QLoRA variant — add these lines before loading the model to enable 4-bit quantization:
from transformers import BitsAndBytesConfig
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_use_double_quant=True,
)
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map="auto")Merge adapter into base model for deployment:
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16)
merged = PeftModel.from_pretrained(base, "./lora-adapter").merge_and_unload()
merged.save_pretrained("./merged-model")Constraints
MUST DO
- Validate dataset quality before training
- Use parameter-efficient methods for large models (>7B)
- Monitor training/validation loss curves
- Document hyperparameters and training config
- Version datasets and model checkpoints
- Always include a learning rate warmup
MUST NOT DO
- Skip data quality validation
- Overfit on small datasets — use regularisation (dropout, weight decay) and early stopping
- Merge incompatible adapters (mismatched rank, base model, or target modules)
- Deploy without evaluation against a held-out set and latency benchmark
Output Templates
When implementing fine-tuning, always provide:
- Dataset preparation script with validation logic (schema checks, token-length histogram, deduplication)
- Training configuration (full
TrainingArguments+LoraConfigblock, commented) - Evaluation script reporting perplexity, task-specific metrics, and latency
- Brief design rationale — why this PEFT method, rank, and learning rate were chosen for this task
Maintained by @jeffallan, Principal Consultant at Synergetic Solutions
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

