agentleFS
ScientiaCapitalScientiaCapital/unsloth-mcp-server/.claude/skills/unsloth-finetuning/SKILL.md

unsloth-finetuning

Skill2 starsChanged 9 months agoApache-2.0View on GitHub

Fine-tune LLMs 2x faster with 80% less memory using Unsloth. Use when the user wants to fine-tune models like Llama, Mistral, Phi, or Gemma. Handles model loading, LoRA configuration, training, and model export.

The file

We split this skill into the rules it gives an agent

6 rules. About 0 are common, 6 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈6

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Use smaller model (1B instead of 3B)

    line 110
  2. Distinctive:

    Use larger model if resources allow

    line 128
  3. Distinctive:

    Use: Llama-1B or Phi-3-mini

    line 147
  4. Distinctive:

    Check model name spelling

    line 262
  5. Distinctive:

    Save checkpoints: every 100 steps

    line 282
  6. Distinctive:

    Validate on test set: before exporting

    line 283

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Unsloth Fine-Tuning

Expert guidance for fine-tuning Large Language Models using Unsloth's optimized library.

Core Capabilities

  • Load models with 4-bit quantization and gradient checkpointing
  • Configure LoRA/QLoRA for efficient fine-tuning
  • Train on custom or Hugging Face datasets
  • Export models to GGUF, Ollama, vLLM, or Hugging Face formats
  • Monitor training with progress tracking
  • Optimize for different hardware configurations

Quick Start

1. Load a Model

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Llama-3.2-1B-bnb-4bit",
    max_seq_length=2048,
    load_in_4bit=True,
    use_gradient_checkpointing="unsloth"
)

Supported Models:

  • Llama 3.3 (70B), 3.2 (1B, 3B), 3.1 (8B)
  • Mistral v0.3 (7B), Small Instruct
  • Phi 3.5 mini, Phi 3 medium
  • Gemma 2 (9B, 27B)
  • Qwen 2.5 (7B)

2. Apply LoRA

model = FastLanguageModel.get_peft_model(
    model,
    r=16,                    # LoRA rank (8-64)
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    lora_alpha=16,          # Scaling factor
    use_gradient_checkpointing="unsloth",
    random_state=3407,
    max_seq_length=2048
)

3. Configure Training

from trl import SFTTrainer, SFTConfig

trainer = SFTTrainer(
    model=model,
    train_dataset=dataset["train"],
    tokenizer=tokenizer,
    args=SFTConfig(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        warmup_steps=10,
        max_steps=100,
        learning_rate=2e-4,
        logging_steps=1,
        output_dir="./output",
        optim="adamw_8bit",
        seed=3407
    )
)

4. Train

trainer.train()

5. Export

# GGUF format
model.save_pretrained_gguf(
    "model",
    tokenizer,
    quantization_method="q4_k_m"
)

# Hugging Face format
model.save_pretrained("./hf_model")
tokenizer.save_pretrained("./hf_model")

Performance Optimization

Memory Optimization

Out of Memory? Try:

  1. Reduce per_device_train_batch_size to 1
  2. Increase gradient_accumulation_steps to 8
  3. Reduce max_seq_length to 1024
  4. Use smaller model (1B instead of 3B)

Speed Optimization

Training too slow? Check:

  1. GPU is being used: nvidia-smi
  2. Batch size isn't too small
  3. Using load_in_4bit=True
  4. Using use_gradient_checkpointing="unsloth"

Quality Optimization

Poor results? Adjust:

  1. Increase max_steps to 500-1000
  2. Try learning rates: 1e-4, 2e-4, 5e-4
  3. Increase dataset quality/size
  4. Use larger model if resources allow

Hardware Requirements

Minimum (1B models)

  • GPU: RTX 3060 (12GB VRAM)
  • RAM: 16GB
  • Training time: 20-40 min for 100 steps

Recommended (3B-7B models)

  • GPU: RTX 4090 or A100
  • RAM: 32GB+
  • Training time: 10-30 min for 100 steps

Budget (Small experiments)

  • GPU: RTX 3060 Ti (8GB)
  • Use: Llama-1B or Phi-3-mini
  • Reduce batch_size=1, max_seq_length=1024

Common Patterns

Pattern 1: Quick Prototype

# Minimal setup for fast experimentation
model, tokenizer = FastLanguageModel.from_pretrained(
    "unsloth/Llama-3.2-1B-bnb-4bit",
    max_seq_length=1024,    # Shorter for speed
    load_in_4bit=True
)

model = FastLanguageModel.get_peft_model(model, r=8)  # Lower rank

trainer = SFTTrainer(
    model=model,
    train_dataset=dataset["train"],
    tokenizer=tokenizer,
    args=SFTConfig(
        per_device_train_batch_size=2,
        max_steps=50,        # Few steps
        learning_rate=2e-4,
        output_dir="./quick_test"
    )
)

Pattern 2: Production Quality

# Full setup for best results
model, tokenizer = FastLanguageModel.from_pretrained(
    "unsloth/Llama-3.1-8B-bnb-4bit",
    max_seq_length=2048,
    load_in_4bit=True,
    use_gradient_checkpointing="unsloth"
)

model = FastLanguageModel.get_peft_model(
    model,
    r=16,                   # Standard rank
    lora_alpha=16,
    use_gradient_checkpointing="unsloth"
)

trainer = SFTTrainer(
    model=model,
    train_dataset=dataset["train"],
    tokenizer=tokenizer,
    args=SFTConfig(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        max_steps=500,       # More steps
        learning_rate=2e-4,
        warmup_steps=10,
        logging_steps=10,
        save_steps=100,
        output_dir="./production_model"
    )
)

Pattern 3: Large Model (70B)

# Special settings for very large models
model, tokenizer = FastLanguageModel.from_pretrained(
    "unsloth/Llama-3.3-70B-Instruct-bnb-4bit",
    max_seq_length=2048,
    load_in_4bit=True,
    use_gradient_checkpointing="unsloth"
)

model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],  # Fewer targets
    use_gradient_checkpointing="unsloth"
)

trainer = SFTTrainer(
    model=model,
    train_dataset=dataset["train"],
    tokenizer=tokenizer,
    args=SFTConfig(
        per_device_train_batch_size=1,    # Must be 1
        gradient_accumulation_steps=8,    # Compensate
        max_steps=200,
        learning_rate=1e-4,               # Lower LR
        output_dir="./large_model"
    )
)

Troubleshooting

Error: "CUDA out of memory"

Solution:

# Reduce memory usage
batch_size = 1
max_seq_length = 1024
gradient_accumulation_steps = 8
# Or use smaller model

Error: "Model not found"

Solution:

  • Check model name spelling
  • Verify internet connection
  • Try with Hugging Face token: export HF_TOKEN=your_token

Error: "Training loss not decreasing"

Solution:

# Adjust hyperparameters
learning_rate = 5e-4  # Try higher
max_steps = 500       # Train longer
# Or check dataset quality

Best Practices

  1. Always use 4-bit quantization unless you have >80GB VRAM
  2. Start with small models (1B) for experimentation
  3. Monitor GPU usage with nvidia-smi
  4. Save checkpoints every 100 steps
  5. Validate on test set before exporting
  6. Use appropriate LoRA rank: 8 for experiments, 16 for production, 32 for complex tasks

Dataset Format

Unsloth works with Hugging Face datasets. Example format:

{
  "text": "### Instruction: Explain quantum computing\n### Response: Quantum computing uses quantum bits..."
}

Or instruction format:

{
  "instruction": "Explain quantum computing",
  "input": "",
  "output": "Quantum computing uses quantum bits..."
}

Performance Benchmarks

ModelVRAMSpeed (vs standard)Memory Reduction
Llama 3.2 1B~2GB2x faster80% less
Llama 3.2 3B~4GB2x faster75% less
Llama 3.1 8B~6GB2x faster70% less
Llama 3.3 70B~40GB2x faster75% less

Additional Resources

For more advanced topics, see:

  • ADVANCED.md - Multi-GPU, custom optimizers, advanced LoRA
  • DATASETS.md - Dataset preparation and formatting
  • EXPORT.md - Detailed export options and formats

Version Compatibility

  • Python: 3.10, 3.11, 3.12 (not 3.13)
  • CUDA: 11.8 or 12.1+
  • PyTorch: 2.0+
  • Transformers: 4.37+

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.