agentleFS
dtunaidtunai/agent-skills-for-compute/skills/tinker/SKILL.md

tinker

Skill8 starsChanged 8 months agoMITView on GitHub

Tinker by Thinking Machines Lab — low-level training API for LLM fine-tuning. LoRA training, RL environments, supervised learning, DPO, and custom loss functions with full algorithmic control over remote GPU clusters.

  • Reads credentials
  • Installs packages

3 other projects use a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

1 rule. About 0 are common, 0 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈0

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    LoRA learning rate: use hyperparam_utils.get_lr(model_name) — LoRA needs ~10x higher LR than full fine-tuning

    line 273

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Tinker (Thinking Machines Lab)

Low-level training API that abstracts distributed LLM fine-tuning without hiding the knobs. Write a simple Python script on a CPU machine; Tinker executes the GPU-heavy computation on remote clusters.

Official Sources:

Philosophy

You controlTinker handles
Data, RL environments, loss functionsDistributed GPU training at scale
Training loop logic, hyperparametersHardware failures, scheduling, reliability
Evaluations, custom algorithmsEfficient execution up to 235B-param models

Not a black box — full algorithmic control. Changing models requires changing a single string.

Quick Start

# 1. Sign up: https://auth.thinkingmachines.ai/sign-up
# 2. Get API key: https://tinker-console.thinkingmachines.ai
export TINKER_API_KEY=<your-key>

# 3. Install
pip install tinker

# 4. Install cookbook (for recipes and helpers)
git clone https://github.com/thinking-machines-lab/tinker-cookbook
cd tinker-cookbook && pip install -e .

Core API (4 Primitives)

import tinker

# Create a training client
service_client = tinker.ServiceClient()
training_client = service_client.create_lora_training_client(
    base_model="Qwen/Qwen3-8B",
    rank=32,
)

# 1. FORWARD_BACKWARD — compute gradients
training_client.forward_backward(data, loss_fn="cross_entropy")

# 2. OPTIM_STEP — update weights
training_client.optim_step(tinker.AdamParams(learning_rate=1e-4))

# 3. SAVE + SAMPLE — checkpoint and generate
sampling_client = training_client.save_weights_and_get_sampling_client("step-100")
responses = sampling_client.sample(
    prompt=[tinker.TextChunk(text="Hello")],
    num_samples=4,
    sampling_params=tinker.SamplingParams(temperature=0.7, max_tokens=256),
)

# 4. SAVE_STATE — full checkpoint (weights + optimizer)
training_client.save_state("checkpoint-100")

Data Types

import tinker
import numpy as np

# Datum = model_input + loss_fn_inputs
datum = tinker.Datum(
    model_input=tinker.ModelInput(
        chunks=[tinker.TextChunk(text="What is 2+2?")],
    ),
    loss_fn_inputs={
        "target_tokens": tinker.TensorData.from_numpy(
            np.array([42, 43, 44]),
            name="target_tokens",
        ),
    },
)

# Batch is just a list of Datums
batch = [datum1, datum2, datum3]
training_client.forward_backward(batch, loss_fn="cross_entropy")

Loss Functions

Loss FunctionUse Case
cross_entropySupervised fine-tuning
importance_samplingREINFORCE with IS correction
ppoProximal Policy Optimization
cispoClipped Importance Sampling Policy Optimization
droDirect Reward Optimization
forward_backward_customArbitrary differentiable loss

Supervised Learning

from tinker_cookbook.supervised.train import train as sl_train
from tinker_cookbook.renderers import llama3

# Minimal SFT
config = SLConfig(
    base_model="meta-llama/Llama-3.1-8B-Instruct",
    renderer=llama3.Renderer(),
    dataset_builder=my_dataset_builder,
    learning_rate=1e-4,
    rank=32,
    num_epochs=3,
)
sl_train(config)

RL Training

Environment Interface

from tinker_cookbook.rl.types import Env, StepResult, EnvGroupBuilder

class MathEnv(Env):
    def __init__(self, problem: str, answer: str):
        self.problem = problem
        self.answer = answer

    def initial_observation(self) -> list[dict]:
        """Return the initial messages for this rollout."""
        return [
            {"role": "system", "content": "Solve the math problem."},
            {"role": "user", "content": self.problem},
        ]

    def step(self, action: str) -> StepResult:
        """Score the model's response. Called once per turn."""
        correct = self.answer.strip() in action
        return StepResult(
            reward=1.0 if correct else 0.0,
            done=True,
            next_observation=None,
        )

Environment Groups (for GRPO)

class MathEnvGroup(EnvGroupBuilder):
    def __init__(self, problem: str, answer: str, group_size: int = 8):
        self.problem = problem
        self.answer = answer
        self.group_size = group_size

    def build(self) -> list[Env]:
        """Create a group of identical envs for advantage estimation."""
        return [MathEnv(self.problem, self.answer) for _ in range(self.group_size)]

    def compute_group_rewards(self, trajectories: list) -> list[float]:
        """Optional: override for group-level reward shaping."""
        rewards = [t.reward for t in trajectories]
        mean = sum(rewards) / len(rewards)
        return [r - mean for r in rewards]  # baseline subtraction

RL Training Loop

from tinker_cookbook.rl.train import train as rl_train

config = RLConfig(
    base_model="Qwen/Qwen3-8B",
    renderer=qwen3.Renderer(),
    dataset_builder=my_rl_dataset_builder,
    learning_rate=3e-5,
    rank=32,
    group_size=8,
    max_turns=1,
    kl_coeff=0.01,
)
rl_train(config)

DPO Training

from tinker_cookbook.preference.train_dpo import train as dpo_train

config = DPOConfig(
    base_model="meta-llama/Llama-3.1-8B-Instruct",
    renderer=llama3.Renderer(),
    dataset_builder=preference_dataset_builder,
    beta=0.1,
    learning_rate=5e-5,
)
dpo_train(config)

Renderers

Bridge between chat messages and token sequences. Must match the model:

RendererModels
llama3Llama 3.x family
qwen3Qwen 3.x family
deepseek_v3DeepSeek V3
kimi_k2Kimi K2
role_colonGeneric fallback
from tinker_cookbook.renderers import qwen3
renderer = qwen3.Renderer()

Async Pattern (Performance)

# Submit both operations before awaiting — pipeline the GPU work
fb_future = training_client.forward_backward_async(batch, loss_fn="cross_entropy")
optim_future = training_client.optim_step_async(tinker.AdamParams(lr=1e-4))

# Now await
fb_result = await fb_future
optim_result = await optim_future

Recipes (Worked Examples)

RecipeDescription
sl_basic / sl_loopMinimal supervised fine-tuning
rl_basic / rl_loopMinimal RL training
math_rlMath reasoning via RL (GSM8K)
code_rlCode generation with sandbox execution
search_toolTrain LLMs to use retrieval tools
preference/dpoDPO training
preference/rlhfFull 3-stage RLHF pipeline
multiplayer_rlMulti-agent RL (twenty questions, text arena)
distillationOn/off-policy distillation
prompt_distillationInternalize long system prompts
harbor_rlTerminal/tool-use RL
rubricRubric-based evaluation training
vlm_classifierVision-language model classification

Evaluation

from tinker_cookbook.eval import Evaluator

evaluator = Evaluator(
    sampling_client=sampling_client,
    renderer=renderer,
    eval_tasks=["gsm8k", "mmlu"],
)
results = evaluator.run(num_samples=100)

Conventions and Pitfalls

  1. Tensor naming: subscript suffixes _P (problems), _G (groups), _T (tokens), _D (datums)
  2. LoRA learning rate: use hyperparam_utils.get_lr(model_name) — LoRA needs ~10x higher LR than full fine-tuning
  3. Renderer must match model: wrong renderer = garbage tokenization
  4. Env objects are single-use: no reset() method — create new instances per rollout
  5. Always create new sampling client after save_weights: old clients point to stale weights
  6. Use chz.Blueprint for CLI-constructable configs that serialize for sweeps
  7. Supported models: up to 235B parameters — check docs for current model list

CLI

# List active runs
tinker run list

# Get checkpoint info
tinker checkpoint info <checkpoint-id>

# Resume from checkpoint
training_client = service_client.resume_training_client(checkpoint_id="...")

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.