agentleFS
alibabaalibaba/ROLL/CLAUDE.md

ROLL

CLAUDE.md3.4k starsChanged 5 months agoApache-2.0View on GitHub

ROLL (Reinforcement Learning Optimization for Large-Scale Learning) is an efficient RL library for Large Language Models (LLMs) using large-scale GPU resources. It enhances LLM performance in human preference alignment, complex reasoning, and multi-turn agentic interactions.

Trending

Quiet since February.thin dataabout the whole repository

Dec 2025: 4 mentionsJan 2026: 0 mentionsFeb 2026: 1 mentionMar 2026: 0 mentionsApr 2026: 0 mentionsMay 2026: 0 mentionsJun 2026: 0 mentionsJul 2026: 0 mentionsAug 2026: 0 mentionsSep 2026: 0 mentionsOct 2026: 0 mentionsNov 2026: 0 mentions
Dec 2025Jun 2026Nov 2026
Mentions a month of the alibaba/ROLL repository that our weekly crawl found: only 9 in all, so this is a hint, not a trend. There is no adoption data for this file yet.

Who's talking about alibaba/ROLL

The biggest 5 of 9 mentions linking to the repository that our crawl found. Each line is their own headline.

See all 9

Talked about on GitHub, Hacker News and Bluesky (9 times).

On GitHub · 7

2 more on GitHub

The file

Show its rules and text

We split this CLAUDE.md into the rules it gives an agent

19 rules. About 0 are common, 19 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈19

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Follow existing patterns exactly

    line 16
  2. Distinctive:

    Follow existing module structure when adding new workers/pipelines

    line 18
  3. Distinctive:

    Use f-strings for formatting

    line 24
  4. Distinctive:

    Simplicity: Write simple, straightforward code

    line 28
  5. Distinctive:

    Performance: Consider performance without sacrificing readability

    line 30
  6. Distinctive:

    Maintainability: Write code that's easy to update

    line 31
  7. Distinctive:

    Testability: Ensure code is testable

    line 32
  8. Distinctive:

    Reusability: Create reusable components and functions

    line 33
  9. Distinctive:

    Early Returns: Use to avoid nested conditions

    line 38
  10. Distinctive:

    Descriptive Names: Use clear variable/function names (prefix handlers with "handle")

    line 39
  11. Distinctive:

    Constants Over Functions: Use constants where possible

    line 40
  12. Distinctive:

    DRY Code: Don't repeat yourself

    line 41
  13. Distinctive:

    Functional Style: Prefer functional, immutable approaches when not verbose

    line 42
  14. Distinctive:

    Minimal Changes: Only modify code related to the task at hand

    line 43
  15. Distinctive:

    Build Iteratively: Start with minimal functionality and verify it works before adding complexity

    line 47
  16. Distinctive:

    Run Tests: Test your code frequently with realistic inputs and validate outputs

    line 48
  17. Distinctive:

    Build Test Environments: Create testing environments for components that are difficult to validate directly

    line 49
  18. Distinctive:

    Functional Code: Use functional and stateless approaches where they improve clarity

    line 50
  19. Distinctive:

    Clean logic: Keep core logic clean and push implementation details to the edges

    line 51

≈ “Common” is an estimate: at least 1 in 5 of the 27,641 CLAUDE.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

ROLL (Reinforcement Learning Optimization for Large-Scale Learning) is an efficient RL library for Large Language Models (LLMs) using large-scale GPU resources. It enhances LLM performance in human preference alignment, complex reasoning, and multi-turn agentic interactions.

Core Development Rules

  1. Code Quality
    • Type hints required for all code
    • Public APIs must have docstrings
    • Document with docstrings
    • Functions must be focused and small
    • Follow existing patterns exactly
    • Line length: 119 characters
    • Follow existing module structure when adding new workers/pipelines
    • Configuration-driven design - avoid hardcoding parameters
    • Python 3.10+ target
    • PEP 8 naming (snake_case for functions/variables)
    • Class names in PascalCase
    • Constants in UPPER_SNAKE_CASE
    • Use f-strings for formatting

Development Philosophy

  • Simplicity: Write simple, straightforward code
  • Readability: Make code easy to understand
  • Performance: Consider performance without sacrificing readability
  • Maintainability: Write code that's easy to update
  • Testability: Ensure code is testable
  • Reusability: Create reusable components and functions
  • Less Code = Less Debt: Minimize code footprint

Coding Best Practices

  • Early Returns: Use to avoid nested conditions
  • Descriptive Names: Use clear variable/function names (prefix handlers with "handle")
  • Constants Over Functions: Use constants where possible
  • DRY Code: Don't repeat yourself
  • Functional Style: Prefer functional, immutable approaches when not verbose
  • Minimal Changes: Only modify code related to the task at hand
  • Function Ordering: Define composing functions before their components
  • TODO Comments: Mark issues in existing code with "TODO:" prefix
  • Simplicity: Prioritize simplicity and readability over clever solutions
  • Build Iteratively Start with minimal functionality and verify it works before adding complexity
  • Run Tests: Test your code frequently with realistic inputs and validate outputs
  • Build Test Environments: Create testing environments for components that are difficult to validate directly
  • Functional Code: Use functional and stateless approaches where they improve clarity
  • Clean logic: Keep core logic clean and push implementation details to the edges
  • File Organsiation: Balance file organization with simplicity - use an appropriate number of files for the project scale

Common Development Commands

Running Pipelines

# RLVR Pipeline
python examples/start_rlvr_pipeline.py --config_name sppo_config

# Agentic Pipeline
python examples/start_agentic_pipeline.py --config_name sokoban_ppo_config

# Override config parameters
python examples/start_rlvr_pipeline.py rollout_batch_size=128 max_steps=1000

High-Level Architecture

1. Pipeline Definitions (roll/pipeline/)

The framework provides two main training pipelines:

  • RLVR Pipeline (Reinforcement Learning with Verifiable Rewards)
    • Multi-domain training with dynamic reward routing
    • Supports PPO, GRPO, Reinforce++ algorithms
    • Handles math, code, general reasoning, and other domains
  • Agentic Pipeline
    • Environment-based RL training
    • Supports environments like Sokoban, WebShop, FrozenLake
    • Trajectory collection and policy optimization

2. Distributed System (roll/distributed/)

Multi-role distributed architecture using Ray:

  • Executor: Worker management, clusters, model update groups
  • Scheduler: Resource management, generation/reward scheduling
  • Strategy: Multiple backend support:
    • Megatron-Core: Model parallelism (TP, PP, CP, EP)
    • DeepSpeed: ZeRO optimization, CPU offloading
    • vLLM/SGLang: High-throughput inference
    • FSDP: PyTorch native sharding
    • HuggingFace: Standard transformers integration

3. Worker Types (Role-based architecture)

  • Actor Workers: Policy model training and inference
  • Critic Workers: Value function estimation
  • Reference Workers: KL divergence calculation
  • Reward Workers: Domain-specific reward computation
    • Math reward models
    • Code evaluation (sandbox)
    • LLM-as-judge
    • Rule-based rewards (IFEval, CrossThinkQA)
  • Environment Workers: For agentic tasks

4. Model Support (roll/models/)

  • Model providers for different frameworks
  • Function providers for specialized operations
  • TRL (Transformers Reinforcement Learning) patches
  • Vision-Language model support (Qwen-VL)

📁 Key Components

Configuration System

  • Uses Hydra for hierarchical YAML configs
  • Supports CLI overrides
  • Modular configuration for different components

Utilities (roll/utils/)

  • Collective operations: Distributed training primitives
  • Checkpoint management: Save/resume functionality
  • Metrics tracking: Performance monitoring
  • Code evaluation: Sandboxed code execution

Third-party Integrations (roll/third_party/)

  • Custom patches for vLLM, SGLang, DeepSpeed, Megatron
  • Optimizations for offloading and memory management

🚀 Example Configurations

The examples/ directory contains ready-to-use configurations:

  • Small models: Qwen2.5-0.5B for testing
  • Production models: Qwen2.5-7B, Qwen3-30B
  • Vision-Language: Qwen-VL models
  • Various environments: Sokoban, WebShop, FrozenLake

🧪 Testing Infrastructure

Comprehensive test suite covering:

  • Unit tests for individual components
  • Integration tests for pipelines
  • Performance benchmarks
  • Multi-GPU/multi-node tests

📚 Documentation

  • User guides in English and Chinese
  • API documentation
  • Configuration guides
  • Step-by-step tutorials

Key Configuration Parameters

When configuring training:

  • rollout_batch_size: Number of prompts per generation batch
  • num_return_sequences_in_group: Samples per prompt for variance reduction
  • ppo_epochs: PPO update epochs (usually 1)
  • init_kl_coef: KL penalty coefficient (0.1-0.5 typical)
  • reward_clip: Clip extreme rewards (5-10 typical)
  • advantage_clip: Clip advantages for stability
  • domain_interleave_probs: Multi-domain sampling ratios

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.