ESM3: Multimodal Generative Protein Design

Overview

ESM3 is a generative protein language model from EvolutionaryScale that reasons simultaneously over sequence, structure, and function. Unlike ESM2 (discriminative, embeddings only), ESM3 generates proteins using iterative masked language modeling — you provide partial information in any combination of modalities and ESM3 fills in the rest.

Core use cases:

• Complete partial/masked sequences
• Structure prediction from sequence (fast alternative for design iterations)
• Inverse folding — design sequences that fold to a target structure
• Function-conditioned generation — proteins with specific functional annotations
• Chain-of-thought design — iterate across sequence → structure → function tracks

Installation

pip install esm
# Optional: Flash Attention for 2–4× faster inference on GPU
pip install flash-attn --no-build-isolation

Model Selection

Only esm3-sm-open-v1 runs locally. All other models require a Forge API token from https://forge.evolutionaryscale.ai.

Core Concepts

ESMProtein — the data container

from esm.sdk.api import ESMProtein

protein = ESMProtein(
    sequence="MPRT____KEND",      # use '_' to mark positions to generate
    coordinates=coords,           # optional: (L, 37, 3) numpy array of atom coords
    secondary_structure="HHHCCC", # optional: per-residue H/E/C annotation
    function_annotations=[...],   # optional: FunctionAnnotation objects
    sasa=sasa_array,              # optional: per-residue solvent accessibility
)

# Load from / export to PDB
protein = ESMProtein.from_pdb("structure.pdb")
pdb_str  = protein.to_pdb()

_ is the masking token. Unmasked positions are hard constraints — ESM3 will not change them.

GenerationConfig — control what and how to generate

from esm.sdk.api import GenerationConfig

config = GenerationConfig(
    track="sequence",    # "sequence", "structure", or "function"
    num_steps=16,        # demasking iterations — more steps can improve quality
    temperature=0.7,     # 0.0 = deterministic, 0.5–0.7 = balanced, 1.0 = diverse
    top_p=0.9,           # nucleus sampling threshold (optional)
    condition_on_coordinates_only=False,  # set True for inverse folding
)

num_steps rule of thumb: n_masked_positions // 2 to n_masked_positions. For generating a 100-residue sequence from scratch, start with 50 steps.

Core Workflows

1. Sequence Completion (Local)

Fill in masked gaps between known anchors:

from esm.models.esm3 import ESM3
from esm.sdk.api import ESMProtein, GenerationConfig
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"
model = ESM3.from_pretrained("esm3-sm-open-v1").to(device)

protein = ESMProtein(sequence="MPRTK__________LIVHSP")
result = model.generate(
    protein,
    GenerationConfig(track="sequence", num_steps=10, temperature=0.7),
)
print(result.sequence)

2. Structure Prediction

protein = ESMProtein(sequence="MPRTKEINDAGLIVHSPQWFYK")
result = model.generate(
    protein,
    GenerationConfig(track="structure", num_steps=len(protein.sequence)),
)
with open("predicted.pdb", "w") as f:
    f.write(result.to_pdb())

3. Inverse Folding

Design a sequence that folds to a given target structure:

protein = ESMProtein.from_pdb("target.pdb")
protein.sequence = None  # remove sequence, keep coordinates

result = model.generate(
    protein,
    GenerationConfig(
        track="sequence",
        num_steps=50,
        temperature=0.7,
        condition_on_coordinates_only=True,
    ),
)
print(result.sequence)

4. Function-Conditioned Generation

from esm.sdk.api import FunctionAnnotation

protein = ESMProtein(
    sequence="_" * 150,
    function_annotations=[
        FunctionAnnotation(label="kinase", start=30, end=120),
    ],
)
result = model.generate(
    protein,
    GenerationConfig(track="sequence", num_steps=75, temperature=0.6),
)

5. Chain-of-Thought Design

Iteratively refine across tracks — let each modality inform the next:

# Start with a partial sequence prompt
protein = ESMProtein(sequence="MPRT" + "_" * 80)

# Step 1: complete the sequence
protein = model.generate(protein, GenerationConfig(track="sequence", num_steps=40, temperature=0.6))

# Step 2: predict structure for the completed sequence
protein = model.generate(protein, GenerationConfig(track="structure", num_steps=50))

# Step 3: annotate predicted function
protein = model.generate(protein, GenerationConfig(track="function", num_steps=20))

print(protein.sequence)
print(protein.function_annotations)
with open("design.pdb", "w") as f:
    f.write(protein.to_pdb())

6. Via Forge API (Larger Models)

from esm.sdk.forge import ESM3ForgeInferenceClient

model = ESM3ForgeInferenceClient(
    model="esm3-medium-2024-08",
    url="https://forge.evolutionaryscale.ai",
    token="<your-forge-token>",
)
protein = ESMProtein(sequence="MPRT" + "_" * 50 + "KEND")
result = model.generate(protein, GenerationConfig(track="sequence", num_steps=25, temperature=0.7))

Scripts

scripts/generate.py — CLI for generation and inverse folding. Run without arguments to see usage:

# Complete a masked sequence (10 samples)
python scripts/generate.py "MPRTK______LIVHSP" --num-samples 10

# Predict structure and save PDB
python scripts/generate.py "MPRTKEINDAGLIVHSP" --track structure --output predicted.pdb

# Inverse folding from a PDB
python scripts/generate.py --from-pdb target.pdb --track sequence --output designed.fasta

# Use a Forge model
python scripts/generate.py "MPRT____KEND" --model esm3-medium-2024-08 --forge-token <token>

Best Practices

• Start small: prototype with esm3-sm-open-v1 locally; upgrade to Forge models for final designs
• Temperature: 0.5–0.7 for functional design; 0.8–1.0 when you want to explore diverse sequences
• Validate outputs: always verify generated sequences with structure prediction — ESM3 is a statistical model, not a physics simulator; some outputs will be non-functional
• Constrained design: fix active sites, binding motifs, or disulfide cysteines by leaving them unmasked
• Generating from scratch: "_" * L with num_steps = L // 2 is a good starting point

Resources

• GitHub: https://github.com/evolutionaryscale/esm
• Forge API: https://forge.evolutionaryscale.ai
• Paper: Hayes et al., Science 2025 — https://doi.org/10.1126/science.ads0018
• Responsible Biodesign: https://responsiblebiodesign.ai/

References

• references/esm3-api.md — full API reference: ESMProtein fields, GenerationConfig parameters, constrained generation, secondary structure conditioning, Forge async batching, error handling