ESM-C: Efficient Protein Embeddings

Overview

ESM-C (Cambrian) is EvolutionaryScale's embedding-focused protein language model family, designed as a drop-in upgrade to ESM2 with ~3× faster inference and improved embedding quality across all model sizes.

Choosing between ESM-C and ESM2:

• Use ESM-C when embeddings are the primary goal — it's faster and produces better representations
• Use ESM2 when you also need variant effect scoring (EsmForMaskedLM), contact prediction, or ESMFold structure prediction — those capabilities are not available in ESM-C

Installation

pip install esm

ESM-C models are also available on HuggingFace (evolutionaryscale/esmc-300m-2024-12, esmc-600m-2024-12) if you prefer the transformers ecosystem.

Model Selection

Start with esmc-600m; drop to esmc-300m for real-time or CPU applications.

Core Usage

Basic Embeddings

from esm.models.esmc import ESMC
from esm.sdk.api import ESMProtein
import torch
import torch.nn.functional as F

model = ESMC.from_pretrained("esmc-600m").to("cuda")
model.eval()

sequence = "MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQAPILSRVGD"
protein = ESMProtein(sequence=sequence)
output = model.forward(model.encode(protein))

# output.embeddings: (1, seq_len, hidden_dim) — residues only, no special tokens to strip
per_residue  = output.embeddings[0]                              # (L, 1152)
per_sequence = per_residue.mean(dim=0)                           # (1152,)
per_sequence_norm = F.normalize(per_sequence.unsqueeze(0), dim=-1)  # L2 normalized

Key difference from ESM2: there are no [CLS]/[EOS] special tokens to strip. output.embeddings[0] is already residue-only.

Batch Processing

ESM-C processes sequences individually via encode → forward. Wrap in a helper for batches:

def get_embeddings(model, sequences):
    """Mean-pooled per-sequence embeddings for a list of sequences."""
    embeddings = []
    for seq in sequences:
        protein = ESMProtein(sequence=seq)
        output = model.forward(model.encode(protein))
        emb = output.embeddings[0].mean(dim=0).detach().float()
        embeddings.append(emb)
    return torch.stack(embeddings)  # (N, hidden_dim)

For large datasets, see scripts/embed.py for an optimized batching loop with GPU cache management.

Common Tasks

Similarity Search

import torch.nn.functional as F

# Build a normalized embedding matrix for a database
db_embeddings = F.normalize(get_embeddings(model, db_sequences), dim=-1)  # (N, D)

# Query
query_emb = F.normalize(get_embeddings(model, [query_seq]), dim=-1)        # (1, D)
scores = (query_emb @ db_embeddings.T)[0]                                  # cosine similarities
top_hits = scores.topk(10)

For large-scale search (>10k sequences), use FAISS — see references/esmc-api.md.

Classification / Regression

Use embeddings as features with sklearn (no fine-tuning needed):

from sklearn.linear_model import LogisticRegression

X = get_embeddings(model, train_sequences).numpy()
clf = LogisticRegression(max_iter=1000)
clf.fit(X, train_labels)

Or fine-tune ESM-C end-to-end with a task-specific head:

import torch.nn as nn

class ProteinClassifier(nn.Module):
    def __init__(self, esm_model, hidden_dim, num_labels):
        super().__init__()
        self.esm = esm_model
        self.head = nn.Linear(hidden_dim, num_labels)

    def forward(self, sequences):
        embs = []
        for seq in sequences:
            protein = ESMProtein(sequence=seq)
            out = self.esm.forward(self.esm.encode(protein))
            embs.append(out.embeddings[0].mean(0))
        return self.head(torch.stack(embs))

Comparison with ESM2

Scripts

scripts/embed.py — CLI for batch embedding from FASTA:

# Embed a FASTA file, save as numpy
python scripts/embed.py proteins.fasta --output embeddings.npy

# Save as HDF5 (one dataset per sequence ID), L2-normalized
python scripts/embed.py proteins.fasta --output embeddings.h5 --normalize

# Use a different model
python scripts/embed.py proteins.fasta --output embeddings.npy --model esmc-300m

Best Practices

• L2-normalize before computing cosine similarity or building a FAISS index
• Cache embeddings for datasets you'll query repeatedly
• Half precision (model.half()) reduces GPU memory by ~50% with minimal quality loss
• Sequences > 1024 residues must be truncated or chunked
• For very large batches, clear GPU cache periodically: torch.cuda.empty_cache()

Resources

• Blog: https://www.evolutionaryscale.ai/blog/esm-cambrian
• GitHub: https://github.com/evolutionaryscale/esm
• HuggingFace: evolutionaryscale/esmc-300m-2024-12, evolutionaryscale/esmc-600m-2024-12

References

• references/esmc-api.md — batch processing patterns, FAISS integration, fine-tuning, embedding caching, per-residue analysis, attention visualization