Evo2: Genome-Scale DNA Foundation Model

Overview

Evo2 is a DNA language model from Arc Institute that operates at single-nucleotide resolution with up to 1 million base pair context. It is trained on 8.8 trillion tokens from OpenGenome2 — sequences spanning all domains of life.

This is a DNA model, not a protein model. Input is raw nucleotide sequence (A/C/G/T). For protein tasks, use ESM2 or ESM-C instead.

Core capabilities:

• Variant effect scoring — zero-shot log-likelihood scoring of SNPs, indels, regulatory variants
• Sequence embeddings — genomic representations for downstream ML (classification, regression)
• Sequence generation — autoregressive DNA sequence design from a prompt
• Positional entropy — per-nucleotide uncertainty across a sequence
• Perplexity analysis — sliding window perplexity for detecting unusual regions

Architecture: StripedHyena 2 — hybrid attention + gated convolutions. Not a standard Transformer. Attention only at layers 3, 10, 17, 24, 31; the rest are Hyena convolution blocks.

Installation

# Full install (all models including 20B/40B)
conda install -c nvidia cuda-nvcc cuda-cudart-dev
conda install -c conda-forge transformer-engine-torch=2.3.0
pip install flash-attn==2.8.0.post2 --no-build-isolation
pip install evo2

# Light install (7B models only — no FP8 required)
pip install flash-attn==2.8.0.post2 --no-build-isolation
pip install evo2

Requirements: Python 3.11–3.12, CUDA 12.1+, Linux (WSL2 with caveats).

Model Selection

Start with evo2_7b. Model weights download automatically from HuggingFace on first use.

Core Usage

Load the model

from evo2 import Evo2

model = Evo2('evo2_7b')    # downloads from HuggingFace on first run
# model.model      — StripedHyena2 backbone
# model.tokenizer  — CharLevelTokenizer (vocab size 512)

1. Variant Effect Scoring

Score the effect of a variant by comparing log-likelihoods between reference and variant sequences.

reference = "ACGTACGTACGTACGT"
variant   = "ACGTACGAACGTACGT"   # T→A at position 7

scores = model.score_sequences(
    seqs=[reference, variant],
    batch_size=1,
    reduce_method='mean',            # 'mean' or 'sum'
    average_reverse_complement=True, # recommended: average both strands
)

delta_ll = scores[1] - scores[0]
# delta_ll > 0 → variant more likely than reference (neutral or beneficial)
# delta_ll < 0 → variant less likely (potentially deleterious)
print(f"Δlog-likelihood: {delta_ll:.4f}")

Scoring is log-likelihood, not masked marginals — Evo2 is autoregressive. Every position is scored conditioned on all preceding positions. Reverse complement averaging (average_reverse_complement=True) improves robustness for double-stranded DNA.

2. Batch Variant Scanning

Scan all single-nucleotide variants (SNVs) at a specific position:

import numpy as np

BASES = list("ACGT")

def scan_position(model, sequence, position):
    """Score all single-nucleotide substitutions at one position."""
    ref_aa = sequence[position]
    variants = []
    for alt in BASES:
        if alt == ref_aa:
            variants.append(sequence)  # reference
        else:
            mut = list(sequence)
            mut[position] = alt
            variants.append("".join(mut))

    scores = model.score_sequences(variants, batch_size=4, reduce_method='mean')
    ref_score = scores[BASES.index(ref_aa)]
    return {alt: s - ref_score for alt, s in zip(BASES, scores)}

position_scores = scan_position(model, reference, position=7)
# {'A': 0.0, 'C': -0.23, 'G': -0.15, 'T': -0.41}  (ref T has score 0.0)

See scripts/score_variants.py for a CLI that scans a genomic window or named VCF variants.

3. Sequence Embeddings

Extract per-token embeddings from an intermediate layer for downstream classification or regression.

import torch

sequence = "ACGTACGTACGTACGT"
input_ids = torch.tensor(
    model.tokenizer.tokenize(sequence),
    dtype=torch.int,
).unsqueeze(0).to('cuda:0')    # shape: (1, seq_len)

# Extract from layer 28 MLP output (recommended for genomic tasks)
logits, embeddings = model(
    input_ids,
    return_embeddings=True,
    layer_names=['blocks.28.mlp.l3'],
)

per_token = embeddings['blocks.28.mlp.l3'][0]   # (seq_len, 4096)
per_seq   = per_token.mean(dim=0)               # (4096,) — mean-pooled

Layer choice: Layer 28 MLP output (blocks.28.mlp.l3) is the recommended default. Intermediate layers outperform the final layer for most downstream tasks. For attention-specific representations, use blocks.31 (last attention layer).

See scripts/embed.py for a FASTA → embeddings CLI.

4. Sequence Generation

Generate DNA sequence from a prompt:

generated_seqs, scores = model.generate(
    prompt_seqs=['ACGTACGT'],
    n_tokens=500,
    temperature=1.0,
    top_k=4,
    top_p=1.0,
    batched=True,
    cached_generation=True,
    verbose=1,
)
print(generated_seqs[0])   # full generated string (prompt + continuation)
print(f"Score: {scores[0]:.4f}")

For batch generation from multiple prompts, pass a list of strings. Each prompt can have a different length.

5. Positional Entropy

Measure per-position uncertainty — high entropy = model is unsure what nucleotide should be there. Useful for identifying low-complexity regions, repeats, or unusual sequence features.

from evo2.scoring import positional_entropies

entropies = positional_entropies(
    seqs=[sequence],
    model=model.model,
    tokenizer=model.tokenizer,
    prepend_bos=False,
    device='cuda:0',
)
entropy_array = entropies[0]   # numpy array, shape (seq_len,)
print(f"Mean entropy: {entropy_array.mean():.4f}")
print(f"Max entropy pos: {entropy_array.argmax()}")

6. Perplexity Along Sequence

Sliding window perplexity for detecting unusual regions:

from evo2.scoring import score_perplexity_along_sequence

perplexity = score_perplexity_along_sequence(
    model=model,
    seq=sequence,
    reverse_complement=True,   # average both strands
    entropy=False,             # True for entropy, False for perplexity
)
# Returns: numpy array of per-position perplexity values

Tokenizer

# Tokenize a DNA sequence
tokens = model.tokenizer.tokenize('ACGT')   # list of ints
ids = torch.tensor(tokens, dtype=torch.int).unsqueeze(0).to('cuda:0')

# Vocabulary size: 512 (character-level)
# Special tokens:
model.tokenizer.pad_id   # padding
model.tokenizer.eod_id   # end-of-document

Input must be uppercase A/C/G/T. N or ambiguous bases are in the vocabulary but may affect scoring. Convert all ambiguous bases to one of ACGT (or mask them) before scoring.

Long Sequences

Evo2 supports up to 1M tokens natively with evo2_7b (1M context). For sequences approaching this limit:

# Process a large sequence in one pass (if < 1M bp)
long_seq = open("chromosome.fna").read().replace("\n", "").upper()
print(f"Length: {len(long_seq):,} bp")

# For sequences > 1M bp: use a sliding window
window = 500_000
step   = 250_000
windows = [long_seq[i:i+window] for i in range(0, len(long_seq), step)]
scores = model.score_sequences(windows, batch_size=1, reduce_method='mean')

Scoring Conventions

• score_sequences returns mean (or sum) log-likelihood per sequence — more negative = less likely
• For variant scoring: delta = variant_score - reference_score
  • delta > 0 → variant more consistent with training data distribution
  • delta < 0 → variant less likely (potentially deleterious for functional sequences)
• No absolute cutoff exists — comparisons are relative; always score against a reference
• average_reverse_complement=True is strongly recommended for coding sequences and regulatory elements, as they can be on either strand

Scripts

# Score all SNVs in a 50bp window around a target position
python scripts/score_variants.py genome.fasta chr1:230100 \
  --window 50 --output variants.csv

# Score named variants from a VCF
python scripts/score_variants.py genome.fasta --vcf mutations.vcf \
  --flanking 100 --output scores.csv

# Embed a FASTA file of genomic sequences → .npy
python scripts/embed.py sequences.fasta --output embeddings.npy

# Embed with layer selection
python scripts/embed.py sequences.fasta --layer blocks.28.mlp.l3 \
  --output embeddings.npy --normalize

When to Use Evo2 vs. Protein Models

Resources

• GitHub: https://github.com/arcinstitute/evo2
• Paper: Brixi et al., Nature 2026 — https://doi.org/10.1038/s41586-026-10176-5
• HuggingFace: https://huggingface.co/arcinstitute
• NVIDIA hosted API: https://build.nvidia.com/arc/evo2-40b

References

• references/api.md — full API reference: all scoring functions, embedding extraction, generation parameters, long-sequence strategies, fine-tuning