How to predict peptide-protein affinity? New benchmarks

Imagine having to find the right key to open a lock among millions of possible combinations. That's the challenge researchers face when predicting how strongly a peptide (a short amino acid chain) binds to a protein. This interaction is crucial for drug discovery, but computational models are often tested too simply.

A new study on arXiv put 10 peptide representation models, ESM-2 protein embeddings, and 6 regressors to the test on 11,349 peptide-protein pairs. The twist? They used three different test types: one with peptides similar to training data, one with known proteins but new peptides, and one with completely unseen proteins.

Results are striking: mean Spearman correlation ranges from 0.46 to 0.67 depending on the test. The best model shifts dramatically: with known proteins, a classic combo (ECFP fingerprints + Random Forest) wins, but for new proteins a transformer-based approach (HELM-BERT + Extra Trees) takes the lead. Also, representation differences matter most with limited training data and fade with more data.

Surprisingly, advanced techniques like PeptideCLM-2 adaptation or element-wise interaction features gave no consistent edge over simpler methods. The lesson? To truly evaluate these models, the test must match the real-world use case: a single score isn't enough.

The study integrates three databases (BindingDB, PDBbind, and a proprietary dataset) to obtain 11,349 unique peptide-protein pairs with quantitative affinities (Kd, Ki, IC50 transformed to pKd). The 10 peptide representations include: one-hot encoding, physicochemical properties, embeddings from protein language models (ESM-2, ProtBERT, PeptideCLM-2), and circular fingerprints (ECFP). For proteins, ESM-2 embeddings (33 layers, 1280 dim) are used. The 6 regressors: Random Forest, Extra Trees, XGBoost, LightGBM, MLP, and Ridge Regression.

Three partitioning strategies:
1. Peptide-similarity split: clusters peptides by similarity (MMseqs2, 30% identity), tests on unseen clusters.
2. Within-target split: same protein in train/test, different peptides.
3. Leave-target-out split: entire proteins excluded from training (exact-sequence split).

Key results: mean Spearman correlations 0.462 (peptide-sim), 0.669 (within-target), 0.530 (leave-target-out). Top configuration: ECFP-16 count + RF for first two, HELM-BERT + Extra Trees for leave-target-out. Representation rank correlation: -0.042 to 0.624 (low agreement), regressor rank correlation: 0.771-0.943 (high agreement). Learning curves: representation gap largest at 100 samples, smallest at 10k. Fine-tuned PeptideCLM-2 and Hadamard (element-wise product) features do not consistently improve over frozen encoder + concatenation.

Limitations: pooling of heterogeneous Kd/Ki/IC50, target exclusion at exact-sequence level (not family), only binary/quantitative affinity no kinetics. Implication: benchmarks must align partitions with intended use (e.g., design for new target vs optimization for known target).