FirsthandTech
arXiv: cs.AI preprintsInternational20 November 2026

The Effectiveness and Durability of Pretraining Data Filtering in Protein Language Models

This is an official announcement record

Firsthand records what arXiv: cs.AI preprints announced and links to the original. The wording below is theirs, not ours.

arXiv:2610.26651v2 Announce Type: replace-cross Abstract: Pretraining filtering is one of the most common safeguards for open-weight language models. Here we study its effectiveness, durability and scalability using the ESM-2 family of protein language models. We find that filtering viral proteins from the pretraining data meaningfully degrades masked language modelling, mutation effect prediction and long-range contact prediction on viral proteins, while largely preserving performance on non-viral proteins. These effects are concentrated among proteins from human-infecting viruses, and grow a
Source: arXiv: cs.AI preprints

More from arXiv: cs.AI preprints

This content is for informational purposes only and is not professional advice. Specifications, prices, plan tiers, and features change frequently and may differ from what is shown here; verify current details on the manufacturer's or company's official page before purchasing. Ratings are based on analysis of published documentation, not independent lab testing.