Foundation models are reshaping computational biology. Pretrained on massive corpora of protein or genomic sequences, models such as ESM2 (a protein language model) and Evo 2 (a DNA language model) capture statistical regularities of biological sequences. These transfer well to a wide range of downstream tasks, including structure prediction, variant effect, and functional annotation.
Yet adapting these models to a specific task is nontrivial: at billions of parameters, full fine-tuning quickly becomes impractical, both in compute and storage of optimizer state and checkpoints.
Low-Rank Adaptation (LoRA) directly addresses this challenge. By keeping the pretrained backbone frozen and training only a small set of low-rank adapter matrices, LoRA can match full fine-tuning quality on many tasks while training ~1% of the parameters, fitting a single billion-scale model and its adapter state on a single workstation GPU.
To reduce the difficulty of building these workflows, NVIDIA BioNeMo Recipes provide step-by-step training recipes built on familiar PyTorch, Hugging Face, Megatron-Bridge patterns. Performance-oriented components such as NVIDIA Transformer Engine (TE) and scale-out strategies are integrated where they pay off, but the recipes themselves stay readable.
