Fine-tuning LLMs for financial natural language processing (NLP) is constrained by limited, imbalanced data. Real-world financial news overrepresents earnings and stock movements, while rarer events such as credit-rating changes, product approvals, and labor issues are harder to capture at scale. Synthetic generation can help fill those gaps for trading research, risk modeling, and surveillance, but diversity requires more than a single large generation run.
This post walks through an iterative pipeline for generating 500,000 unique financial news headlines across 12 topics plus an “Other” category. The workflow combines NVIDIA NeMo Data Designer for structured generation, NVIDIA NeMo Curator for semantic deduplication, and NVIDIA Nemotron models for high-throughput headline synthesis. It also adds a farthest-from-centroid few-shot strategy to steer each round toward novel outputs.
In a naive run of 50,000 headlines, 65% were removed as near-duplicates. Instead of scaling a single batch, the pipeline iterates: generate, filter, deduplicate globally, select distinctive few-shot examples, correct category weights, and repeat until the corpus reaches the target size. Our full run produced 502,536 unique headlines across 13 categories in 82 iterations, using approximately 6 days of compute on a single 8-way NVIDIA B200 node with checkpointing, crash recovery, and SLURM job chaining.
