featureMay 18, 202610 mins
In the beginning, everyone was surprised that large language models (LLMs) could speak in words. Those days are long past, and now the focus is on the depth of knowledge. The best way to deliver this is with specialization. Instead of developing a one-size-fits-all leviathan, the best teams are building specialized models for niches—one for the doctors, one for the lawyers, one for the bankers, and so on. The trend won’t end. Soon, orthopedic surgeons who do shoulder replacements may have one model for right-handed patients and another model for the left-handed ones.
The trend toward specialization is driven as much by efficiency as quality. Focused models are smaller, and smaller models cost less to run. Indeed, some of the most prominent large models are really collections of small models that are unified by “mixture of experts” algorithms.
Training the focused models can also be cheaper, at least once there’s a solid training corpus. There’s no reason to burn a supertanker filled with oil just to teach a legal LLM the details of 17th century French poetry or the mating habits of river otters. As the kids say, “Skip to the good parts.”
Creating the training corpus, though, can be a challenge. Many of the teams are hiring their own experts to build out ontologies and double-check the answers. They’re relying on humans to make sure the facts are solid and backed by trustworthy references. When LLMs were new, users would forgive a few hallucinations. That won’t fly with users who have serious questions like legal or medical decisions.
