LLaMA is a new open-source language model from Meta Research that performs as well as closed-source models. Similar to Stable Diffusion, there’s been a ton of experimentation and innovation since the model was publicly released. As Simon Willison articulated, LLaMA is easy to run on your own hardware, large enough to be useful, and open-source enough to be tinkered with.
LLaMA is powerful, but it was not built for answering questions. It functions more like a fancy version of autocomplete than a conversational bot. This is where Stanford’s Alpaca comes in. Alpaca is a fine-tuned version of LLaMA that can respond to instructions like ChatGPT. And, like LLaMA, it’s open-source.
The problem is, the weights for Alpaca have not been released, so you can’t tinker with it. We do have all the component parts we need to replicate it though: the LLaMA weights, the training data, and the training script.
In this post we’ll show you how to train Alpaca so you can tinker with it on your own machine.
Note: LLaMA and anything built on LLaMA is for research purposes only. You can’t build anything commercial with it.
