Generative AI is a type of artificial intelligence that can produce new content including text, images, audio, and synthetic data. Large Language Models (LLMs) and Generative AI intersect, and they are both a subset of deep learning. But what exactly is an LLM?
At a high level, LLMs refer to large general-purpose language models that can be pre-trained and then fine-tuned for specific purposes. Let's break down exactly what that means.
• Large ➡️ This refers to a large training dataset. It also refers to a large number of parameters, which are often called hyperparameters in machine learning. Parameters are basically the memories and the knowledge that the machine learned from the model training. Because of the huge datasets and tremendous number of parameters required, only certain organizations have the capability to train these models.
• General Purpose ➡️ This means the models are sufficient to solve common language problems across industries. This approach works because of the commonality of human language regardless of specific tasks, and it helps overcome resource restrictions.
• Pre-trained & Fine-tuned ➡️ You pre-train a large language model for a general purpose with a large dataset. Then, you fine-tune it for specific aims with a much smaller dataset.
