Part 1 left us with a model that talks back beautifully but without structure. This is fine, even pretty cool, for a chatbot used by humans. We understand natural language and fuzziness. Unstructured output is, however, terrible for software. You cannot parse the output into a typed object if the shape is different every time.
To build an application around a model, you must make it respect an interface. This part covers the two skills that turn model output into something you can build on: steering the model with prompts, and enforcing structured output.
Prompting: the new runtime configuration
A prompt (an input message you give to the model) is best understood as runtime configuration. The model responds to the information, constraints, and examples you provide. Prompting the right way is therefore crucial for taming the model's output. Entire books have been written about prompt engineering. Here are a few quick principles:
Use the system prompt. The first message the model receives is often not the user's first message, but a system prompt that precedes the conversation. In chat interfaces such as ChatGPT, the system prompt is invisible to the user, but it is sent to the model on every turn along with the conversation history. Use it to define the model's overall behavior by specifying rules, constraints, and tone that should apply to every request. For example, "Do not invent fee figures if the knowledge base contains no relevant data" is a business rule that belongs in the system prompt.
