As AI-based systems integrate into every aspect of our lives, prompt security is becoming increasingly important. However, in my experience, I've found that not every AI application requires the same level of stringent security measures. Whether it's my own projects or a client project, adjusting the security level according to the application's risk profile is critical for both efficient resource utilization and maintaining development speed. Otherwise, we face an unnecessary operational burden.
The security level expected from a chatbot is certainly different from that expected from an AI-powered production planning module in an enterprise ERP system. In this post, I will explain how I approach AI prompt security for different scenarios, what risks I consider, and the practical lessons I've learned during this process.
Why Is Prompt Security Important and What Are the Risks?
AI prompt security means managing the risks that can arise from manipulating or misusing the inputs (prompts) given to language models. These risks vary depending on the application's purpose and the sensitivity of the data it processes. The main risks I generally encounter are:
Prompt Injection: Forcing the model to exhibit unwanted or malicious behavior by overriding its original instructions. This is when a user tries to hijack the model's internal logic to infiltrate the system or leak sensitive information.
