You fine-tune a model to write better code. It becomes excellent at Python. It forgets how to write poetry. You fine-tune a model to diagnose medical conditions. It becomes excellent at radiology. It forgets basic history. This is catastrophic forgetting. Teach a model one new thing, and it forgets five old ones. The new knowledge overwrites the old. The model is not learning. It is trading.
This is the danger of fine-tuning. It is a double-edged sword. It makes the model better at a specific task. It makes the model worse at general tasks.
What Is Catastrophic Forgetting?
Catastrophic forgetting is a phenomenon in neural networks.
The Concept:
