Research is crucial for progress and innovation, which is why at JetBrains we are passionate about both scientific and market research
ResearchOur Research on Membership Inference Attacks and Preventing Privacy Leaks
There’s a stranger out there who has nothing but API access to your chatbot. They are interested in knowing whether a specific patient, employee, or customer appears in the data you trained it on. Without breaching the database or stealing backups, this person can theoretically figure out this information with carefully crafted prompts and a bit of patience.
Depending on model size and susceptibility to overfitting, this approach can pose a severe risk to user privacy. We think that this privacy risk is greatly underestimated, especially for fine-tuned models, and that current detection methods are insufficient. And at JetBrains, we take user data and privacy very seriously.
In this post, we discuss this important privacy risk. In particular, we:
