Back to Articles
We released a pretrained medical encoder. Here is how to run it, and how to finetune it for the tasks you actually have.
Introduction
Much of clinical NLP is about reading text, not generating it. You have a consultation note and you need the drugs, the pathologies, and the symptoms out of it. You have a pile of documents to group by specialty, or a patient record where a single passage matters. For all of this, encoders are often the right tool.
An encoder is a BERT-style model that reads the whole text at once and turns every token into a vector that captures its meaning, in the context of the text. A word like "patient" will have a different vector depending on whether it appears in "patient medical file" or "patient was treated". This token representation is what makes encoders effective on many tasks, such as entity recognition and classification tasks. The encoder needs to be fine-tuned to adjust those token representations by giving enough task-specific examples. And that is why we are going to walk you through in the blog.
