Locking Pretrained Weights via Deep Low-Rank Residual Distillation

The quality of open-weight language models has dramatically improved in recent years. Sharing weights greatly facilitates model adoption by…

Raccontata damachinelearning.apple.com