Parallax: A Parameterized Local Linear Attention That Keeps Softmax and Adds a Learned Covariance Correction Branch

Parallax is a parameterized Local Linear Attention that keeps softmax, adds a learned covariance correction, and codesigns with Muon.

Raccontata damarktechpost.com