VIDRAFT Releases Aether-7B-5Attn: A Fully Open-Source MoE LLM with Five Heterogeneous Attention Mechanisms
TL;DR: Korean AI startup VIDRAFT has released Aether-7B-5Attn, a 6.59B-parameter Mixture-of-Experts foundation model on Hugging Face under Apache-2.0 — shipping not just weights but training data recipes, full training code, hyperparameters, training logs, intermediate checkpoints, and evaluation code. If you care about reproducibility, sovereign AI, or heterogeneous attention research, this one is worth a close look.
What it is
Aether-7B-5Attn is an open-source foundation LLM developed by VIDRAFT (비드래프트), a Seoul-based Pre-AGI AI startup. It was released on Hugging Face under the Apache-2.0 license and is positioned as a truly open model — not just "open weights" — following the philosophy of projects like Allen AI's OLMo, Apertus, and LLM-jp.
Key facts from the release:
