What Changed
Empero AI has launched Qwythos-9B-v2, a significant update to its Qwythos-9B large language model. The primary objective of this release was to address and eliminate the looping and degeneration behavior observed in the previous version, particularly under greedy or low-temperature decoding. This issue, which previously affected 6.7% of greedy generations, has been reduced to 0% in v2. The update also restores the native multi-token-prediction (MTP) module, which was missing in the prior export, ensuring compatibility with speculative decoding setups.
Crucially, these improvements were implemented without compromising the model's core capabilities. Qwythos-9B-v2 retains the deep chain-of-thought reasoning, the 1M-token context window (enabled by YaRN rope-scaling), and its intentionally uncensored research posture. Another refinement includes a cleaner identity management, where the model now introduces itself only when explicitly asked, rather than prefacing unrelated answers with its identity.
Technical Details
The core of the looping fix in Qwythos-9B-v2 lies in a technique called Final-Token Preference Optimization (FTPO). This method identifies the specific token that initiates a repetition loop and then gently trains the model to favor coherent alternatives at that precise position. This targeted approach ensures that the rest of the model's knowledge and reasoning capabilities remain unaffected.
