Attention Mathematics: Encoder-Only vs Decoder-Only vs Encoder-Decoder LLMs

Hello, I'm Shrijith Venkatramana, and I'm building LiveReview — a blast-radius aware AI code review built for your business-critical systems. Star us to help devs discover the project, give it a try, and share your feedback to help improve the product.

In 2017, eight researchers published a paper with an almost provocative title: ""

Attention Is All You Need.

They were not proposing another small improvement to recurrent neural networks. They were removing recurrence itself.

That decision eventually became the architectural foundation for BERT, GPT-style models, T5, and most modern large language models.

Attention Mathematics: Encoder-Only vs Decoder-Only vs Encoder-Decoder LLMs

Related reading

Multimodal Transformers: How LLMs Learn to See

Adam and AdamW: The Optimizer That Made Modern LLM Training Possible

BPE-Style Tokenizers: The Small Algorithm That Decides What an LLM Can See

Attention Mechanisms in LLMs: The Idea That Changed AI Forever

Reward Hacking in LLMs: When the Model Learns to Win the Game Instead of Doing…

LLMs Don't Have to Generate One Token at a Time: How Medusa and Multi-Token…