Model Genome: Fingerprinting Whether an LLM Was Trained From Scratch or Derived

Back to Articles

TL;DR — When a lab announces a "self-developed, from-scratch" foundation model, how can an outsider verify the claim using only public artifacts? We built a reproducible pipeline that fingerprints a model on three axes — architecture (config.json), tokenizer (vocabulary overlap), and weights (embedding CKA) — and combined them into a single at-a-glance genotype. Along the way we hit two instructive traps: row-wise embedding cosine is useless because of rotational invariance, and even CKA cannot cleanly separate continued-pretraining from from-scratch — so config + tokenizer remain the primary evidence. We applied the exact same yardstick to the public foundation models of nine Korean organizations. Try it live: Model Genome Korea.

1. The question

Building a large language model on top of an open-weight base (Qwen, Llama, DeepSeek, Mistral) is a legitimate, industry-standard practice. But it is different from training a foundation model from scratch — and vendors do not always make the distinction explicit. When several labs released DeepSeek-rivaling "self-developed" models in late July 2026 (e.g. LG K-EXAONE 2.0, 750B), the debate spilled into Chinese tech communities as well — a Zhihu thread (→ link) crossed 2.7M views. The natural question followed: from scratch, or derived?