I wanted to deeper understand how modern image generation works, not by reading about it but by training one: a text-to-image model where every part that learns is written by me. The result is tinydit, a 210M-parameter diffusion transformer trained from scratch in 3.5 days on a single RTX PRO 6000, on 4.2M images at 256². It sits on two frozen pretrained pieces, the FLUX.2 autoencoder and the flan-t5-base text encoder; the transformer, the training recipe, the data pipeline and the evaluation are original. You can try it in the browser, download the weights, and read every decision with its sources in the repo.
This is the write-up of what mattered, in the order it mattered, with the numbers.
1. The data decided the outcome before training started
My first attempt at scaling, months earlier, used a recaptioned web crawl of about a million images. It made the model worse than COCO alone: the images were random internet quality and the captions did not fit them. So this time I did the unglamorous thing first: pulled twenty real rows from every candidate dataset, looked at them as a 256² model would see them, and only then chose.
The mix that trained this model:
