If you have ever tried to run a massive Large Language Model (LLM) or a high-resolution computer vision model directly on a mobile device, you’ve likely encountered the "Thermal Wall." Your device gets hot, the frame rate drops, and the battery percentage plummets faster than a falling stone.
This isn't just bad code; it is a battle against the fundamental laws of physics.
When we deploy models like Gemini Nano to an Android device, we aren't just executing logic; we are moving massive tensors of floating-point numbers from RAM to the processor (GPU, NPU, or DSP) and back. In a standard training environment, models use FP32 (32-bit Single Precision Floating Point). For a model with just 1 billion parameters, that is 4GB of raw memory just to hold the weights. On a mobile device, this is catastrophic. It consumes massive amounts of RAM and the energy cost of moving that data across the memory bus will drain a battery in minutes.
The solution to this crisis is Quantization. In this guide, we will dive deep into the mechanics of Post-Training Quantization (PTQ), compare the industry standards of INT8 and FP16, and look at how to implement a production-ready quantization pipeline using modern Kotlin.
