Gemma 4 on Android: Tricks for Faster On-Device Inference

When I tried building an on-device AI app with Gemma 4, the pitch was clear: model weights on the device, no server, no API calls, works offline. Getting it to actually run fast was a different problem.

This post covers what I learned working with LiteRT-LM 0.12.0 and Gemma 4 E2B on Android in Kotlin. Some of it is configuration. Some of it is understanding what the bottleneck actually is before reaching for a fix. If you're building with Gemma 4 E2B on Android and inference feels too slow to ship, here are the tricks that actually helped.

1. Basic Setup

Add the dependency:

// build.gradle