Build an On-Device LLM Chatbot with Kotlin and TensorFlow Lite

Large language models are usually accessed through cloud APIs, but modern Android devices can also run smaller AI models locally. This makes it possible to build applications that work offline and keep sensitive prompts on the device.

In this tutorial, we will design the architecture of an on-device LLM chatbot using Kotlin and TensorFlow Lite. The focus is on the mobile integration layer, model execution, prompt handling, and performance considerations.

What We Will Build

The application will have:

Build an On-Device LLM Chatbot with Kotlin and TensorFlow Lite

Related reading

Build a Local LLM Chatbot with Ollama and Python

Build a Federated Learning System on Android with Kotlin

A local AI chat app in a single HTML file. No build, no sign-up.

Building a 3-tier on-device AI concierge: Gemini Nano -> MiniLM -> keyword,…

Finally, a dev kit for designing on-device, mobile AI apps is here: Liquid AI's…

Building an On-Device Q&A Agent with Chrome Built-in AI and the Prompt API