A Blog post by ggml-org on Hugging Face
llama.cpp now supports multiple small OCR models, including GLM-OCR and Deepseek-OCR, optimized for 4GB GPU VRAM and CPU. On-device OCR eliminates cloud dependencies, reducing cost and latency for document processing while maintaining data privacy.
Back to Articles
Supported OCR models Quick start Example server usage Tips and tricks Input prompts Quality and performance Halucination and incorrect results Conclusion
llama.cpp now supports various small OCR models that can run on low-end devices. These models are small enough to run on GPU with 4GB VRAM, and some of them can even run on CPU with decent performance.
In this post, I will show you how to use these OCR models with llama.cpp.
Supported OCR models
Getting Started: Run Your First Local LLM in 5 Minutes
Running Gemma 4 on a Modest Machine: Unsloth vs LM Studio vs llama.cpp vs Ollama
Gemma 4 vs GPT-4o vs Llama 3: What Actually Works Locally?
Train and run Stanford Alpaca on your own machine – Replicate blog
Run Llama 2 with an API – Replicate blog
Real-time video classification with PaliGemma: architecture patterns for low-latency VLM inference
A guide to prompting Llama 2 – Replicate blog
Run 30,000+ LoRAs on Hugging Face with Replicate – Replicate blog
Fine-tune Llama 2 on Replicate – Replicate blog
Fine-tune LLaMA to speak like Homer Simpson – Replicate blog
Tested Gemma-4 E2B-it on Raspberry Pi 4. the way to convert Gemma-4 E2B-it to gguf Quantizing...
A Blog post by ggml-org on Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
First steps: The usual first step with getting Gemma 4 running on Ollama is to pull the...
In April 2026 Google shipped Gemma 4, a multimodal model with a native audio path. I wanted to add it...
A fast TUI, CLI, daemon, and OpenAI-compatible proxy for running local LLMs via llama.cpp, in one Rust binary
