Deploying LocalAI Self-Hosted AI Model Management Platform on Ubuntu 24.04

LocalAI is an open-source platform for running Large Language Models locally with an OpenAI-compatible API, so you can swap it in behind existing OpenAI client code without paying per-token or sending data off-server. This guide deploys LocalAI using Docker Compose with Traefik handling automatic HTTPS, persistent model and cache directories, and a working chat-completion test. By the end, you'll have LocalAI serving an OpenAI-compatible API securely at your domain.

Set Up the Directory Structure

1. Create the project directories:

$ mkdir -p ~/localai/{models,cache}

$ cd ~/localai