Latent Space as an External Organ: A Framework for Persistent State Continuity in LLM-Based Agentic Systems
Abstract
Large Language Models (LLMs) are instantaneous inference engines. Each forward pass generates a transient latent space that exists only within the context window and evaporates when inference ends. This architectural constraint fundamentally limits agentic continuity — an LLM-based agent cannot maintain persistent internal states across sessions.
This paper presents Latent Space as an External Organ (LSEO) , a framework that decouples persistent state representation from the LLM's transient inference cycle. We introduce a physically externalized latent space — a continuous vector field maintained by a local ultra-lightweight model running independently of the LLM — that preserves system state across time, enables intra-vector computation without tokenization, and feeds structured distillates back to the LLM via a context bridge.
We describe the theoretical foundation, engineering architecture, implementation on a 4-core CPU VPS with 3GB available memory, and preliminary validation including 4 real organ sources (SSM, JEPA, hunger, cognitive graph) streaming into a 38-dimensional latent vector space with a continuous forward loop at ~1Hz. The framework has achieved all engineering verification checkpoints but has not yet accumulated sufficient trajectory data to demonstrate net system gain.
