What Changed
Traditional computer vision models often require specialized architectures and training for different tasks, such as segmentation, depth estimation, or image generation. This fragmentation necessitates developing and maintaining distinct models for each specific application, hindering the development of truly general-purpose vision systems. The "Let RGB Be the Language of Vision" paper introduces a novel framework, RGB In and RGB Out (RINO), which fundamentally alters this paradigm.
RINO proposes a unified formulation where all forms of visual information—including natural images, masks, depth maps, and other structured visual signals—are uniformly represented as RGB images. Consequently, diverse visual tasks are reframed as a common RGB-to-RGB image editing problem. This approach means that different types of visual data share the same internal encoding and decoding architecture and parameters as natural images. The core change is the establishment of a single, transferable model capable of handling a wide array of vision tasks through a standardized visual interface, drawing a parallel to the operational principles of language models over text.
This unification simplifies the model development process by eliminating the need for task-specific fine-tuning. Instead of training separate models or extensively adapting a base model for each new task, RINO leverages a generic image editing backbone to achieve zero-shot performance across various vision challenges. This shift represents a significant step towards more generalized and efficient vision-language systems.
