Researchers propose a Cognitive-structured Multimodal Agent that externalizes visual information into an Episodic Visual Memory to address the limitations of shared context windows in unified multimodal models. The system utilizes a Perceptual Abstraction Engine, a Cognitive Retrieval Engine, and a Multimodal Executive Controller to enable selective reactivation of relevant episodes during reasoning.
- The agent employs a Unified Scenario Engine to generate structured multi-turn conversations with fine-grained retrieval annotations for reinforcement learning.
- An 8B parameter version achieves 91.4% retrieval accuracy over 20-turn sessions, surpassing 32B baselines by +8.2%.
- Per-turn inference time is nearly halved from 23.1s to 12.7s compared to baseline models.
- The Cognitive-structured Multimodal Agent Harness (CMA-Harness) provides a tool-augmented deployment with persistent memory and OpenAI-compatible serving.
Structured memory and modular decision-making offer a more scalable and efficient paradigm for long-horizon multimodal agents than monolithic parameter scaling.