MemCollab: Cross-Agent Memory Collaboration via Contrastive Trajectory Distillation

MemCollab: Cross-Agent Memory Collaboration via Contrastive Trajectory Distillation: https://arxiv.org/abs/2603.23234 Large language model (LLM)-based agents rely on memory mechanisms to reuse knowledge from past problem-solving experiences. Existing approaches typically construct memory in a per-agent manner, tightly coupling stored knowledge to a single model's reasoning style. In modern deployments with heterogeneous agents, a natural question arises: can a single memory system be shared across different models? We found that naively transferring memory between agents often degrades performance, as such memory entangles task-relevant knowledge with agent-specific biases. To address this challenge, we propose MemCollab, a collaborative memory framework that constructs agent-agnostic memory by contrasting reasoning trajectories generated by different agents on the same task. This contrastive process distills abstract reasoning constraints that capture shared task-level invariants while suppressing agent-specific artifacts. We further introduce a task-aware retrieval mechanism that conditions memory access on task category, ensuring that only relevant constraints are used at inference time. Experiments on mathematical reasoning and code generation benchmarks demonstrate that MemCollab consistently improves both accuracy and inference-time efficiency across diverse agents, including cross-modal-family settings. Our results show that the collaboratively constructed memory can function as a shared reasoning resource for diverse LLM-based agents. MemCollab introduces a cross-agent memory collaboration framework for multi-agent AI systems, solving memory entanglement in shared LLM memory. It uses contrastive trajectory distillation to extract model-agnostic reasoning constraints from episodic logs. The system stores distilled knowledge in structured task taxonomies and enables task-aware retrieval, improving accuracy, inference speed, and token efficiency. Benchmarks show consistent gains across mathematical reasoning and code generation tasks, allowing heterogeneous large language models to share memory without degradation. This approach replaces naive vector memory transfer with scalable, efficient multi-agent knowledge sharing for advanced AI infrastructure and collaborative reasoning systems. TimeStamps: 0:00 Multi-Agent AI Memory Architecture 0:22 Naive Memory Sharing and RAG Limitations 0:48 Cross-Agent Memory Performance Drop 1:10 Memory Entanglement in LLM Systems 1:50 Reasoning Trajectory Differences Across Models 2:26 Task Invariance vs Stylistic Artifacts 2:42 MEM Collab Framework Overview 3:01 Contrastive Trajectory Distillation Process 3:25 Abstract Reasoning Constraint Extraction 3:45 Agent-Agnostic Memory Storage System 🤖 multi-agent AI systems and shared memory 🧠 memory entanglement and reasoning conflicts 📊 contrastive trajectory distillation 📄 model-agnostic reasoning constraints 🗂️ structured memory taxonomy and storage ⚡ task-aware retrieval and token efficiency 📈 benchmark improvements in accuracy and speed 🔗 cross-model compatibility and scaling ⚙️ AI infrastructure and collaborative reasoning High-performance AI systems now depend on structured memory, not raw logs. Cross-agent knowledge sharing improves inference efficiency, reduces compute cost, and increases scalability. Engineers who implement contrastive distillation and task-aware retrieval gain measurable advantages in multi-agent workflows, where clean reasoning signals outperform noisy memory transfer across heterogeneous models. #MultiAgentAI #LLMMemory #AIResearch