Towards Reliable Multi-Agent Systems for Marketing Applications via Reflection, Memory, and Planning

The research focuses on addressing the limited reliability of AI agents developed using large language models (LLMs) in real-world applications. It presents RAMP, a multi-agent framework tailored for marketing tasks like audience curation. RAMP operates by iteratively planning, utilizing tools, verifying outputs, and generating suggestions for improving audience quality. A key feature of RAMP is its integration of a long-term memory store, which functions as a knowledge base containing client-specific information and past queries. The study demonstrates that employing LLM planning and memory within this framework boosts accuracy by 28 percentage points across 88 evaluation queries. Furthermore, the paper highlights the impact of iterative verification and reflection on ambiguous queries, showing an approximate 20 percentage point increase in recall with more iterations and higher user satisfaction. These findings offer valuable practical insights for the deployment of reliable LLM-based systems in dynamic, industry-specific environments.