Jan – Sep 2026
DynamicMem
A Long-Horizon Memory Benchmark in Real-World Settings
Co-authored an LLM evaluation benchmark for long-horizon memory, designing evaluation methodology, analyzing model failure modes, and synthesizing findings to improve reliability in conversational foundation-model systems.
LLM evaluation research
Research contribution
- Co-authored DynamicMem, a long-horizon memory benchmark with synthetic user histories spanning 15 months across 16 applications, where agents infer changing attributes, habits, and preferences from scattered activity.
- Contributed evaluation methodology and failure analysis. The benchmark tests profile reconstruction and personalized assistance at five quarterly checkpoints as each user’s history grows.
- The study compared five memory systems and traced over 93% of failures to retrieved evidence, revealing difficulty retaining stable facts while replacing information that has changed.