Paper summary
Paper summary: RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversations
RealCompanion releases ten real long-term human–AI companion relationships (27,218 messages over up to 120 days) and finds that people rarely refer back, that AI can't tell when the past matters, and that it reads more into people than they revealed.
- Only 3.4% of messages depend on something said earlier, and then usually about 2,000 messages back.
- Calling earlier messages "memories" made models bring up the past 10–14 points more often, even when it wasn't needed.