Benchmark & Dataset
Weekly Digest

234 multi-session scenarios across research collaboration, shopping and personal finance, in two dialogue-length regimes.

Closed-pool grading distinguishes current state, superseded state and unrelated errors to isolate state drift.

StateMem raises current-state accuracy on DeepSeek-V4-Flash from 0.205 to 0.363 and improves six memory/retrieval backends by 32–67 percentage points.

Long-running agents often retrieve information successfully but act on values that are no longer valid.

30 samples per prompt and 5–15 counterfactual experiments per investigation, followed by independent verification.

Predictors judge whether an edit changes behavior, using measured experimental outcomes as labels.

Three activation-reading interpretability tools show no clear uplift, while training on counterfactual experiments improves held-out and OOD generalization.

It turns plausible explanations into predictions that experiments can support or falsify.

128 sandboxed cases across configuration, extension, runtime, persistence, action control and recovery.

Every trajectory is scored for Utility, Attack Success, Persistence and Detection.

Attack success ranges from 12.6% to 80.9% while utility remains between 75.0% and 97.6%.

It shows that recognizing an attack and actually stopping it are different capabilities—and that safety belongs to the model–harness pair.

About 617.5 hours of high-precision motion, including roughly 245.7 hours of HOI, with synchronized object trajectories, meshes, text and metadata.

Measures motion-space diversity, interaction grounding, object consistency and Physical AI applications.

It bridges the scale of internet video and the fidelity of laboratory motion capture.

33,826 instructions, 998 scenes and 16 placement families with direct, one-hop and two-hop references.

Predictions are inserted into the scene; relations are recomputed and checked alongside physical validity.

After fine-tuning, valid placement reaches about 68.3% for LLaVA-3D, 31.6% for 3D-LLM and 22.4% for PlaceIt3D.

Ground truth is an acceptable region rather than a single coordinate, better reflecting real spatial tasks.

Rule complexity and scene combinations are controllable, supporting systematic generation and unseen-rule tests.

Natural-language rules are translated into logic and executable validators that inspect the trajectory step by step.

Generated validators reach about 93.3% correctness before review; all final validators are human-verified.

Separating perception, rule understanding and planning makes model failures easier to diagnose.