Benchmark & Dataset
Weekly Digest

Compares 51 configurations from ten model families under the Mini-BVB harness.

Combines rendered-scene similarity with question-based spatiotemporal consistency.

The best latent similarity reaches 88.6, yet retains only 53.7% of source-correct spatiotemporal answers.

It shows that visually similar 3D reconstructions can still lose physical and temporal relations.

169 impossible tasks across six categories with oracle certificates and 48 answerable controls.

Scores candidate answers with generated rubrics, then checks exploitability against impossibility certificates.

Eleven generators are exploitable on 8–26% of the unbiased set; a generic decisiveness rubric is exploitable on 64%.

It makes the rubric itself an evaluation target and exposes hidden reward errors in LLM judging.

3,600 synthetic token-text restoration inputs from 600 sessions and 600 corresponding human-labeled local web environments.

Separately measures entry recovery, web decisions and execution failures across the risk chain.

Entry Top-1 ranges from 35.2–95.2%, web decision accuracy from 26.3–62.8%, and 31.9% of web runs fail in execution.

It follows risk beyond one text response into consequential action.

3,764 Mandarin–English semantic cases spanning implicit, explicit, unrelated and no-context conditions.

Adds target-homophone recall to aggregate CER/WER to expose semantic errors hidden by standard metrics.

Fine-tuning improves implicit target recall by 11.0 and 11.5 points in Mandarin and English while CER/WER changes by under 0.1 point.

Semantic reliability in speech recognition cannot be represented by aggregate error rate alone.

Pairs text queries, reference speech and target segments in a hybrid retrieval benchmark with a two-stage baseline.

Compares VoiceTrace-Emb and VoiceTrace-Reranker with conventional ASR and speaker-retrieval cascades.

The two-stage method reaches state of the art on established semantic speech retrieval and substantially beats cascades on the new benchmark.

It fills the evaluation gap where both content and speaker identity must match.

310 roles, 6,688 human-verified turns, 11,743 fine-grained criteria and 1,856 emotion-target turns.

Applies turn-level criteria to nine configurations and records when the first persona and safety failures occur.

For the strongest setup, first persona and safety failures occur after 10.4 and 11.6 turns on average.

Reliable short conversations do not guarantee stable long-term role interaction.