Auditing Protocol-Level Shortcuts in Large Audio Language Model Judges for Speech Evaluation
arXiv cs.CL2w4 min read
arXiv:2607.13477v1 Announce Type: cross Abstract: Large audio-language models (LALMs) are increasingly used as automatic judges for speech evaluation. However, high agreement with human ratings does not guarantee that their verdicts are grounded in the audio. A judge may instead rely on specialist labels or reference data supplied by the evaluation protocol itself, taking a shortcut in place of listening to the audio. In this paper, we audit such protocol-level ``shortcuts'' in LALM judges across three common deployment protocols: feature-blueprint judging, where the audio is replaced by a str