Trang chủ / Tin tức / Khoa học & công nghệ
Khoa học & công nghệ

Persona Conditioning as an Assessor-Sensitivity Probe for LLM-Based IR Evaluation

03/11/2026 · Thành viên: Roy Nguyễn · Nguồn: OpenAlex

Large language models (LLMs) are increasingly used as relevance assessors in information retrieval (IR) evaluation, raising questions about how assessor framing affects judgment reliability and dow...

<p><strong>Tác giả:</strong> Samaneh Mohtadi, Pietro Bernardelle, Joel Mackenzie, Gianluca Demartini</p>

<p>Large language models (LLMs) are increasingly used as relevance assessors in information retrieval (IR) evaluation, raising questions about how assessor framing affects judgment reliability and downstream system comparison. We study persona conditioning as a diagnostic mechanism for exposing LLM assessor sensitivity. Using task-oriented personas drawn from two complementary sources (PersonaHub and NVIDIA Nemotron-Personas-USA), we instantiate five assessor roles emphasizing intent interpretat...</p>

<p>📄 DOI: <a href="https://doi.org/https://doi.org/10.1145/3799682.3841131" target="_blank" rel="noopener">https://doi.org/10.1145/3799682.3841131</a></p>

<p>📖 <a href="https://doi.org/https://doi.org/10.1145/3799682.3841131" target="_blank" rel="noopener">Xem bài báo đầy đủ</a></p>

Bài viết được biên soạn lại bởi LIFVietnam. Nguồn tham khảo: OpenAlex