The Progress Illusion: Revisiting meta-evaluation standards of LLM evaluators

This paper challenges current practices in evaluating LLM evaluators and introduces new standards for meta-evaluation that better capture real-world evaluation scenarios.

Recommended citation: Vedant Gaur et al. 2025. The Progress Illusion: Revisiting meta-evaluation standards of LLM evaluators. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP). #