Pass or Fail? Evaluating LLMs on Two Greek Examination Benchmarks
This study addresses the absence of evaluation benchmarks for Greek large language models and the inadequacy of traditional metrics in assessing complex reasoning. We introduce two benchmarks, Prot-Ex and Pan-Ex, employing an LLM-as-a-Judge paradigm and textualized visual context techniques to systematically evaluate diverse models on academic tasks. Our analysis reveals a few-shot prompting paradox and context overload phenomena in smaller models, demonstrating that localization adaptation can effectively compensate for parameter disadvantages. Experimental results indicate that KriKri-8B achieves performance comparable to larger models on humanities tasks, validating the efficacy of domain-specific linguistic adaptation. This work establishes a novel evaluation paradigm for low-resource language models.