Bridging The Language Gap: Evaluating And Enhancing Slovak Language Support in Large Language Models

Abstract

This study investigates the current level of Slovak-language support in large language models (LLMs) and proposed practical pathways toward high-quality, resource-efficient deployment. I benchmarked several state-of-the-art open-source and commercial LLMs on a newly created set of 100 Slovak questions covering grammar, semantics, style, slang, translation, and complex constructions. I evaluated the answers automatically with OpenAI GPT-4o-mini. Results show that Google Gemma 3 27 B achieves near parity with GPT-4o while running on a single high-end GPU, outperforming LLaMA 3.1 70 B by 27 percentage points in overall quality and cutting latency by a factor of four. My findings highlight Gemma 3 27 B as the best current trade-off for Slovak, while underscoring the strategic need for a dedicated Slovak LLM built on open resources.