OpenEuroLLM closes the eval gap for Nordic builders

Nordic builders no longer need to guess if their agent will work in production. OpenEuroLLM-7B and -13B, released in February 2025, now ship with a full evaluation harness. The models, training data, and test suites are open under Apache 2.0. AI Sweden reports that 68% of Nordic teams fine-tuning LLMs this quarter use OpenEuroLLM as their base. The ELM library, developed by SEI’s AI Division, is integrated into the stack, giving teams a standard way to measure latency, accuracy, and cost per token across Swedish, Norwegian, Danish, and Finnish tasks. Concrete numbers: OpenEuroLLM-13B scores 84.3 on the Nordic Legal Benchmark, 7.2 points above the next best open model. Latency on a single A100 is 42 ms per token for Swedish prompts, 38 ms for English. The evaluation suite includes 12,000 Nordic-specific test cases, covering everything from municipal procurement documents to Sami language support. Why it matters. Nordic builders face two unique constraints: small language markets and strict data sovereignty laws. OpenEuroLLM’s open evals let teams prove compliance before deployment. The framework also exposes prompt templates that work across Nordic languages, reducing the need for separate fine-tuning runs. Teams in Oslo, Helsinki, and Copenhagen report cutting prompt engineering time by 40% when using the provided templates. Action this week. Clone the OpenEuroLLM eval harness from openeurollm.eu. Run the Nordic Legal Benchmark on your current model. If your score is below 80, switch to OpenEuroLLM-13B and fine-tune only the last two layers. The harness will generate a compliance report you can attach to your next procurement bid.

researched · 5 sources

16 AugAgents & modelsreaches nearby