Apertus 1.5 Brings Multimodal Understanding and Longer Context to Open 8B and 70B Models

Key Enhancements

Apertus 1.5 extends the 8B and 70B open models with native multimodal understanding, accepting images and audio alongside text. The models feature an optional thinking mode, a 4x longer context window at 262,144 tokens, improved instruction-following, and better tool use. The model weights were released to Hugging Face on July 24, 2026 under an Apache 2.0 license.

Multimodality is introduced via early fusion: images are tokenized with the Emu3.5 tokenizer and audio with WavTokenizer, entering as discrete tokens in the same sequence as text. The models understand images and audio but generate only text.

Training Process

The models underwent continued pretraining with 4 trillion additional tokens for 8B and 2 trillion for 70B, bringing totals to 19T and 17T respectively. Pre-training ran in four stages: unpaired (text, images, audio), paired (image-caption and audio-transcript pairs), cooldown (higher-quality data with math and code), and long-context extension from 8k to 262k tokens. More than half of all training tokens remain text-only, preserving language abilities from Apertus 1.0.

Post-Training and Alignment

Post-training followed five stages: SFT on 3.7M text samples plus image and speech data, long-context SFT at the full 262k window, RL with verifiable rewards on about 350k prompts, DPO with judge-scored pairs, and SDPO for charter alignment. Reasoning traces were added only to prompts the model could not already solve without them. Tool-use data was restricted to execution-verified trajectories. Intermediate checkpoints and curated datasets are publicly available on Hugging Face.

Evaluation and Reproducibility

Evaluation expanded to include multimodal and thinking-mode benchmarks. Apertus 1.5 8B outperforms similarly sized open-weight models on instruction-following and multilingual tasks, with competitive multimodal performance in spatial and document understanding. The project collaborates with the EvalEval Coalition, publishing results in the Every Eval Ever schema that records run conditions and generation settings for full transparency.

The models are available through providers including Swisscom, Infomaniak, and Phoeniqs, and have surpassed five million downloads on Hugging Face. At the AI+X Summit in Zurich, the team announced the Apertus 2.0 roadmap targeting a sparse/MoE architecture with approximately 600B total parameters, with agents, mathematics, and coding as priorities.

Sources: Hugging Face Blog