IBM Granite Switch 4.1 Combines Chat, Decisions, and Speech in One Model

One Endpoint for Chat, Decisions, and Speech

IBM has released Granite Switch 4.1, a modular model family that handles natural-language chat, general and specialized decision-making, and speech-to-text transcription from a single hosted endpoint. The Granite-Switch-4.1-8B and Granite-Switch-4.1-30B models combine a base model with three components: a new Decision aLoRA for general-purpose decisions, a specialized Guardian aLoRA for safety-related tasks, and Granite-Speech-5.0-Turbo-CTC, which the team describes as the fastest ASR model on the OpenASR Leaderboard.

Because the Switch adapters are built as cache-efficient Activated LoRAs, the KV cache created during a base model generation can be reused for any subsequent adapter-based decision. Combined with recent vLLM contributions for low-rank adapter inferencing and a co-located ASR encoder, the models can switch between chat, decision, and spoken requests at high speed. IBM says the full composition takes up approximately the same VRAM on a GPU as the base model alone, meaning more capabilities with fewer GPUs.

Safety Decisions and Real-Time Workloads

The Guardian aLoRA is designed for safety-related decision tasks and, on average, outperforms Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions, and Clef-Flash on those tasks, according to the team. The 8B model can transcribe audio, generate summaries, screen for harmful content, and test more than 15 generic flags, all running on a single endpoint in real time.

IBM also reports a head-to-head result in which Granite-Switch-4.1-30B, built from a frozen Granite base plus four specialized aLoRAs, completed a real-world five-step RAG workflow much faster than Jev-27B, which pairs a frozen Qwen base with a single generic decision LoRA. The team notes that because adapters share the base model's KV cache, adding specialist adapters alongside a generalist decision adapter costs almost nothing, giving broad coverage from the generalist and higher accuracy from specialists on targeted tasks.

Composing and Extending Switch Models

The released 8B and 30B models ship with a predefined set of adapters, but IBM says anyone can compose their own Granite Switch model by mixing and matching adapters from Granite Libraries, the repository of specialized decision and task adapters introduced with the original Granite Switch release. The team plans to release an Adapter SDK for training adapters in Mellea, along with tutorials and broader Granite Switch support.

As a demonstration, the team built a custom 3B parameter Granite Switch model that plays the classic game Doom. Similar to Jev, the model makes split-second decisions to navigate the game, but at 60 to 100 decisions per second compared with Jev's reported 10 decisions per second. Unlike Jev, the Granite Switch model can also narrate and answer questions while playing. IBM says more details and data on Granite Switch will be released in the coming weeks.

Sources: Hugging Face Blog