transcribe.cpp: a ggml speech engine with 60+ models and local inference

The maintainer of Handy, an open-source dictation application, has published transcribe.cpp, a transcription library built on the ggml machine-learning library. The project is released at version 0.1.0 and supports what its author describes as all the latest transcription models. Every model published under the handy-computer organisation on Hugging Face has been numerically validated and WER tested to match its reference implementation.

Why another ASR stack

The project grew out of the difficulties of distributing a cross-platform speech-to-text application. According to the author, the practical options for on-device automatic speech recognition are whisper.cpp and ONNX, with MLX as a third path on Apple hardware, which forces developers to support several engines and port models to each of them. ONNX, he notes, leaves a great deal of performance unused because it runs on the CPU. Other libraries that claim broad model support often have unknown authors and unclear testing, which raises questions about long-term maintenance, bindings and benchmarks.

What the library offers

transcribe.cpp supports 16 ASR families, covering more than 60 models, with more planned. Acceleration is available through Vulkan, Metal, CUDA and TinyBLAS, and the project publishes benchmark runs for every supported model on a Ryzen 4750U with CPU and Vulkan under Fedora, as well as on an M4 Max. Both streaming and batch transcription are supported. The library is described as more or less a drop-in replacement for whisper.cpp: it can run the popular .bin model files, and its whisper implementation performs at roughly the same level. Bindings are maintained in Python, JavaScript and TypeScript, Rust, and Objective-C and Swift.

Making local speech-to-text more accessible

The author's stated goal is to make locally run recognition easier. He notes that a Rockchip RK3566, a low-cost embedded chip, can run models through transcribe.cpp faster than real time, and that state-of-the-art transcription runs in a handful of watts, so sending voice to a cloud service is not strictly necessary. The work was supported by Mozilla AI's BiR programme, ggml, Modal, Blacksmith and Hugging Face. The engine was written with AI assistance, the author adds, though the words in his announcement were not.