Whistle transcribes seven languages. needle whistle finetune adapts it to your speakers and vocabulary, or teaches it a new language, on your own machine. needle whistle build turns the result into a .cact that the same engine runs.
Install
pip install "cactus-needle[train]"Data
Three JSONL files, for training, validation and test. Each line is one JSON object: a WAV file, its transcript and its language.
{"audio": "clips/0001.wav", "text": "Quero reservar uma mesa para dois.", "language": "pt"}audiois relative to the JSONL file. Clips over 30 seconds and empty transcripts are skipped.languageis a language code. A code Whistle knows adapts that language; any other code adds a new one.- To keep the other languages as they are, mix some of their clips into the training file.
New characters
Characters the tokenizer lacks, such as Portuguese ã and õ or any Cyrillic letter, are split into bytes. List the ones your language uses in tokens.txt, one per line, and pass it with --tokens. Each one becomes a single token.
ã
õ
ç
â
êTrain and build
needle whistle finetune train.jsonl validation.jsonl test.jsonl --tokens tokens.txt --out adapter.safetensors
needle whistle build --lora adapter.safetensors --out tuned.cactThe first run downloads the base checkpoint. Training prints the training and validation loss after each epoch and ends with the word error rate on the test set. That number is a quick greedy decode on the 4-bit weights; for real-world numbers, build the .cact and transcribe the test set with it.
The defaults are 3 epochs, batch 16 and learning rate 1e-4. Treat them as a starting point: try a few values of --epochs, --lr and --batch-size on your data, and keep the run with the lowest validation loss.
build merges the adapter and writes a .cact that carries your languages and tokens. It is about 29 MB, against 16.9 MB for the shipped model, because local fine-tuning exports at 4 bits instead of 2.
Run
import needle
needle.transcribe("clip.wav", weights="tuned.cact")Streaming, keywords and word timestamps work as they do with the shipped model.
How much data
A new language has to learn its tag and its characters, so it needs more steps or a higher learning rate than adapting a language Whistle knows. Brazilian Portuguese from about ten hours of FLEURS, at --lr 1e-2 with the other defaults, went from 106.2% to 30.5% word error rate on FLEURS test and was detected as Portuguese on every test clip.
Two new languages in one run need enough data each before detection tells them apart. Scripts the base has never seen, such as Cyrillic, Arabic, Chinese, Japanese and Korean, need far more data before the text is usable.