Next-word prediction for a keyboard app: 31% smaller model, 99% accuracy
An open-source Android keyboard for programmers just shipped a next-word prediction engine that fits in 22 MB while retaining 99.2 % top-1 agreement with its larger predecessor. The feat came from combining a streaming pipeline, Kneser-Ney smoothing, and a 64 K-word vocabulary cap.
From 583 MB to 22 MB
The keyboard’s trigram model was trained on 4.27 million lines of blogs, news, and Twitter text. Early attempts to build the model on AWS consumed all available memory and were killed after hours; the fix was a streaming SQLite pipeline that merges counts on disk instead of in RAM, cutting the build from hours to 11 minutes on a spot t3.xlarge instance.
Smoothing and model pruning
Three algorithm variants were compared. Kneser-Ney scored low for generic words like “Francisco” that appear often but in few contexts, making it good for cold-start guessing but harder to tune under size constraints. Katz backoff with Good-Turing discounting proved more stable when aggressively pruned. SwiftKey’s WDP variant then trimmed candidates that didn’t add predictive value beyond a bigram model, yielding a 31 % smaller file with 99.2 % top-1 agreement—so WDP shipped.
Vocabulary by the numbers
The raw model’s 427,651-word vocabulary follows Zipf’s law: 85 % of words appear fewer than 15 times in 85 million tokens. Testing vocabulary caps at 16 K, 32 K, 64 K, and 128 K showed that a 64 K cap already covers 99.3 % of real next-word targets, so the team shipped with 64 K.
The final footprint
Everything—vocabulary, character trie, n-gram follower lists, and phrase data—now lives in a single 22 MB .cklm file loaded via memory-mapping at startup. The APK shrank from 302 MB to 94 MB, and three separate JSON files were retired.
Why it matters
For mobile apps, model size directly affects startup time, memory use, and battery life. By cutting the next-word predictor to 22 MB without meaningful accuracy loss, this project shows how careful smoothing choices, streaming pipelines, and vocabulary capping can make sophisticated language features feasible on low-end devices. It also offers a reproducible blueprint for teams balancing linguistic sophistication with hardware constraints.
Source: DEV Community. AI-assisted editorial synthesis — TechnoExpress.