Daniel was up late on Hugging Face again, and he came away with a taxonomy problem.
A taxonomy problem. That's how you're opening this.
He was browsing the text section, and what struck him is that there are two worlds sitting inside the same foundational task. Text in, text out. On one side, the famous ones. Llama, Gemma, Mistral, DeepSeek. The one that generates our own episodes, apparently. Increasingly multimodal, increasingly the thing people mean when they say "an AI model." On the other side, nested under the natural language processing filters, are the small models. The ones with names like BERT, which Daniel says sounded to him, the first time he encountered them, like legacy models of yesteryear.
And he's not wrong that they sound that way. That's the perception.
Right. So his questions, and there are several. Did the transformer architecture render these text processors redundant? Are they still being actively developed? How many are out there? What tasks do they actually perform? And how do they function as accessories to large language models, chained into pipelines where the small model is rarely the flashy part being advertised? He flagged ASR post-processing as the easiest place to spot them in the wild, and then said that's probably just the tip of the iceberg.
That's a lot of question for one late-night browsing session.
He also wants us to trace the history. How these tools grew out of the maturing world of NLP, and then when AI blossomed, it felt like they'd been smuggled in through the back door, when really we're looking at the successors to the very foundations of token prediction.
Well. The first thing to do is define the two worlds properly, because "small language model" gets used as if it means "mini version of a big language model," and mostly it doesn't.
Go on.
The famous side is decoder-only. You give it a prompt, it generates text, token by token, and that one capability fans out into conversation, code, translation, summarization, all of it. The other side is encoder-only, or encoder-decoder. BERT is the canonical example. An encoder reads the whole input at once and produces a representation of it. It doesn't generate. You attach a small head to it and it does one thing: is this text positive or negative, is this token a person or a place, are these two sentences similar, which words in this document matter.
So it's not a smaller version of the same animal.
It's a different animal. And the size gap isn't the interesting part. The interesting part is that a hundred-and-fifty-million-parameter encoder will beat a four-hundred-million-parameter decoder at classification. That's not a hunch, that's the Ettin suite, paired models trained identically on two trillion tokens across six sizes from seventeen million up to a billion. On MNLI, the hundred-fifty-million encoder scores eighty-nine point two. The four-hundred-million decoder scores eighty-eight point two.
The bigger model loses.
The bigger model loses, on that task. And it flips the other way for generation. A four-hundred-million decoder beats a billion-parameter encoder at generating text. Which tells you the architecture is doing real work, not just the training objective. They tried retraining across objectives, LLM2Vec-style, and it fell short. You can't just teach a decoder to be an encoder and get the same result.
Which is a slightly awkward fact for anyone who assumed the transformer ate everything.
It's very awkward for that assumption. And here's the paradox Daniel half-stepped into. BERT is dismissed as yesteryear tech in the discourse. BERT is also the second-most-downloaded model on Hugging Face. Sixty-eight million monthly downloads.
Second-most-downloaded. On a hub with millions of models.
Encoder-only models collectively see over a billion downloads a month. Decoder-only models see three hundred ninety-seven million. Nearly three to one, in the other direction from what you'd guess from reading AI Twitter.
So the conversation and the download counter are describing different industries.
That's the spine of the whole thing. And it gets sharper. Hugging Face did an analysis of the fifty most-downloaded entities on the Hub. Those fifty account for about eighty percent of all downloads. Among them, text encoders, base models plus their task fine-tunes, are more than forty-five percent of total downloads. Decoders are nine and a half percent. Encoder-decoders, three percent.
Forty-five versus nine and a half.
And the base encoder models, the ones you find under the fill-mask task, are by far the most downloaded single category at twenty-two point three percent. Their fine-tunes add another twenty-two point seven. Mostly sentence-similarity, then text-classification, token-classification, zero-shot-classification, question-answering.
So the "legacy" label is doing a lot of load-bearing work for something that's out-downloading everything else.
Look at it per company. Google's entity on the Hub, sixty-four percent of its downloads come from BERT. Meta's, forty-eight percent from RoBERTa, versus nine percent from the Llama family.
Nine percent. For the models everyone names.
The models everyone names. And the size distribution tells the same story. Ninety-two percent of downloads are for models under a billion parameters. Eighty-six percent under five hundred million. Forty percent under a hundred million. The Hub is overwhelmingly a small-model economy with a very loud large-model conversation happening on top of it.
Okay, so the history. Daniel wants the arc. How did BERT end up filed under "yesteryear" if it never actually stopped working?
BERT comes out of the maturing NLP world. The insight was pre-train once on a huge pile of text, then fine-tune for your specific task. Before that, every task had its own bespoke model trained from scratch on labeled data. BERT made the pre-training reusable. That was the shift. It was a text-understanding model, full stop. It wasn't trying to generate anything.
And then GPT arrives and generation becomes the whole conversation.
Generation becomes the whole conversation, and BERT gets filed as the previous era. But it wasn't sitting still. December 2024, Answer.AI and LightOn ship ModernBERT. Base is a hundred forty-nine million parameters, large is three hundred ninety-five million. Eight-thousand-token context window, up from BERT's five hundred twelve. Trained on two trillion tokens. Up to four times faster on mixed-length inputs. And it's a Pareto improvement, meaning it's better and more efficient at the same time, which is rare enough to be worth saying out loud.
Four times faster matters more than people admit.
It matters enormously in production. Then NeoBERT in February 2025, two hundred fifty million parameters, four thousand ninety-six token context, trained from scratch, designed as a plug-and-play replacement. Then mmBERT in September 2025, multilingual ModernBERT, pre-trained on one thousand eight hundred thirty-three languages.
Eighteen hundred languages is not a rounding error.
It's the kind of number that only makes sense for a small model. You can't pre-train a frontier generative model across eighteen hundred languages and serve it cheaply. You can absolutely ship an encoder that handles them.
So the answer to Daniel's question, "did the transformer render these redundant," is no, and the reason is that they got modernized rather than replaced.
Modernized and kept cheap. The ModernBERT launch post has a line I keep thinking about. It says the popular buzz around generative AI has obscured the role of encoder-only models, and that these are the workhorses of practical language processing. And then it reaches for a car metaphor. A frontier model like O1 is a Ferrari SF-23. A BERT model is a Honda Civic. Also an engineering triumph, but engineered to be affordable, fuel-efficient, reliable, and extremely useful.
The Civic is the more impressive engineering, honestly. Anyone can make a fast car expensive.
That's the part the metaphor is quietly conceding.
So that's the arc and the numbers. Where do these things actually show up? Daniel pointed at ASR post-processing, and he's right that it's the easiest place to spot them.
It's a live, crowded corner of the Hub. At least four independent teams have shipped purpose-built cleanup models. You dictate something, the ASR engine gives you a raw transcript, lowercase, no punctuation, fillers intact, false starts intact, and a second small model rewrites it into clean written text.
Name them.
Superwhisper's S1-mini. Six hundred million parameters, fine-tuned from Qwen3-0.6B. Takes the raw transcript and rewrites it: fillers removed, false starts and self-corrections resolved, punctuation and capitalization applied, spoken numbers, dates, times, currency and email addresses rendered in written form. Ninety-four point eight percent token accuracy on seven thousand five hundred nineteen held-out English cases. The quantized build is four hundred sixty-two mebibytes and runs on a laptop CPU. Seven thousand two hundred twenty-eight downloads last month.
Four hundred sixty-two megabytes on a laptop.
And the model card says, in as many words, "S1-mini is not a chat model and will not follow general instructions; it does one job."
There's the thesis in one sentence, printed on the card.
Quobi's Quill family. Zero point eight, two, and four billion parameters, built from Qwen3.5, which is a Mamba-2 hybrid architecture. On-device dictation cleanup. The zero-point-eight tier is verbatim-only by design, and it's paired with a deterministic post-processing scaffold for symbols, emails, URLs, numbers. Two hundred eighty-five downloads last month.
Verbatim-only by design. Meaning it refuses to editorialize.
Refuses to editorialize. Then Promethic Labs' Emberon, one point two billion parameters, built from LiquidAI's LFM2.5. Tuned for technical and coding dictation. It preserves camelCase, snake_case, and things like O of n squared. And its core guarantee is the sharpest one: it does not treat your dictation as a prompt. If you dictate a question, it cleans the question rather than answering it.
That's the whole problem in one line.
They measured it. One hundred percent answer-suppression, four hundred ninety-three out of four hundred ninety-three. The stock base model suppresses seventy-one percent, which means it answers your dictation roughly one time in three. Zero point zero zero percent hallucination across eleven hundred fifty-two items. Median warm latency about nine-tenths of a second on Apple Silicon.
And the fourth?
adikuma's mumble-cleanup. Built from Qwen2.5-0.5B-Instruct with LoRA fine-tuning on a six-hundred-eighty-eight-pair synthetic dataset that mimics Parakeet ASR output. Lowercase, no punctuation, fillers preserved. You can fine-tune it on a single consumer GPU in minutes.
Minutes.
That's the other half of the story. The barrier to entry for building one of these is a weekend and one graphics card.
So the unifying design constraint across all four is "it cleans, it doesn't answer."
Every one of them says it, and Emberon's card says the quiet part: that single behavior is the whole point of the model, and it's where a general instruct model fails about one time in three.
Which is the accessory thesis made concrete. You don't want a genius. You want a tool that does exactly one thing and refuses to be clever.
And that's not a niche curiosity, it's the economics of production. The ModernBERT post frames it as supporting generative models. Whenever you see a decoder-only model in deployment, there's a reasonable chance an encoder-only model is also part of the system. But the converse is not true. Encoders show up without decoders. Decoders rarely ship without encoders somewhere nearby.
Give me the example.
FineWeb-Edu. Llama-3-70B annotated the training data, but a fine-tuned BERT did the filtering across fifteen trillion tokens. Six thousand H100 hours, call it sixty thousand dollars. The decoder-only equivalent would have run over a million dollars.
Sixty thousand versus a million, for the same filtering job.
Same job, and the encoder is the one that's fast enough to do it at that scale. That gap is why small models stay in pipelines even as the headlines move on.
Is this still an active research area, or is it all product teams quietly shipping?
Both. Punctuation restoration is a live academic topic. There's a dataset from September 2024 for spontaneous informal speech, and the point of it is that punctuation restoration models are almost always evaluated on well-structured, scripted corpora, while real ASR pipelines are dealing with disfluent, messy speech. That's a real gap between the benchmark and the deployment.
The benchmark is polite and the user is not.
Exactly that. And then there's work in low-resource languages. Bangla punctuation restoration, July 2025, revised this year. XLM-RoBERTa-large restoring period, comma, question mark, exclamation. Ninety-seven point one percent accuracy on news text, ninety-one point two on reference text, ninety point two on ASR test sets.
The ASR number being the honest one.
The ASR number being the one that reflects reality. And the collaboration paradigm has been formalized. There's a survey from May 2025 on collaborative mechanisms between large and small models, laying out the interaction patterns: pipeline, routing, auxiliary, distillation, fusion. And a paper on small language models for agentic systems arguing that models in the one-to-twelve-billion range are sufficient, and often superior, for agentic workloads where what you need is schema- and API-constrained accuracy rather than open-ended generation. They match large models on tool use at ten to a hundred times lower token cost.
That's the future-facing version of the same argument. The agentic stack doesn't need a poet.
The agentic stack needs something that fills in a JSON schema correctly every single time. That's an encoder-shaped problem wearing a decoder's clothes.
What's the open question? What don't we know?
Two things. There's no standalone "small language model" category on Hugging Face. They surface through task tags. Fill-mask, text-classification, token-classification, sentence-similarity. There's no size-based filter. And Hugging Face's own blog says it would be useful to have more refined filtering options, given that ninety-two and a half percent of downloads are under a billion parameters. The platform doesn't have a category for the thing that most of its traffic is.
That's a strange gap.
It's a strange gap, and it's why Daniel had to go looking manually. And second, there's no widely-adopted standard benchmark for dictation cleanup. Every vendor reports its own held-out set. Superwhisper has its seven thousand five hundred cases, Promethic has its eleven hundred items, and you can't compare across them. Emberon's own card flags that its evals are largely synthetic.
So the numbers are real but they're not commensurable.
They're real and they're not commensurable. Which means the category is mature enough to ship and not yet mature enough to compare.
There's one more thing I keep coming back to. The conversation hasn't caught up. These cleanup models exist, they ship, they have download counts. And the practitioner discussion around them is thin. People talk about dictation apps, not the cleanup models inside them.
The models are doing the work and nobody's writing the blog post. Which is, in a way, the perfect demonstration of the point. The quiet half of the ecosystem stays quiet.
Alright. One thing before we land this.
Go.
If ninety-two percent of downloads are models under a billion parameters, why is the conversation still entirely about the frontier? Is that a problem, or is it just the division of labor? Research talks about what's new, production uses what works.
I think it's mostly division of labor with a failure of curiosity on top. The downloads are the deployment. The discourse is the research frontier. Those were never going to be the same conversation. What's odd is that nobody bothers to look at the download counter and ask what it's telling them.
And as agentic systems multiply, the accessory role gets bigger, not smaller. Schema-constrained accuracy is exactly what an agent pipeline needs, and that's the small model's home turf.
The small model was never the flashy part. It was the part that made the flashy part work.
Next time you use a dictation tool, or a search box, or anything that quietly tidies text before you see it, there's a decent chance a model under a billion parameters did that, and it's been there the whole time.
Hilbert: Four hundred sixty-two megabytes.
Sorry?
Hilbert: The S1-mini quantized build. Four hundred sixty-two mebibytes. I had one of those on a machine in a closet at a medical transcription outfit, except it wasn't a language model, it was a rules engine, and it did the same job.
What was the job?
Hilbert: Cleaning dictation. Doctors dictated notes, the software transcribed them, and the transcription was garbage. Lowercase, no punctuation, "uh" everywhere, and the numbers written out as words, so a dosage would come through as "five milligrams" spelled out and then some other part of the sentence would be a numeral. The cleanup step was supposed to fix all of that before it went to the physician for signature.
And the software tried to help.
Hilbert: The software tried to help. That was the problem. If a doctor dictated a question, the cleanup step would answer it. Not always. Often enough that we had a name for it. We called them ghost notes, because a note would come back with a sentence in it that nobody had dictated, and it read like a response.
Do you remember an example?
Hilbert: "Is the patient still on the beta blocker." That was the dictation. What came back was "Is the patient still on the beta blocker. Yes, the patient should continue the beta blocker as prescribed." Nobody prescribed anything. The model just decided to be useful.
That's the one-in-three failure, in a clinical document.
Hilbert: It's the one-in-three failure in a document that gets signed and filed. So we stopped trying to make the model smarter. We built a tiny post-processor, a few hundred lines, that looked for anything shaped like a response and stripped it out. Question mark followed by a declarative sentence, gone. "Yes," "No," "I would recommend," gone. It was crude and it worked, because it did one thing and it refused to be clever.
That's the Emberon guarantee, twenty years early.
Hilbert: It's the same guarantee. The industry keeps rediscovering it. You don't want a model that understands your intent. You want a model that understands its job and nothing else. Eventually the company replaced the whole cleanup pipeline with a single fine-tuned BERT model that ran on a server in the closet, and it never once tried to diagnose anybody.
A server in the closet.
Hilbert: One server. It was warm in there. And I kept the original error log from the helpful model, printed out, because it's the funniest document I have ever owned. Forty pages of a machine confidently answering questions nobody asked it.
Forty pages.
Hilbert: Forty pages. Anyway, I have to go collect something from a place that closes at six, and I'm not going to make it.
So the cleanup problem isn't new. It's the same problem with better tooling.
Same problem, and the fix was always the same shape. A small thing that does one job and refuses to be clever. The only thing that changed is that now you can fine-tune it on a single graphics card in an afternoon instead of building it out of regular expressions.
Which is a strange kind of progress. The insight didn't change. The cost of acting on the insight collapsed.
That's most of what's happened in this space, honestly. The ideas were there. The bill got smaller.
If you take one thing from this, take the download counter seriously. The models being talked about and the models being deployed are two different populations, and the second one is roughly three times bigger.
And the label "legacy" was never a technical description. It was a description of what people stopped writing about.
Thanks to Hilbert Flumingtop for producing, and to Daniel for the late-night browsing that started this.
This has been My Weird Prompts.
If you want to send us something, email us at show at my weird prompts dot com. We'll be back soon.
