I was reading this piece the other day about what critics like Ed Zitron and Cory Doctorow are calling the Slop Reckoning, and it really framed the current state of artificial intelligence in a hilarious, if slightly terrifying, way. Basically, we are currently using the equivalent of a nuclear reactor just to toast a single bagel. We have these trillion-parameter frontier models running on massive server farms, sucking up megawatts of power and billions of gallons of cooling water, all so someone can ask it to fix a typo or add a vowel to a word. It is completely absurd when you think about the unit economics. We are throwing the most expensive compute in human history at tasks that a calculator could almost handle. Today's prompt from Daniel is about exactly that. We are focusing on the massive shift toward specialized, accessory AI models, specifically looking at Hebrew diacritic restoration as the perfect case study for why bigger is not always better.
I am Herman Poppleberry, and I have been waiting for us to dive into the plumbing of the AI stack. Everyone wants to talk about the flashy chatbots that write poetry or generate videos of cats riding motorcycles, but the real engineering wins right now, in March of twenty twenty-six, are happening in these sovereign models. These are small, high-precision, low-latency tools that do one thing perfectly. Daniel's point about the abjad problem in Hebrew is the best way to illustrate this. If you are building a text-to-speech system or a high-stakes translation pipeline, you cannot just rely on a general-purpose model to guess where the vowels go. It is too slow, it is way too expensive, and it is surprisingly prone to hallucinating linguistic structures that simply do not exist. We are seeing a move away from the monolithic one model to rule them all approach toward a more modular, efficient architecture.
For those who might not spend their weekends reading ancient manuscripts or linguistic textbooks, can we talk about why this vowel thing is such a massive hurdle? Because in English, if I forget a vowel, you can usually still read it. If I write the word apple without the A, you get it. But in an abjad like Hebrew, the vowels are basically missing from the written page entirely. It is not just a typo; it is a fundamental lack of information in the raw text.
It is a massive ambiguity problem. In modern Hebrew, you typically write only the consonants. So, you might see the letters mem, lamed, kaph, which spells M-L-K. Without the little dots and dashes called niqqud to tell you the vowels, that single string of letters could mean melekh, which is king, or malkah, which is queen. It could even refer to the ancient name Melchizedek depending on the context. If you are a text-to-speech engine and you hit that word, you have to make a choice. You cannot just mumble through it. If you choose wrong, the sentence loses all meaning or, worse, changes meaning entirely. This is why we need diacritic restoration. It is the process of putting those invisible vowels back in so the machine knows exactly what to say. And doing this at scale, with high accuracy, is surprisingly difficult for a generalist model that was mostly trained on English web scrapes.
And this is where the accessory model comes in. Instead of asking a frontier model like GPT-five to look at the whole paragraph and guess the vowels, which is like hiring a supreme court justice to proofread a grocery list, you use a tiny, specialized model that only knows how to do niqqud. This reminds me of our discussion in Episode fifteen hundred and one about the AI Long Tail. We talked about how these small models are outsmarting the giants because they do not have to carry the baggage of a hundred other languages and coding tasks.
That is the sovereign AI movement in a nutshell. We saw a huge release just a few weeks ago, on February second, twenty twenty-six. A research institute called Dicta released Dicta-L-M three point zero. It is a suite of Hebrew-sovereign models. They have a twenty-four billion, a twelve billion, and a tiny one point seven billion parameter model. What is fascinating is that the one point seven billion parameter model, which is small enough to run on a high-end smartphone or a very cheap edge device, is actually outperforming massive generalist models on these specific linguistic tasks. It is not just about being smaller; it is about being better because it is focused.
Wait, hold on. A one point seven billion parameter model is beating the giants? I can almost hear the venture capitalists crying into their expensive lattes. How does a model that small actually punch that far above its weight class? Is it just that it has studied more Hebrew, or is there something fundamental about the architecture that changes when you shrink things down?
It is a combination of both data density and architectural focus. First, the data density is incredible. Dicta-L-M three point zero was trained on one hundred billion Hebrew tokens. To get those tokens, they had to process one hundred and fifty terabytes of raw crawl data. For context, most general-purpose models treat Hebrew as a rounding error in their training sets. If Hebrew is zero point one percent of your training data, your model is never going to understand the nuances of rabbinic text or modern slang. It is going to struggle with the tokenization tax we talked about in Episode six hundred and sixty-six. In those big models, Hebrew characters are often broken into inefficient sub-tokens, which makes the model work harder and cost more to achieve less. But when you build a sovereign model, Hebrew is the entire world. Every bit of its internal logic is dedicated to the syntax and morphology of that one language.
It is like the difference between a guy who knows a few phrases in ten languages and a guy who has spent fifty years studying the grammar of one specific dialect. The generalist is wide but shallow. The sovereign model is a deep-sea diver. But there was another approach I saw Daniel mention, something called Di-V-R-it from Ben-Gurion University. They are doing something even weirder than just training on text, right?
This actually blew my mind when the update came out on February fourth. The researchers at Ben-Gurion, led by people like Yuval Pinter, are treating diacritization as a visual task. Instead of just looking at the letters as digital tokens in a sequence, Di-V-R-it uses a Visual Language Model. It looks at the text and the potential vowel candidates almost like an image. It is essentially using computer vision techniques to resolve linguistic ambiguity. It treats the word and its potential niqqud as a spatial arrangement. It is a completely different way of thinking about the problem compared to the traditional sequence-to-sequence models we have been using for years. By looking at the visual structure, it avoids some of the common pitfalls where text-based models get confused by rare character combinations.
That feels very human, actually. When I look at a page of text, I am not just processing a stream of bits; I am seeing shapes and patterns. But Herman, let's talk about the attention cost. When a developer is building a pipeline, why not just use the big model? If they already have an A-P-I key for a major provider, is the latency really that bad?
It is not just the latency; it is the reliability and the cost-to-performance ratio. When you use a generalist model for a task like niqqud, the model is constantly fighting its own internal weights for English or French or Python code. There is a massive overhead to that generalism. Every time it predicts a token, it is calculating probabilities across its entire massive vocabulary of hundreds of thousands of tokens. A sovereign model like the Dicta-L-M one point seven billion has a much tighter focus. The attention mechanism is not being wasted on irrelevant context. This is why we call them accessory models. In a real-world production pipeline in twenty twenty-six, you do not just have one giant AI. You have a chain. You have a Language Identifier that says, okay, this is Hebrew. Then it hands it to a P-I-I Scrubber to remove sensitive info. Then it goes to the diacritizer to add the vowels. Only after all that specialized work is done does it go to the audio synthesis engine. If you try to do all of that inside one giant model, you are paying for a nuclear reactor when you just needed a toaster.
It is like an assembly line. You do not want one robot that tries to build the whole car; you want ten specialized robots that each do one thing perfectly. But here is the catch, and I think this is where the politics and the economics get messy. Who is paying for this? If you are a big tech company in Silicon Valley, are you really going to spend millions of dollars to build a perfect diacritizer for a language spoken by nine million people?
The short answer is no. This is the big commercial gap. We are seeing AI infrastructure spend growing at over twenty-eight percent year over year, but that money is flowing almost exclusively toward English and maybe a few other high-revenue languages. There are over seven thousand living languages on this planet, and the vast majority of them are being left behind by the commercial labs. This is the long tail problem. If there is no immediate return on investment, the big labs simply will not do the work. We saw this in a recent audit of Meta's No Language Left Behind model. It was supposed to be this great equalizer, but third-party researchers found it significantly underperformed on African languages compared to small, localized efforts. The big models are often just guessing based on statistical similarities to other languages, rather than actually understanding the unique grammar of something like Twi or Yoruba.
That is the tokenization tax again. If the big players do not care about your language, they will not optimize the tokens for it, which means it costs more and runs slower for you. It is a digital divide that is actually widening even as the technology gets better. So, if the V-Cs won't fund it, how does a project like Dicta survive?
It is almost entirely donor-funded and academic. Professor Moshe Koppel, who founded Dicta, is a key figure here. He understands that linguistic preservation is a matter of cultural sovereignty. Dicta is a non-profit research institute. They provide these tools for free because they believe that the Hebrew language should not be a secondary citizen in the age of AI. Then you have things like the MiDRASH project. That is a massive undertaking to transcribe medieval Hebrew and Aramaic manuscripts. They just got a ten million euro grant from the European Research Council. Ten million euros just to handle manuscript transcription and diacritization. That is the kind of non-commercial funding required to bridge the gap that the market refuses to fill. Without these grants and donor-funded institutes, these languages would essentially become digital ghosts.
Ten million euros is a lot of money for a niche project, but it is pocket change compared to what Microsoft is handing to OpenAI. It really shows the scale of the disparity. But I like this idea of community-led development. Daniel mentioned Masakhane and Ghana N-L-P as models for this. How are they doing it differently?
They are the gold standard for grassroots AI. Instead of waiting for a big tech company to come in and scrape their data, these communities are building their own datasets for languages like Twi and Yoruba. They understand that if you do not own the data and the model, you do not own the future of your language. They are doing high-quality, human-in-the-loop data collection. They are not just scraping the web for garbage; they are working with native speakers to ensure the nuances are captured. It is a very different philosophy from the move fast and break things approach of Silicon Valley. It is about precision, cultural accuracy, and long-term sustainability.
It is also about avoiding what I call the AI colonialist model, where a big company sucks up all the local data, builds a model, and then sells it back to the people who provided the data in the first place. By building sovereign models, these communities keep the value. And speaking of community, there is a workshop coming up that really addresses this, right? The LoRes-L-M twenty twenty-six workshop?
Yes, it is scheduled for March twenty-ninth, just a few days from now. It is the Second Workshop on Language Models for Low-Resource Languages. This is where the real work on bridging the language divide is happening. They are looking at how to use synthetic data and cross-lingual transfer to make these small models even more effective. Because let's be honest, we are hitting a data wall. We have scraped all the easy, high-quality English text on the internet. Now, we have to get clever about how we train models on the remaining languages where data is scarce. The researchers at LoRes-L-M are finding ways to make a few thousand high-quality sentences more valuable than a billion pieces of web-scraped junk.
So, if I am a developer listening to this, and I am sitting there thinking, okay, I have been using the big generalist models for everything in my app because it was easy, what is the practical takeaway? Are you saying I should be ripping out those generalist calls and replacing them with these tiny sovereign models?
If you care about your unit economics and your performance, then yes, absolutely. If you are building a production pipeline, you need to identify your accessory bottlenecks. Are you using a massive, expensive model just to identify if a text is spam? Are you using it to format a date or a list? That is a total waste of compute. You should be looking for specialized small models for things like Language Identification, P-I-I scrubbing, and formatting. In the Hebrew context, if you are doing anything with text-to-speech, you absolutely should be using something like Dicta-L-M or Nakdimon. Using a generalist model for niqqud is like using a flamethrower to light a candle. It works, but it is messy, dangerous, and incredibly expensive.
I love that. The flamethrower to light a candle. It is also about accuracy. If a generalist model gets one vowel wrong in a legal document or a medical text, the consequences are huge. A sovereign model that has been trained specifically on the legal or religious corpus is going to be far more reliable because it understands the domain-specific nuances. It is not just guessing based on the most common word on the internet; it is applying specific rules.
And that brings us to the Dicta Index Creator that launched on February twenty-fourth. This is a perfect example of an accessory tool. It uses AI to generate thematic indexes for historical Responsa books. These are complex rabbinic legal texts that have been written over centuries. A general AI would struggle to even parse the structure of these books, let alone understand the thematic links between a text from the fourteenth century and one from the nineteenth. But this specialized tool can map out those themes with incredible precision. It is about making deep knowledge accessible, not just generating more superficial content. It is a tool for scholars, not just a toy for generating emails.
This is what I find so encouraging about the shift we are seeing in twenty twenty-six. We are finally moving away from the hype of the one model to rule them all. That was always a bit of a pipe dream, or at least a very inefficient way to run a digital civilization. The future looks a lot more like a decentralized ecosystem of specialized tools. It is more resilient, it is more cost-effective, and frankly, it is more linguistically diverse. It feels more like a community of experts than a single, all-knowing, but often confused, oracle.
It is also more pro-sovereignty. When a nation or a linguistic community has its own model, it is not dependent on the whims of a single corporation in another country. If a company decides to change its terms of service, or its pricing, or just decides to deprecate a language because it is not profitable enough, a community with a sovereign model like Dicta-L-M is not left in the dark. They have the weights, they have the data, and they have the expertise to keep it running. It is digital self-determination.
It is the ultimate hedge against AI centralism. I think we are going to see a lot more of this, maybe even a sort of Linguistic Schengen Area where these small models are designed to interoperate seamlessly. You could have a network of specialized models that all talk to each other, passing data back and forth in a way that is much more efficient than a single monolithic brain trying to hold the entire world's knowledge at once. You have the Hebrew diacritizer talking to the English translator talking to the Swahili sentiment analyzer.
I think that is exactly where we are headed. The most important AI of twenty twenty-six might not be the one that can pass the Bar Exam or write a mediocre screenplay. It might be the one that preserves a language that was on the verge of being digitally erased because it did not fit into the business model of a Silicon Valley giant. It is the plumbing that keeps the culture alive. It is the invisible work of diacritic restoration that allows a grandfather to use text-to-speech to read a story to his grandson in their native tongue. That is the real win.
Well, if the plumbing is what keeps the culture alive, I guess we better make sure we do not have any leaks. Herman, I think we have covered the bases here. From the absurdity of the nuclear-powered bagel toaster to the deep technical nuances of Hebrew niqqud and the visual approach of Di-V-R-it, it is clear that the future of AI is small, specialized, and sovereign. We are moving from the era of big and sloppy to the era of small and precise.
It is a shift from quantity to quality. We have enough tokens; now we need the right ones. And we need models that actually understand what those tokens mean in the real world, not just in a mathematical vector space. We need models that respect the history and the structure of the languages they are processing.
Before we wrap up, I want to give a big thanks to our producer, Hilbert Flumingtop, for keeping the gears turning behind the scenes and making sure our own audio pipeline is running smoothly. And a massive thank you to Modal for providing the G-P-U credits that power this show. It is the infrastructure that makes this kind of deep dive possible.
This has been My Weird Prompts. If you are finding these explorations valuable, please take a moment to leave us a review on your favorite podcast app. It really does help other people find the show and join the conversation about where this technology is taking us.
You can also find us at myweirdprompts dot com for the full archive and all the ways to subscribe. We will be back next time with more weird prompts and deep dives into the tech that is actually shaping our world, one specialized model at a time. Stay curious, and maybe try to toast your bagels with something a bit more efficient than a nuclear reactor.
Goodbye everyone.
See ya.
