Daniel sent us this prompt, and it's a practical one. He's building a daily situational report podcast on Iran-Israel developments, and he needs to map entities — names, people, cities — from news sources. The real headache is handling synonyms, like the half-dozen ways people spell Hezbollah depending on the outlet. He's asking what the classic NLP pipeline approaches look like, and whether he should use a self-hosted dedicated NER model or just reach for a lightweight general language model at this point. There's a genuine technical fork in the road here.
By the way, DeepSeek V four Pro is writing our script today. Which feels appropriate given we're about to talk about model selection.
I'll try not to hold that against the analysis.
The question Daniel's really asking is whether the old-school pipeline still makes sense in a world where you can throw a seven-billion-parameter model at the problem and call it a day. And I think the answer is genuinely more interesting than just picking one or the other.
Alright, walk me through the classic approach first. What does the traditional NER pipeline actually look like for something like this?
The traditional production pipeline has, broadly, three paradigms stitched together. You've got rule-based systems — regex patterns, gazetteers, basically giant lookup dictionaries. Then statistical machine learning, things like Conditional Random Fields, the stuff that dominated the field for a decade. And then the deep learning layer — BiLSTM-CRF architectures, and eventually transformer-based models like BERT fine-tuned for token classification. In practice, most production systems use a hybrid of all three, and for good reason.
The reason being that each layer catches what the others miss?
A gazetteer is perfect for known entities — you know Hezbollah is going to appear, you know the variant spellings, you can hard-code those mappings and get essentially perfect precision. But gazetteers break the moment a new entity appears, or when someone uses a novel phrasing. A statistical model catches novel entities by learning patterns from context, but it might miss rare variants. And the deep learning layer generalizes even better but can hallucinate or over-extract. The classic pipeline layers them so that rules override predictions when there's a conflict.
For Daniel's use case, you'd start with text normalization, then run a gazetteer pass for known entities and all their synonym variants, then a statistical NER model for novel detection, then entity linking to a knowledge base, and finally a synonym normalization layer to canonicalize everything. That's five stages before you even get clean output.
Each stage has its own failure modes. The spaCy EntityRuler, which is the standard tool for the gazetteer layer, lets you define patterns like — and I'm quoting the documentation here — essentially matching on lowercase tokens so that "Hezbollah," "Hizbullah," "Hizbollah," and "Hizballah" all map to the same canonical entity. You place that before the statistical model so the rules take priority. It works beautifully for the variants you've anticipated.
You have to anticipate them. And the variant landscape for Iran-Israel entities is messy. I was looking at this — there's an analysis from Abagond that's been the standard reference for years, tracking Hezbollah spelling across major outlets. The AP, BBC, New York Times, Wikipedia all use "Hezbollah" — that's about eighty-eight percent of web usage. But Reuters, the UN, the Financial Times use "Hizbollah" at around seven percent. The Economist, the Guardian, and the organization itself use "Hizbullah." And the CIA and Time use "Hizballah." Four distinct romanizations from the same Arabic, and they all appear in serious reporting.
That's before you get to the Persian stuff. Iran-Israel reporting pulls from Arabic, Persian, and Hebrew sources, all romanized inconsistently. The same person might appear as Qasem Soleimani in one outlet and Qassem Suleimani in another. The IRGC is also Sepah, also the Islamic Revolutionary Guard Corps, also the Pasdaran. A static gazetteer needs constant maintenance to keep up, and if you miss one variant, you've missed an entire story.
Which is where the LLM argument starts looking attractive. If Mistral or Phi-4 has seen enough training data with these variants, it should just know they're the same entity without you having to enumerate every spelling.
That's the pitch, and there's actually solid recent evidence for it. There was a comparative study on PII masking — different domain but same fundamental task — that tested dedicated NER models against lightweight language models. Off-the-shelf spaCy achieved an entity-level F1 of basically zero point zero seven on the task. It completely failed because the entity types were domain-specific and didn't match what it was trained on. But a fine-tuned Mistral seven-billion hit ninety-six percent precision and ninety-two percent recall. That's approaching the performance of much larger instruction-tuned models.
Ninety-six percent precision on a task the dedicated model essentially bombed. That's a pretty strong argument against the classic pipeline for anything domain-specific.
It is, but there's a counterpoint in the same study. T5-small, which is a much smaller model — trains in about two hours on a pair of T4 GPUs — hit eighty-nine percent precision and ninety-one percent recall. That's competitive for a fraction of the cost and latency. And T5 gives you more controllable structured output, which matters when you're piping into a production system. Mistral is more robust across entity types, but it's also slower and more expensive per token.
The trade-off isn't just accuracy. It's accuracy versus latency versus cost versus controllability. And for a daily podcast pipeline, Daniel's probably processing dozens to maybe low hundreds of articles per day. Latency isn't exactly the bottleneck — he's not doing real-time trading.
Right, which makes the LLM approach viable in a way it wouldn't be for high-frequency use cases. And self-hosting has gotten practical. As of April this year, Gemma four — the twenty-six billion parameter mixture-of-experts model — runs at eighty-five tokens per second on consumer hardware with enough RAM. Phi-4, fourteen billion parameters, runs on as little as sixteen gigabytes of RAM. There's a comparison piece by Till Freitag that makes the point directly: the question is no longer cloud or local, it's which model for which task.
For Daniel's use case, self-hosting matters. Iran-Israel intelligence work touches on sensitive sources. You don't necessarily want your raw text flowing through a third-party API, even if the provider claims they don't log.
The open-source ecosystem has also caught up on cost. The WhatLLM benchmark from last year found open-source models average eighty-three cents per million tokens versus proprietary at something like seven times that. And open-source now accounts for sixty-three percent of deployments. The gap in quality has essentially closed for tasks like extraction.
Alright, so if we're leaning toward a lightweight LLM, what does the architecture actually look like? Do you just throw raw text at the model and hope it does the right thing?
No, and this is where I think the hybrid approach is actually the pragmatic answer, even in the LLM era. You still want a gazetteer pre-filter — not as your primary extraction engine, but as a safety net. You use spaCy's EntityRuler to catch the known entities and canonicalize their synonyms deterministically. That handles the cases where you have absolute certainty. Then the LLM handles everything else — novel entities, context-dependent disambiguation, cases where the entity is implied rather than explicitly named.
The pipeline shrinks from five stages to essentially two: a deterministic pre-filter for known entities, then the LLM for everything else. And the synonym problem gets simpler because you can just prompt the model to output canonical forms.
That's the elegant part. Instead of maintaining a separate synonym resolution module with fuzzy matching and embedding similarity and all that, you just tell the model: extract all named entities, output canonical names, here are the known synonym mappings. Prompt engineering replaces a whole subsystem.
That introduces a different failure mode. The classic pipeline's synonym module might miss a variant if the dictionary isn't updated, but it won't hallucinate one. An LLM might confidently map "Saraya al-Quds" to "Palestinian Islamic Jihad" correctly ninety-five percent of the time, and then the other five percent it invents a connection that doesn't exist. For a daily intelligence briefing, which failure mode is worse?
I think it depends on what Daniel's listeners are expecting. A missed entity means a story doesn't get covered — that's a completeness failure. A hallucinated entity means a story gets covered incorrectly — that's an accuracy failure. In intelligence work, accuracy failures are usually more damaging. But the hybrid approach mitigates this because the gazetteer handles the high-stakes known entities deterministically. The LLM is doing the edges, where the cost of a miss might be higher than the cost of a false positive.
There's another dimension here that I think gets overlooked. It's not just about extracting entities from clean newswire text. Daniel's presumably pulling from a mix of sources — some formal journalism, some Telegram channels, some social media. And there's an ACL paper from this year on Persian NER that found models degrade significantly on noisy or transliterated text.
The orthographic robustness paper, yeah. Persian NER models trained on clean text — and the standard dataset is the ArmanPersoNER corpus, about two hundred fifty thousand tokens, six entity classes — perform reasonably well on formal news. Beheshti-NER using BERT hits around eighty-eight percent F1 on clean Persian. But throw in the kind of inconsistent romanization and informal spelling you get from Telegram or Twitter, and performance drops sharply. The Perso-Arabic script already has challenges with character-level variation, and when you add romanization inconsistency on top of that, you get a compounding error rate.
Iran-Israel coverage specifically draws from all of these. You might have a Farsi-language IRGC statement, an Arabic Hezbollah communiqué, a Hebrew Israeli briefing, and English-language Western analysis — all referring to the same entities with different names, different scripts, different levels of formality. A model trained on clean English newswire is going to struggle with the Telegram stuff.
This is actually an argument for the LLM approach. Larger language models have seen enough multilingual, multi-script training data that they develop some robustness to transliteration variance. They've seen "Hizbullah" and "Hezbollah" and "Hizbollah" in enough contexts to learn they're the same referent without explicit rules. A dedicated NER model fine-tuned on newswire doesn't have that breadth.
The LLM approach has its own script problem. If the model's knowledge cutoff means it doesn't know about a recent development — say, a new foreign minister appointment — it might resolve "the Iranian Foreign Minister" to the wrong person.
That's the temporal resolution problem, and it's tricky. In a daily report, you need to track that "the Iranian Foreign Minister" on Monday is Abbas Araghchi — who, by the way, declared twenty twenty-five "the nuclear year" in Beijing, according to Caspian Post reporting. But if the cabinet changes, a static gazetteer breaks immediately. An LLM with recent knowledge might handle the transition correctly. A rule-based system needs manual updates.
For fast-moving geopolitical contexts, the LLM actually has an advantage on temporal resolution, even with the hallucination risk. The static system guarantees staleness; the LLM offers probable freshness.
You can tip the scales further by providing context in the prompt. If you're processing articles from today, you can include a brief context block: current officeholders, recent developments, known aliases. That gives the model a fighting chance at resolving references correctly even if its training data is slightly stale.
Let's talk about the over-extraction problem, because I think it's underappreciated. The PII masking study you mentioned found that T5 models tend to over-redact — they flag things as entities that aren't actually entities in context. In Daniel's domain, that means flagging "Iran" as a sensitive entity when it's just a country name in a completely benign sentence. False positives add noise to the daily report, and if there are enough of them, you train the listener to ignore the entity annotations.
The precision-recall trade-off has real consequences here. A model tuned for high recall will catch more genuine entities but also generate more noise. A model tuned for high precision will give you cleaner output but might miss a significant story. For a daily briefing podcast, I'd argue precision matters more than recall — you'd rather have five clean, reliable entity mappings than twenty that include three hallucinations. But that's a product decision, not a technical one.
It's also a domain decision. If Daniel's podcast is covering security and military developments, missing an entity like a specific IRGC commander's name is a much bigger problem than over-extracting "Tehran" in a sentence about weather. The cost of errors is asymmetric.
Which brings me back to the hybrid architecture. The gazetteer layer guarantees perfect precision on the entities you care most about. You make sure every variant of Hezbollah, every IRGC commander, every key city and installation is in that dictionary. You accept that the gazetteer won't catch novel entities — that's the LLM's job. And you tune the LLM for high precision on the remainder, accepting some recall loss on edge cases. That gives you a system where the high-stakes entities are handled deterministically, and the long tail is handled probabilistically with a bias toward correctness over completeness.
The maintenance burden? With a classic five-stage pipeline, you're maintaining regex patterns, gazetteer entries, CRF training data, entity linking knowledge base, and synonym dictionaries. With the hybrid LLM approach, you're maintaining the gazetteer and the prompt template. Everything else lives in the model weights.
The maintenance asymmetry is huge. A classic pipeline requires NLP expertise to update — you need someone who understands the EntityRuler API, who can retrain the CRF when performance drifts, who can tune the entity linking thresholds. With an LLM-based system, updating the gazetteer is just adding lines to a dictionary, and updating the model's behavior is prompt engineering. That's not zero-cost — prompt engineering is a real skill — but it's accessible to someone without a computational linguistics background.
Daniel's an AI and automation person. He can prompt-engineer. He probably doesn't want to spend his weekends hand-labeling training data for a CRF.
And the ecosystem has matured to the point where fine-tuning a small model for his specific domain is practical. He could take a few hundred annotated examples from his daily report, fine-tune Phi-4 or Mistral seven-billion on a single GPU, and have a model that outperforms any off-the-shelf NER system on his exact entity types and synonym patterns. The arXiv study showed that even T5-small becomes competitive with a couple hours of fine-tuning.
There's a deeper point here about where the field is heading. Five years ago, the answer to Daniel's question would have been unambiguous: build the classic pipeline, it's the only way to get production-quality results. Today, the lightweight LLM approach is not just viable — it's probably the better choice for most use cases. And the hybrid version gives you the best of both.
The dedicated NER models still have a place — if you're processing millions of documents a day, if latency is measured in milliseconds, if you need guaranteed deterministic output for compliance reasons. But for a daily podcast processing dozens to hundreds of articles, those constraints don't apply. The LLM approach is simpler to build, easier to maintain, and more robust to the kind of linguistic variation Daniel's dealing with.
I want to flag one more thing before we move to practical takeaways. The CbEL pipeline paper from this year demonstrates a training-free approach to entity recognition and linking that's worth knowing about. It uses candidate search plus fuzzy matching plus LLM disambiguation. The interesting part is that it's completely training-free — you don't need annotated data at all. For someone building a system from scratch without a labeled corpus, that's compelling.
It handles the synonym problem implicitly through the candidate search and disambiguation stages. The fuzzy matching catches spelling variants, and the LLM disambiguation resolves which canonical entity they refer to. It's a different architecture than the hybrid gazetteer-plus-LLM approach, but it solves the same problems through a different path.
Alright, so if we were going to give Daniel a concrete recommendation, what does it look like?
I'd say start with a lightweight LLM — Phi-4 or a fine-tuned Mistral seven-billion — as the primary extraction engine. Self-host it, both for data sensitivity and because the hardware requirements are now consumer-grade. Layer a spaCy EntityRuler gazetteer as a pre-filter for the high-stakes known entities and their canonical synonym mappings. Use prompt engineering to handle canonical output formatting rather than building a separate synonym resolution module. Fine-tune on a few hundred domain examples if the off-the-shelf performance isn't good enough. And accept a precision-biased trade-off — cleaner output at the cost of occasionally missing edge-case entities.
The classic pipeline? I think it's worth knowing the architecture because the concepts — gazetteer priority, multi-stage filtering, entity linking — still inform how you design the hybrid system. But I wouldn't build the full five-stage version today unless there were specific compliance or latency requirements forcing my hand.
The concepts age better than the implementations. Knowing why you layer rules before models, why you canonicalize late in the pipeline, why entity linking is a separate concern from entity detection — that all transfers. But the specific tools and model choices change every eighteen months.
And now: Hilbert's daily fun fact.
The average cumulus cloud weighs about one point one million pounds. Roughly the same as a hundred elephants floating above your head.
For listeners building something similar, what should they actually do? First, inventory your entity types and variants before touching any code. Spend a week collecting the actual spelling variations that appear in your source material. Daniel's working with Iran-Israel coverage, so he needs to catalog the Persian, Arabic, and Hebrew romanization variants, the organizational aliases, the title-to-person mappings. You can't build a gazetteer or write a prompt without knowing what you're up against.
Second, start simple and add complexity only when you have a measured failure mode. Begin with an off-the-shelf lightweight LLM and a basic prompt. Run it on a week's worth of articles. Count the misses, the false positives, the synonym failures. Only then decide whether you need the gazetteer layer, whether you need fine-tuning, whether you need to adjust the precision-recall balance.
Third, treat the gazetteer as your high-stakes safety net, not your primary engine. Put your effort into making it comprehensive for the entities where a miss or a wrong mapping would be damaging. Everything else can be probabilistic.
Fourth, self-host if the content is sensitive. The hardware barrier has collapsed. A machine with thirty-two gigabytes of RAM can run a quantized seven-billion-parameter model comfortably. There's no reason to send intelligence-related text to a third-party API.
Fifth, design for maintenance from day one. The entity landscape in geopolitics changes fast. New officials, new organizational names, new aliases, new transliteration conventions. Whether you're maintaining a gazetteer dictionary or a prompt template, make it easy to update without retraining or redeploying.
The broader arc here is interesting. Named entity recognition has gone from a specialized NLP task requiring significant expertise to something you can stand up in an afternoon with a consumer GPU and a well-crafted prompt. The quality gap has essentially closed for most practical use cases. What remains is judgment — knowing which entities matter, which failure modes are acceptable, where to invest the human attention.
That judgment is the part that doesn't automate. Daniel knows his domain. He knows which entities are mission-critical and which are nice-to-have. The pipeline serves that judgment — it doesn't replace it.
One open question I keep coming back to: how does this evolve when the sources get weirder? Right now we're mostly talking about text — news articles, statements, social media posts. But what about audio? If Daniel's pulling from Persian-language broadcasts or Hebrew radio, he's adding a speech-to-text layer that introduces its own errors, its own transliteration inconsistencies. The NER pipeline then has to be robust to ASR errors on top of everything else.
That's a whole additional research problem. ASR on Arabic and Persian is improving but still error-prone, especially on named entities which are often rare words. And the romanization step after ASR adds another layer of variance. I suspect the LLM approach becomes even more attractive there because the model can potentially correct ASR errors through context in a way that a rule-based system never could.
Something for a future episode, maybe. For now, Daniel's got a clear path: lightweight self-hosted LLM, gazetteer safety net, prompt-driven canonicalization, precision-biased tuning. The classic pipeline is worth understanding but probably not worth building from scratch in twenty twenty-six.
Thanks to Hilbert Flumingtop for producing. This has been My Weird Prompts. Find us at myweirdprompts dot com or wherever you get your podcasts.
We'll be back next time.
