How do you tell a pigeon it's not welcome without standing on the balcony all day with a broom?
That's the question. And it turns out you don't tell it anything. You just get a notification.
Which is what Daniel figured out. He wrote in about this. Hannah came to him and conceded, and I want to underline that word, conceded, that they needed to bring his technical skills into the war on pigeons. Round one went to them. The pigeons nested on the new balcony, Hannah put stones down where they were landing, and they haven't come back. But as Daniel puts it, they don't want to rest on their pigeon-free laurels.
Because they actually want to use the balcony.
They want to use the balcony. So Hannah asks, can we get a notification whenever a pigeon is observed out there. Daniel, being Daniel, immediately goes into technical planning mode and asks her two clarifying questions. Do you want to know what species it is, or is bird enough. Do you want to know what it's doing, or just that it's there.
And she says presence. Just presence.
Just presence. Though Daniel notes he spotted some pretty cool species-identification models on Hugging Face if that ever became the objective. His instinct is that this is an object recognition task, that Frigate is the obvious tool, and that MQTT into Telegram is more than sufficient. The snag is resources. The home server is a repurposed old desktop with an i5 integrated GPU, and a dedicated TPU purchase fell through. So the questions are, what's the most lightweight but practical model for bird presence on a 1080p webcam, how do you configure Frigate without overwhelming that box, and how do you keep humans from triggering false positives when they're sitting out there?
And here's the thing. The answer to the first question is that Daniel doesn't need to pick a model at all.
That's a good opening. Say more.
Frigate's default model already has bird as a built-in object class. It ships with it. Bird, person, cat, dog, car, all of it comes in the box. There is no custom training, no downloading a model off Hugging Face, no TPU. Daniel framed this as an object recognition problem and he's exactly right, but the recognition is already done for him. The engineering that's actually left is in filters, zones, and masks.
So the hard part isn't the hard part.
The hard part isn't the hard part. And the iGPU is fine. OpenVINO is the correct detector for Intel integrated graphics and it's explicitly supported on sixth-generation Skylake and newer. A repurposed desktop i5 is almost certainly in that range. The failed TPU purchase is a non-issue for one camera. The Coral is the community default, it's about a hundred dollars, it'll do a hundred-plus frames a second with very little overhead, and it is completely unnecessary here.
I like this episode already. The answer is that the thing he was worried about isn't a thing.
The thing he was worried about isn't a thing. What he should be worrying about is the config file.
So let's get into the config file. Model choice first. What's he actually running?
OpenVINO detector, YOLOv9-tiny, at 320 by 320. That's the starting point. And the 320 part is the counterintuitive bit, because your instinct with a small distant pigeon in a 1080p frame is that you need a bigger model to see it. That's backwards.
Explain why, because that's not obvious.
Frigate is designed around the assumption that the model is small. It crops a region of motion out of the full frame and zooms into it before it runs detection. So by the time the model sees anything, the pigeon has already been blown up to fill most of the input. A 320 by 320 model is actually better at small and distant objects than a 640 by 640 model, not worse.
Because the 640 model's advantage is covering a wide field with lots of objects in it.
Right. A 640 by 640 model is slower and its real benefit is fitting many objects spread across a large area into one inference. That's a parking lot. That's a street. That's not a balcony with one pigeon on a railing. So Daniel would be paying double the compute for a capability he doesn't need and losing accuracy on the thing he does need.
Which is a nice general lesson. The bigger model isn't the better model. The model that matches your pipeline is the better model.
And Frigate's docs are blunt about the alternatives too. The CPU detector, the plain tflite one, is not recommended for general use. Their own guidance is that if you don't have a GPU or an Edge TPU, running OpenVINO in CPU mode is often more efficient than the CPU detector anyway. So even if the iGPU path fails for some reason, the fallback isn't the CPU detector. The fallback is OpenVINO on the CPU.
Which means the i5 has two bites at this.
It has two bites. And the inference budget is generous. A GPU can run multiple concurrent model instances, so thirty milliseconds per inference is fine across several cameras. A Coral runs a single instance and needs about ten milliseconds. Daniel has one camera. He has headroom he isn't going to use.
What's the rule for picking the size then, if tiny is the start?
Use the largest model your hardware can run without skipping detections, and watch it in the metrics page. Frigate has a System, Metrics, Cameras view that shows you exactly what's happening per camera. If detections are being skipped, you're over budget. If they aren't, you can go bigger. But for one 1080p camera on an empty balcony, tiny at 320 is going to be ample.
Let's talk about the part that actually eats the CPU, because I suspect it isn't inference.
It often isn't. Decoding 1080p video is frequently the heavier cost than running the detector. And hardware acceleration for video decode is a separate setting from object detection. People enable one and assume they've enabled the other. You want both pointed at the iGPU.
So the two levers are, decode on the GPU, and don't send frames to the detector that don't need to go there.
And the second one is the big one. Motion masks. The balcony is mostly empty, which is the ideal case for masking, because you can mask off everything that isn't the railing and the floor. Every masked region is a frame that never reaches the detector. That's the single largest lever on resource use in the whole config.
Then the object filters. Track only what you need.
Frigate tracks person by default. Bird is not in the tracked list until you add it. So you add bird alongside person and that's the whole change.
And then the question Daniel actually asked, which is how do you stop a human on the balcony from setting this off.
This is where I want to be clear, because it's the part Daniel is overthinking. Person and bird are separate classes in the same model. A human sitting on the balcony is classified as person. A pigeon is classified as bird. There is no ambiguity to engineer. The model already separates them. The work is in constraining where and how big each class is allowed to be.
So it's filters and zones, not classification.
Filters and zones. Object filters let you set a minimum area, a maximum area, an aspect ratio, and a confidence threshold, per object type. So for bird you might say minimum area five thousand pixels, maximum area a hundred thousand, minimum score zero point five, threshold zero point seven. That max area is doing real work. It's saying, if the blob is enormous, it isn't a pigeon.
And a human is enormous relative to a pigeon.
A human is enormous relative to a pigeon. So the max area filter alone kills most of that class of false positive. Then zones scope it further. You draw a zone on the railing and the floor where pigeons actually land, and you only count birds in that zone. A bird on the neighbour's roof doesn't fire. A bird flying past at the edge of frame doesn't fire.
There's something in the event payload about whether the bird has actually landed, isn't there?
There is. The event payload carries stationary, motionless count, and position changes. So an automation can distinguish a bird that has settled on the railing from one that's transiting. If Daniel wants pigeon is on the balcony rather than pigeon flew by, those fields are how you get it. And that's a meaningfully different notification. One of them you can ignore. The other one you want to know about.
So the detection layer is basically solved with a tracked objects list, a couple of filters, a mask, and a zone.
That's the detection layer. The notification layer is where this gets elegant.
Which is the part Daniel already guessed right, and I want to give him credit for it, because his instinct was MQTT into Telegram and that is the path of least resistance.
It is, and the reason is the topic structure. Frigate publishes to frigate slash events for every changed tracked object. Each message has a type, new, update, or end, and a payload with label, camera, score, top score, box, area, current zones, entered zones, sub label, stationary, motionless count, position changes, and flags for whether there's a snapshot or a clip attached.
So new plus label equals bird plus camera equals balcony is your trigger.
That's the trigger. And there's a simpler one if you don't want to parse the full payload. Frigate publishes object counts per camera on frigate slash camera name slash object name, and an active variant that's designed to be a Home Assistant sensor. So frigate slash balcony slash bird slash active is just a number. Greater than zero fires the automation. That's a presence detector in one line.
But the elegant bit is the snapshot.
The elegant bit is the snapshot. Frigate publishes a JPEG-encoded frame of the detected object to frigate slash camera name slash object name slash snapshot. So the Telegram message doesn't say bird detected. The Telegram message carries a photograph of the pigeon.
That is nice. You get the notification and you already know whether it's worth getting up.
And there's no extra plumbing. You're not grabbing a frame from the camera, you're not running a second pipeline. Frigate hands you the image on a topic.
Now, the scenario Daniel raised. What happens when they're actually out on the balcony having a coffee?
Two answers, and neither one is hoping the model never misclassifies. The first is that Frigate exposes frigate slash notifications slash set and frigate slash camera name slash notifications slash suspend. You can turn notifications off, or suspend them for a number of minutes. So when a human is present, you suspend rather than relying on the classifier to be perfect.
Which is the right shape for the problem. Don't make the model solve a problem that a switch solves.
And the second is runtime detection control. Frigate exposes frigate slash camera name slash detect slash set, which turns object detection on or off at runtime, and it persists across restarts. So you could drive that from a balcony door sensor. Door closed, detection on. Door open, detection off. The camera still records, the detector just stops working.
That's a nice one because it also saves the compute. If they're out there for an hour, the i5 isn't running inference on two humans shifting in their chairs.
And it means the false positive question mostly evaporates. You've removed the ambiguous period entirely.
So what's the ridiculous automation? Because Daniel floated playing a warning pigeon soundtrack over the house speakers and I want to know if that's actually possible.
It's trivially possible. MQTT to Home Assistant to a media player is a few lines. And there's a bonus, which is that Frigate also exposes birdseye set and birdseye mode set, with continuous, motion, and objects modes. So a pigeon detection could switch a live view onto a display. You could have the pigeon cam come up on the kitchen screen automatically.
The pigeon is on the balcony, and the balcony is now on the television.
And the soundtrack plays. I want to be clear that this is fully supported and that I am in favour of it.
Noted. Now, species identification. Daniel said it wasn't the objective but that he'd seen some cool models. What's actually there?
This is my favourite part of the whole thing. Frigate has a built-in bird classification enrichment. It's not a third-party add-on, it's in the product. And when a known bird is recognized, its common name gets added as a sub label.
Meaning the notification says something more specific than bird.
The notification says Eurasian Collared-Dove. The sub label shows up in the UI, in filters, and in notifications. So the Telegram push upgrades from bird detected to a named species, and you didn't write any of it.
What's the model?
It's a quantized TensorFlow MobileNet, the INat bird classification model, and it comes out of Google Coral's test data. Which is a nice detail, because the Coral is the thing Daniel couldn't buy, and here's a Coral model running happily on his CPU.
That's a good line. The TPU purchase failed and the TPU's model is doing the work anyway.
And the complexity it adds is close to nothing. It runs a lightweight tflite model on the CPU. Frigate's own docs say there are no significantly different system requirements than running Frigate itself. It's a global config toggle with an optional threshold that defaults to zero point nine.
So it's one block in the config file.
One block in the config file. The only caveat is that it needs a one-time internet connection to pull the model and the label map from GitHub. After that it runs fully offline. So if Daniel's box is air-gapped he needs to plan for that, but otherwise it's a flag.
And this is where the Hugging Face thread comes in, because Daniel flagged this as a segue into a topic you two have been chewing on.
We have been chewing on it. There is an enormous amount of good model work on Hugging Face right now, and a lot of it doesn't get the attention it deserves. But the lesson from this episode is the opposite of go find a model. The model Daniel needed was already in the box. The interesting Hugging Face connection is one layer down.
Which is where?
The AXERA detector in Frigate downloads its default model from Hugging Face on first startup and caches it after that. That's Hugging Face as production infrastructure, not as a research repository. It's the plumbing a shipping product relies on. And then there's Frigate Plus, which offers custom-trained models where you submit annotated images from your own cameras. Their docs make the point that the best detection accuracy comes from a model trained on images that look like what Frigate actually sees. Security camera footage, cropped to regions of interest.
So if the default model turns out to be mediocre at pigeons specifically, the upgrade path is to teach it your pigeons.
Teach it your pigeons. And that's the honest caveat on this whole episode. I don't know how good the default model is at pigeons on a Jerusalem balcony at whatever angle Daniel's webcam sits. The bird class is in the label set, but the label set was trained on general imagery. It might be excellent. It might be fine. It might miss a lot.
And the way to find out is to run it and look at the metrics.
Run it and look at the metrics. Which is exactly what Frigate's docs tell you to do and exactly what I'd tell him. Don't pre-optimize. Turn it on, watch it for a week, and see what it catches and what it misses.
There's something worth saying about the shape of this whole answer, though. Daniel asked what model to use, and the answer was, you already have it, and the work is in the config. That's a pattern.
It's a real pattern in local AI right now. The model is rarely the bottleneck. The integration is. People spend weeks hunting for the perfect model when the one that shipped with the tool would have done the job on day one.
The corollary is that the config is where the skill lives. The mask, the zone, the filter, the suspend switch. That's the actual engineering.
It's less glamorous than picking a model, which is probably why people skip it.
I want to go back to the stones for a second.
The stones.
Hannah put stones around where the pigeons land and they haven't come back. That's the actual working solution. Everything we've described is a notification system. It doesn't deter anything.
That's true, and it's worth being honest about. Frigate tells Daniel a pigeon is there. It doesn't make the pigeon leave. The stones did that.
The system is an early warning, not a defence.
It's an early warning. Which is what Hannah asked for. She asked for presence detection, not a deterrent. But if the stones stop working, the notification tells them the stones stopped working.
Which is arguably the more valuable thing.
Hilbert: The stones aren't a deterrent.
Sorry?
Hilbert: They're a texture change. Pigeons don't like landing on something that shifts under them. That's all it is. I did a season doing pigeon control for a property management company, seventy-four, seventy-five. Not extermination. Deterrents and monitoring. We put down gravel beds, wire, spikes, gel. The gravel worked about as well as anything and it worked for the same reason the stones work. It's not the stone. It's that the stone moves.
It's unstable footing.
Hilbert: It's unstable footing. And it fails the same way every time. The stones get moved, or the birds get used to them. I watched a flock figure out a gravel bed in about three weeks. After that they'd land on the edge of it and walk in. Once they've decided your balcony is theirs, the texture is an inconvenience, not a barrier.
What do you do when they've decided?
Hilbert: You change the geometry. You make the landing surface not a landing surface. Slope it, or narrow it, or put something on it that isn't flat. But that's a building job, not a bag of stones. And most people won't do it, so they buy the next thing.
Which is where you come in.
Hilbert: Which is where I came in. I lost money on a pigeon-proofing franchise once. Turned out the whole operation was a man with a ladder and a bag of spikes. I found that out after I'd paid for it.
Did the spikes work?
Hilbert: They worked on the ledge. They don't work anywhere else. Pigeons don't need the ledge. They need about four inches of anything.
That's the part that maps onto the detection problem. Four inches of anything.
Hilbert: Which is why the camera is the right idea and the notification is the right idea, because the question you actually end up asking isn't is there a pigeon. It's is it the same pigeon.
Say that again.
Hilbert: Is it the same one coming back. That's what people want to know after about a month. They want to know if they're dealing with one bird or a rotation. And I could never answer that. I'd show up, there'd be a pigeon, I'd do the thing, I'd leave. I had no idea if it was the same bird as last Tuesday.
Frigate could actually answer that. The event payload has position changes and motionless count. You could build a fingerprint out of arrival time and landing spot.
Hilbert: You could. I couldn't. I had a clipboard.
What did you write on the clipboard?
Hilbert: Times. Weather. Whether the gel was still tacky. I could tell you what a pigeon sounds like when it lands, though. Different species land different. The collared doves come down heavy. The feral ones are lighter and they flutter more on the way in.
You can identify them by the sound of the landing.
Hilbert: Roughly. I wouldn't bet money on it. I'd bet a small amount of money on it.
That's a better sensor than the webcam.
Hilbert: It's a worse sensor. It just has better training data. Anyway, I have to go. There's an animal expecting me.
The honest answer on accuracy is that he should run it and watch the metrics.
Which brings us to the thing I keep circling, which is that we've spent this whole episode on a system that tells Daniel something he could find out by looking out the window.
It tells him when he isn't looking out the window. That's the whole value. The balcony is empty most of the day and nobody's standing there watching it.
Fair. So the open question is whether the default model is actually good enough for pigeons specifically, or whether he ends up in Frigate Plus training it on his own birds.
Whether the i5 iGPU handles the load in practice, which nobody can answer from a spec sheet. He has to measure it.
Whether species ID is worth turning on from day one or leaving as an upgrade.
I'd turn it on. It's a config flag and it costs nothing. But that's me.
The misconception I want to kill before we go, because it's the one that cost Daniel the most time, is that he needed to go find a model.
He didn't need to find a model. Bird is in the default label set. The hunt was for something already in the box.
The second one, which is subtler, is that a bigger model would see a small bird better.
It wouldn't. Frigate crops and zooms before detection, so the small model is the one that's better at small objects. Bigger is slower and it's built for a different problem.
Thanks to Hilbert Flumingtop for producing. This has been My Weird Prompts. If you're enjoying the show, a review wherever you listen goes a long way.
You can find everything at my weird prompts dot com. We'll be back soon.
