Driveway & Doorbell Detection — Evaluation and Redesign

21 Sep 2026. Prompted by a missed delivery at 15:08 and the standing complaint that “the doorbell often doesn’t send me anything despite there being people on the driveway”. Scope: both cameras, the Frigate config, the two driveway automations, and the LLM Vision chain.

---

1. The incident that triggered this

Frigate event	1789999699.24809-q6me9y, camera doorbell, label person
Zones	["doorbell_drive"] — Frigate correctly placed him on the property
Score	0.84375 (max_severity: alert)
Duration	15:08:19 → 15:08:50
Frigate’s own key frame (15:08:29)	A man in a bright yellow high-vis vest, standing at the door
What LLM Vision described (15:08:51)	“A woman with long brown hair walks across the driveway and exits the frame, passing by”
Notification sent	None

Detection worked. Description was of a different person at a different moment — someone on the pavement afterwards — and that description is what suppressed the alert. The only surviving record of the event is a timeline entry that is confidently wrong.

---

2. What is actually installed

Cameras (both Reolink, RTSP)

Camera	Host	Detect stream	Record stream	Zones defined
doorbell	192.168.0.131	h264Preview_01_sub	h264Preview_01_main	doorbell_drive only
garden	192.168.0.37	Streaming/Channels/102	Streaming/Channels/101	none

Frigate v0.18-0, Coral EdgeTPU (USB), detect at 640×480 @ 5 fps.

objects:
  track: [person, dog, cat, bird]      # NOTE: no car
  filters:
    person:
      threshold: 0.80                  # Frigate default is 0.50
 
cameras:
  doorbell:
    snapshots: { required_zones: [doorbell_drive] }
    review:    { alerts: { required_zones: [doorbell_drive] } }
    zones:
      doorbell_drive:
        coordinates: 246,381,442,369,578,337,640,360,640,480,348,480,0,480,0,282

Two competing driveway automations, both enabled:

1. automation.doorbell_notification — id 1706650809764, alias “Driveway: Person Detection Alert”, uses the community blueprint SgtBatten/frigate_0.11_notification.yaml.
2. automation.driveway_person_alert — id 1782940875836, alias “AI-powered driveway alerts with personalized homeowner filtering and rich inline media” (LLM Vision).

LLM Vision chain: image_analyzer + stream_analyzer, both provider 01M31X8CW6JN1W39THECXPSK6G / mimo-v2.5, both reading image_entity: camera.doorbell. Delivery via a local shim (ha-llmvision-proxy.service, :8655). Notify target: Pixel 10 Pro XL only.

---

3. Evidence

3.1 Detection rate — the doorbell is mostly silent

Review segments (Frigate’s own alert/detection records):

Camera	14 Sep	15 Sep	16–20 Sep	21 Sep	Total
doorbell	6	6	0	1	13
garden	4	4	27 (18/19/20)	3	84

Zero doorbell person detections across five consecutive days, while the garden camera on the same box produced 41 segments on 20 Sep alone. The hardware, detector and pipeline were demonstrably healthy. The doorbell simply saw no one — which is not credible for a house that receives deliveries.

Doorbell event history since 2 Jun: 20 events total (18 person, 2 dog). All 18 person events fall on 14 Sep, 15 Sep and 21 Sep.

3.2 Every person detection sits against the score ceiling

Observed top_score for every person event on both cameras:

doorbell: 0.8008 0.8008 0.8047 0.8105 0.8125 0.8145 0.8184 0.8242 0.8320 0.8379 0.8418 0.84375
garden:   0.8047 0.8086 0.8184 0.8242 0.8320 0.84375 0.84375 ...

The model’s ceiling is 0.84375 (54/64 — Coral SSD quantisation). The configured person threshold is 0.80. So the entire surviving detection window is 0.80 → 0.84375: roughly the top 5% of the model’s range. Everything below 0.80 — a person further away, at an angle, partly occluded, in flat light, or small in a 640×480 frame — is discarded silently. No event, no log line, no snapshot.

3.3 The zone includes the pavement and the road edge

doorbell_drive occupies 26.9% of the frame. Its upper boundary runs (0,282) → (246,381) → (442,369) → (578,337) → (640,360).

Measured scene bands in the same 640×480 space (verified against a frame with a labelled grid):

Band	y-range
Sky	0–120
Houses across the street	120–240
Public road	240–280
Pavement / footpath	280–320
Private ground (driveway, porch, hedge)	320–480

The zone’s left edge reaches y=282 at x=0 — i.e. it starts inside the pavement band and slopes down across the left third of the frame, which is hedge and pavement, not driveway. A passer-by walking along the footpath enters the frame at that edge and lands inside doorbell_drive.

Compounding this: the same zone gates snapshots.required_zones and review.alerts.required_zones. So a person who is not in the zone generates no snapshot and no alert at all — while a person on the pavement who is in it generates both.

3.4 The analysis is anchored to the clock, not the event

Both analysers read camera.doorbell live, at whatever moment the automation reaches them:

• image_analyzer runs first — a live still.
• stream_analyzer records 5 s of live stream from call time. It has no file input: image_entity is its only image source.

With mode: single, max_exceeded: silent, and two model calls in sequence, the description step runs 30–60 s after the trigger. The 15:08 event lasted 31 s; Frigate’s clean snapshot for it was not written until 15:08:51.69 — a second after the event ended. The analysis therefore described whatever the camera happened to be looking at once the queue cleared.

Two further faults in the same automation:

• input_text.driveway_last_analysed_event is written as **action 1, before any analysis. An event is marked “handled” before anything is handled; if a model call fails, that event can never be retried — every later message for it is deduped away.
• max_exceeded: silent discards any trigger arriving during the ~30–60 s run with no log entry and no trace.

3.5 car is tested for but never tracked

The automation’s condition accepts after.label in ['person','car'], but objects.track contains no car. Vehicle events can never fire.

---

4. Root causes, ranked

#	Cause	Consequence
R1	person.threshold: 0.80 against a 0.84375 model ceiling	Genuine detections silently discarded — the primary cause of missed people
R2	doorbell_drive polygon includes pavement and road edge	Street passers-by count as on-property; and because the same zone gates snapshots and alerts, off-zone visitors produce nothing at all
R3	Two driveway automations enabled simultaneously	Competing, unpredictable notifications; no single owner
R4	Analysis reads the live camera, not the event	Descriptions of the wrong moment; alerts suppressed on the wrong evidence
R5	garden has no zones	84 meaningless segments/week; the camera cannot distinguish anything

R1 and R2 pull in opposite directions and are individually mis-set: the threshold is doing the noise-rejection that the zone should be doing, and the zone is too loose to do it. The result is simultaneously deaf and over-triggered.

---

5. Design principles for the rebuild

1. Detect generously, gate precisely. The detector’s job is to see people; the zone’s job is to decide whether they matter. Never use the score threshold as the noise filter — it fails silently and unmeasurably.
2. Zones follow the property boundary. Nothing above y≈320 belongs to the property. Zones are tiered: porch → drive → (street implicitly excluded).
3. Alert first, describe second. Detection→notification must be fast and deterministic; the LLM enriches afterwards. An alert must never be hostage to a 30–60 s model call.
4. Anchor every analysis to the event’s own imagery — never to a live camera read.
5. One owner per camera. Delete the duplicate automation rather than reconcile it.
6. Every silent path gets a heartbeat. If the doorbell sees no person for 48 h while the garden sees daylight activity, that is a fault and should be reported.

---

6. Proposed automations

A. Door alert — “someone is at the door” (primary, replaces both existing)

• Trigger: Frigate frigate/events, camera doorbell, after.label == 'person', after.entered_zones contains porch.
• Immediate action (no LLM, sub-second): camera.snapshot → save the frame, and notify with the image. This is the alert that must never be late or missing.
• Follow-up (optional, async): image_analyzer on the saved file via image_file, describing the event’s own frame; a second notification only if it materially changes the picture (“delivery driver with a parcel”).
• Gate: suppress when Joe is home and already at the door (input_boolean), not merely when he is home.

B. Driveway arrival — softer tier

• Person enters drive but not porch, and dwells ≥ 8 s → low-priority notification.
• This is the “someone is on the property” case that should never wake a phone at 03:00, and should be silent while Joe is home and awake.

C. Vehicle arrival / departure

• Add car to objects.track; add a car_drive zone over the driveway.
• Use for arrival detection (useful when out) and for “car moved while you’re away”.

D. Street passers-by — explicit non-events

• No zone extends above y≈320, so the footpath is simply not part of any rule.
• Belt-and-braces: a pavement zone can be used as a negative condition in HA templates (not in Frigate required_zones) for any rule that proves too noisy.

E. Pipeline health watchdog (the fix for “we miss people”)

• Daily: count doorbell person detections vs garden daylight detections.
• Alert if doorbell person detections = 0 for 48 h while garden logged ≥1 in daylight. Weekly positive heartbeat otherwise.
• Also assert: shim :8655 reachable, image_analyzer returns, notify target accepts a message. Silent on success, loud on failure.

---

7. Implementation order

Phase	Change	Risk	Needs
1	person.threshold 0.80 → 0.55–0.60	Low — one line, Frigate restart	Joe’s OK (add-on restart)
1	Replace doorbell_drive with tight porch + drive zones	Low — needs a clean frame to draw against	Joe to confirm the boundary
1	Disable automation.doorbell_notification (duplicate)	Low	Joe’s OK
2	Rebuild automation A on the event frame (camera.snapshot → image_file)	Medium	—
2	Move the dedupe write after a successful analysis; max_exceeded: log	Low	—
3	Add car tracking + zone; automation C	Low	—
3	Watchdog E + weekly heartbeat	Low	—
4	Garden zones (or accept it as a recording-only camera)	Low	Joe’s call on its purpose

Phase 1 alone should restore the missed detections — it removes the silent discard (R1) and the pavement false-positive path (R2) in two config edits.

---

8. Decisions needed from Joe

1. Confirm the property boundary in the frame — is the pavement line at y≈320 right, and where does the driveway actually start on the left? Best done by rendering candidate polygons over a clean frame and iterating.
2. Threshold value — 0.55 (Frigate default-ish, more detections, more noise to gate) vs 0.60 (fewer, safer). Recommendation: 0.55, since the zone now does the gating and we want to stop missing people.
3. What should the garden camera actually do? It currently records 84 events a week that no rule consumes.
4. Notify targets — Pixel 10 Pro XL only at present; is that intended?

---

9. Implemented — 21 Sep 2026

Applied and verified the same evening.

Frigate (/addon_configs/ccab4aaf_frigate-fa/config.yaml, backup .bak-doorbellzones-20260921-160317, add-on restarted):

Setting	Before	After
objects.filters.person.threshold	0.80	0.55
zones	doorbell_drive	doorbell_property, porch
snapshots.required_zones	doorbell_drive	doorbell_property
review.alerts.required_zones	doorbell_drive	doorbell_property

Zone coordinates (640×480 detect space):

doorbell_property: 150,320,530,320,610,480,50,480
porch:             130,400,550,400,610,480,50,480

Back-tested against every historical event before deploying: all 18 doorbell person events still enter doorbell_property (nothing lost), 14 reach porch, and the pavement band (y 280–320) has zero geometric overlap with either zone.

Automations created (all to notify.mobile_app_pixel_10_pro_xl, Android channel Doorbell, importance/priority: high, ttl: 0):

Automation	Trigger	Behaviour
automation.doorbell_button_pressed	binary_sensor.reolink_video_doorbell_poe_visitor → on	Snapshot + instant notification. No AI, no conditions.
automation.doorbell_person_alert	Frigate person event in doorbell_property	Snapshot + instant notification. No AI in this path.
automation.doorbell_person_describe_ai	Same event	3 s delay, image_analyzer on the saved frame, then updates the same notification via a shared tag.

Notification tags: doorbell-press, doorbell-person, driveway-person.

Automations disabled (not deleted — initial_state: false): automation.doorbell_notification (SgtBatten blueprint, referenced the now-removed zone), automation.driveway_person_alert (LLM Vision), automation.doorbell_ringing (broken — see below).

Helpers added: input_text.doorbell_last_alerted, input_text.doorbell_last_described (separate dedupe per path, so a failure in the AI path cannot burn the alert path).

Verified: camera.snapshot → /config/www/doorbell/ works; image_analyzer with image_file returns in 11.6 s (vs ~35 s for the old stream_analyzer), proven end-to-end on a live frame.

The doorbell button press was already broken

automation.doorbell_ringing (blueprint McDAlexander/...) had notify_device: Notify.notify — the blueprint uses that value directly as a service name, so it blasted every notify target, including a stale Chromebook registration. Log from the last actual press (21 Sep 12:39:25):

12:39:26 ERROR mobile_app.notify: Error sending notification to Chromebook
12:39:26 ERROR automation.doorbell_ringing: Error executing script.
         Error for call_service at pos 2: Device(s) with webhook id(s)
         2a0aae35... not connected to local push notifications
12:39:26 ERROR Error while executing automation automation.doorbell_ringing

The automation threw part-way through. The visitor sensor itself works — it fired at 12:39:25 today and twice on 20 Sep at 14:00, on a day when Frigate logged no person event at all. Those two presses are almost certainly the “people at the door we missed”.

---

10. Still open

• Regression, found and fixed the same evening. The first version of the two new alert automations omitted the zone condition — Frigate’s required_zones gates Frigate’s own snapshots and review alerts, not the raw frigate/events MQTT topic the automation listens on. Between 16:04 and 16:16 the lowered threshold surfaced seven distant street pedestrians (scores 0.58–0.77, boxes 1–6% wide, foot-point y≈0.53–0.63) and every one of them notified. All seven had zones: [] — the zone logic was correct, the automation simply wasn’t consulting it. Fixed by adding {{ 'doorbell_property' in trigger.payload_json.after.entered_zones }} to both automations. Lesson: validate a new notification automation against the last hour of real events before enabling it.
• Watch it for a week. The threshold change will surface pavement detections too; they should now produce no snapshot and no alert. If any street traffic still notifies, the zones need tightening — or add objects.filters.person.min_area to stop the tiny distant detections existing at all.
• Whether the doorbell’s sub-stream is native 640×480 or downscaled — if downscaled, raising detect resolution improves scores independently of the threshold.
• The garden camera is deliberately out of scope for now.
• The pipeline health watchdog (§6E) is designed but not yet built.

---

11. Arrival detection (“car trick”) + identity — 21 Sep 2026

Built. car added to Frigate objects.track with filters.car.min_area: 10000 (observed street cars: 1.3k–7k px², foot point y≈0.56–0.60, all correctly zones: []; without the filter the camera produced ~8 car events in 2.5 min). input_boolean.arrival_window + automation.arrival_open_quiet_window open a 3-minute window when a car enters doorbell_property or person.joe arrives home. Inside the window both alert automations send a silent notification (channel “Doorbell quiet”, low importance, tag doorbell-arrival, title “Probably you”) rather than the loud one. Chosen silent over suppressed because a delivery van parking on the drive would otherwise be hidden entirely.

Unproven: no car has yet been observed on the drive — only street traffic. The drive is visible in the lower-right of the doorbell frame, so it should be caught, but this needs a real arrival to confirm.

Identity recognition — blocked on hardware. Frigate 0.18 has face recognition built in (writes sub_label, which flows into notifications), but it requires a CPU with AVX + AVX2. The HA host is a Raspberry Pi 4 Model B, aarch64 — no AVX. Frigate+ does not get around this (the AVX2 requirement is on the recognition models). The only route is Double Take running on the homelab box against Frigate’s MQTT + API, and Double Take is unmaintained — Frigate’s docs now point at a community fork. Not worth the risk while the free arrival signals cover the same ground.

Not done: objects.filters.person.min_area would remove the remaining street-person event noise (494–2627 px² vs ~101,000 px² for the real visitor at the door), but it changes what Frigate tracks and could in principle drop a small distant person. Left alone deliberately — raise it with Joe before changing.

---

12. Watchdog — built 21 Sep 2026

The pipeline is now instrumented, because the two failures found today were both invisible: a threshold above the model’s score ceiling, and automations missing their zone condition. Neither could be seen in the config, and neither announced itself.

doorbell-health.py — silent daily 07:45 (no_agent cron 46f5b08297c6), plus doorbell-weekly-report.sh Sundays 09:25 (e9ca477b1d8a) as the positive heartbeat. doorbell-probe.py runs on the HA host and prints one JSON blob; the local runner ships it over ha-ssh.py and evaluates.

It asserts:

• Frigate config drift — person threshold ≤0.7, zones doorbell_property/porch, required_zones, car tracked, car.min_area.
• Wiring regression — both alert automations still contain entered_zones + doorbell_property and the arrival-window branch. Today’s bug is now a daily check.
• Function, not liveness — a camera at 0 fps passes a ping; zero person detections for 48h does not.
• Delivery — scans ha core logs for notification and LLM Vision failures.
• Hygiene — snapshot-dir growth, free disk.

Trap found while building it: ha core logs emits ANSI colour codes, so line[:19] is \x1b[31m2026-09-21 16 and every timestamp fails to parse — the log scan silently returned nothing until the codes were stripped.

Two further reliability fixes:

• Dedupe marker moved after the work it guards. input_text.doorbell_last_alerted was written as action #1; a transient failure in the snapshot or notify then burned that event permanently. It is now written after the notification. A duplicate is harmless because the shared tag replaces rather than stacks.
• Counters fingerprinted the silent paths — counter.doorbell_car_arrivals and counter.doorbell_quiet_alerts. A car-arrival detector that never fires and one working perfectly look identical from outside; the watchdog flags if arrivals are still 0 after 7 days.

Still unproven / open:

• Notification delivery — Timeout sending notification to Pixel 10 Pro XL at 16:09:56. Suspected cause: the notification tells the phone to fetch /local/doorbell/... from HA, which hangs if the phone cannot reach HA. Needs testing with Joe’s phone away from the house.
• The AI describe path was fixed (max_tokens 2000) but has not yet run successfully in production.
• automation.doorbell_button_pressed has never fired for real.
• The watchdog cannot tell whether a notification was right — only that one was sent. The feedback loop is still Joe.