Zol AI Lab
Intelligent Tolling
you are here · 07/07
Back to projects
a toll road, rebuilt with AI
Seven phases, from live traffic on camera to a toll you can trust. This page is phase 07, the last gate before anyone is charged: what happens when the camera isn't sure what it read.
You are here 07/07 Verifythe whole road
Every passage under a gantry has to become one record that a person would sign off: who, what, how fast, how big. Each phase gets its own page as it ships. Today you're standing at 07, the final check before a toll is charged.
01
Real traffic, filmed by a dedicated ultra-HD camera at high frame rates, so every plate is caught sharp in many frames, not one lucky shot.
unlocks soon
02
Every vehicle, plate and wheel found and tracked frame by frame, even when a B-double hides the car beside it.
unlocks soon
03
Each plate read in every frame it appears in, then voted character by character into one answer with a confidence.
unlocks soon
04
The detector and the reader retrained on Australian plates from every state and series, real and synthetic.
unlocks soon
05
Axles and axle groups counted from the side view, turned into the Austroads 12-class code that sets the toll.
unlocks soon
06
Speed from a calibrated road plane, plus length, width and height in metres for every vehicle.
unlocks soon
07
Unsure plates checked against Australian registration data and matched to the vehicle on camera, before anyone is charged.
live on this page
The problem
Why a camera can't tell a 1 from an I
km 0.5 · keep left for the evidence
phase 07 · the problem
On a real toll road, a plate the camera can't read with confidence goes to manual image review: a person squinting at a crop. Phase 07 is the step before that person. Here is why this plate is hard, in three steps.
01 the shapes
On a Victorian plate the 1 has a flag and a foot. The I has a bar on top and a bar below. Up close they're nothing alike. Now move the truck away from the camera.
illustration: plate-style glyphs, rendered at the camera's pixel size
02 the approach
The tracker reads the plate in every frame as the truck comes closer. These are real crops from the footage, far to near. Below them, every tick is one read of the first character.
first read · farlast read · near
03 the vote
Every read votes, character by character
the OCR leans I… but VIC plates go 1AB·2CD
▦ Characters 4 and 5 of every plate on this page are masked. These are real vehicles.
The solution
Four moves from doubt to a decision
km 1.2 · phase 07 in action
how it decides
Everything the machine can work out for itself happens first, on a local GPU. Only the last step asks the outside world.
01 Spell free
All 47 reads vote per character, and reads of a bigger plate count for more. Look-alikes the OCR never showed (1/I/T, 0/D/Q, 8/B) keep a small share, because OCR can be wrong the same way every time. A beam search turns the columns into whole plates, and a spelling that fits no Australian format is ranked lower but not thrown away.
02 Look free · local GPU
RF-DETR segmentation finds the vehicle carrying the plate in every frame. The sharpest, largest uncut views win, which is what matters when traffic is fast. A local vision-language model then describes the vehicle blind, before it sees any registration: type, body, make, colour, and every badge it can actually read.
03 Ask third-party API
Only now does anything leave the machine. Each likely spelling is checked against a third-party API for Australian plate details: is it registered, and to what make and model?
04 Decide free
Each registered spelling faces four checks: make or badge, body, model, and a multiple choice where the model picks between the registered vehicles or "none of these". The score decides: verified, likely, or send to a person.
step 01 · candidates ranked for lookup
#1
#2
the OCR prefers I, the format prefers 1. Check both.
step 02 · the vehicle, according to the video
vision model · blind
type prime mover · cab-over (vote of its 3 best views)
make MAN 1.00
badges "MAN" "Opal." "CAMERON" "917"
colour white pixels said grey (cab in shade)
step 03 · third-party API for Australian plate details
GET /lookup plate=1EM••O state=VIC
200 · found MAN TGS 18.360 · 10.5 L diesel · 2009–on
GET /lookup plate=IEM••O state=VIC
not found no vehicle with this plate
one spelling is a real truck. The other doesn't exist.
step 04 · 1EM••O, MAN TGS vs the camera
+0.75match at ≥ 0.35
VERIFIED 1EM••O
phase 07 · the hard one
Victoria's plate series is so full that almost any valid spelling belongs to somebody. Here, every candidate fits the format. "Is it registered?" can't settle it. Only the vehicle can.
OCR said 1IL••L at 0.52 confidence, from 24 reads of a plate 53 pixels wide.
OCR's answer · rank 1
Ford Fiesta
2014–2019 · hatchback
−0.57 ✕ wrong vehicle
rank 2
Subaru XV
2011–2017 · crossover
−0.59 ✕ wrong vehicle
rank 3
Not registered
no vehicle with this plate
· ruled out
rank 4 · T read as I
Isuzu MU-X
2013–2021 · badge "ISUZU" ✓
+0.61 ✓ verified
phase 07 off
Toll sent to the owner of a Ford Fiesta
✕ wrong person charged · complaint · refund · manual review
a twist
The vision model read the ISUZU badge and called it a D-Max ute. The registration said MU-X, the SUV built on the same platform. From the front they're identical, and the caravan hides the rear. So phase 07 treats front-end twins (D-Max/MU-X, HiLux/Fortuner, Ranger/Everest) as "can't tell", not as a mismatch. The rest of the evidence carried it to +0.61.
Reality check
What free footage can and can't do
km 2.0 · honest footnote ahead
the honest footnote, in big type
Everything on this page ran on free YouTube videos filmed from the roadside, not on tolling cameras. The frames are big, but the plates are tiny, compressed and blurred by motion. Phase 01 replaces this footage with a camera built for the job.
today · the source footage
phase 01 · a camera made for plates
same 1, same I, same maths: only the pixels change
Under the gantry
How fast it sees, thinks and measures, and what it runs on
km 3.4 · four platform phases
the platform under the seven phases
A toll road doesn't slow down for a model. Four platform phases sit underneath the road: how fast the camera can see, how fast the edge box can think, how vehicle speed is measured, and the system that holds it together. Each is labelled measured or design target.
P1
How many clean looks does a plate give at speed? Drag the traffic.
phone or YouTube camera · 1/250 s shutter
phase 01 plate camera · 1/2000 s shutter
Simulated with a sample plate. Blur = distance travelled while the shutter is open; a plate character's stroke is about 10 mm.
P2
Every frame must be finished before the next one lands.
one frame every 40 ms at 25 fps · one NVIDIA GB10 edge box
≈ 66–100frames a second on one edge GPU: 3–4 cameras at 25 fps
8–30 sfor phase 07 on a doubtful plate, off the fast path, so traffic never waits measured
5.6 fpstoday on 4K with the unoptimised baseline (FP32, 26 ms a tile): the gap phase 02 closes measured
P3
Not two lines and a stopwatch: a fit over the whole track, in metres.
position samples
0
fitted speed
–
Each frame, the overview camera puts the vehicle on a map of the road in metres, calibrated against aerial imagery. Speed is the slope of position against time over 30–60 m, with every camera's clock synced by GPS/PTP. Target: within 2 km/h, checked on BrnoCompSpeed and a GPS test car.
P4
From glass to database: what runs where.
raw video never leaves the site · only vehicle records and evidence crops travel
doubtful plates settled correctly, both against blind ground truth
of a single plate, voted character by character
per registered vehicle: make, body, model, multiple choice
of vehicle told apart on camera, from car and ute to prime mover
what the road taught me
01
A full plate series means a misread usually lands on a real car. Checking the vehicle, not just the plate, is the whole point.
02
It read MAN, CAMERON and fleet number 917, but repeated the OCR's 1/I doubt and misread a plate outright. So it describes the vehicle and never votes on the plate.
03
The registration API has no colour, and a shaded white cab reads grey in pixels. Colour goes on the record but never decides.
04
The detector's training set files vans, utes and prime movers under one word. A second look by a local vision model, voted over each vehicle's best views, gives the real type.
built with
DGX Spark · GB10
RF-DETR Seg
SigLIP 2
Python
FFmpeg