In 2023 and 2024, Flock sent 1,427 alerts to the Roseville [CA] Police Department.
Roseville police found that Flock misread license plates in 71% of those alerts.
Max Isaacs, director of technology law and policy with the Policing Project at New York University’s School of Law, described Roseville’s misread data as “stunning.”
“A 71% misread rate is really beyond the pale,” he told Business Insider, referring to Roseville’s reported error rate for stolen and felony vehicle alerts. He added that “in the absence of concrete rules that directly address the problem of misreads, this technology is not safe to be deployed in communities.”
- True Positives (TP) 415
- False Positives (FP) 1,012
- False Negatives (FN) (Calculated using 96% Recall) 17 Target plates that were missed.
- Precision TP / (TP + FP) 29.1% When it hits, it’s wrong most of the time.
- Recall (Assumed from Global Acc.) 96.0% It successfully catches almost all targets.
That’s the problem if you are fishing for a very small number of targets out of an absolutely bonkers large pool of potential targets - at some point your false positives start becoming more and more prevalent than actual positives. That is simply statistics and if flock can’t improve the global accuracy by some exponents, this problem will grow.
If they flag all the plates, they’re bound to get all the ones they’re aiming for. It’s a simple solution.
I hope this results in their evidence being inadmissible to judges/juries.
If you’re ever on a jury remember this
since its a mass surveillance apparatus, the juror/court was not its intended purpose. they likely profile(errenously) potential people who are going to be a “nuisance” in the future.
True but its nominal and financial justification would be eliminated
I live in a relatively small mountain town near that City and the wonderful citizens of my town have taken a liking to absolutely fucking destroying those things to the point where we’re having a city hall thing on the 10th so that everybody can air their complaints.
don’t forget to clap
huh. i wonder if they don’t have them around my town because the test one they put on the bike route i take to the next town acquired so many holes without my help i felt i didn’t need to add.
Doing God’s work!
For a Trump admin approved device, we’re all shocked it read that well
If it’s the responsibility of the party that’s been misidenified to correct the mistake, this was intentional. Putting fines to random people, knowing a proportion won’t fight it, is a foundationional pillar of Authoritarianism. What’s worse is even more work will now need to be done to undo and correc this, because it’s already in place. The lawmakers will try to get the percentage up, instead of removing the system.
And then China got issue with speed trading. Competent IT people and math people where monopolized for doing fast trading, optimizing algorytme, systems etc. So China banned part of fast trading which triggered a brain migration into Ai. This made these expert work in ai field and they are now removing us competitive advantage buy open sourcing models. And us European are being depassed, Capitalisme and greed killed innovation.
I would want to dispute what you said but honestly I’m unsure what you were trying to say there
Yeah my bad, tiredness for breaking my brain working on bank systems… I al wasting so much energy maintaining old shitty systems which are in this states because greed, where to could be building other things or optimizing system which serve people instead of robbing people.
Those certainly were words.
Some of them weren’t
algorytme, depassed and Capitalisme for sure aren’t.
Capitalisme
us European
French speaker fighting autocorrect, perhaps?
definitely French speaker: diurnambule is a porte-manteau between diurne which means “during the day” and somnambule which means sleepwalker.
It’s a French instance, so that would make sense
How can they suck that bad? A 29€ raspberry pi with a basic ML model can get 90% accurate results
It’s basic statistics.
- True Positives (TP) 415
- False Positives (FP) 1,012
- False Negatives (FN) (Calculated using 96% Recall) 17 Target plates that were missed.
- Precision TP / (TP + FP) 29.1% When it hits, it’s wrong most of the time.
- Recall (Assumed from Global Acc.) 96.0% It successfully catches almost all targets.
That’s the problem if you are fishing for a very small number of targets out of an absolutely bonkers large pool of potential targets - at some point your false positives start becoming more and more prevalent in relation to actual positives. That is simply statistics and if flock can’t improve the global accuracy by some exponents, this problem will grow - the more Flock gets used and the more licence plates they are scanning, the more the false positives increase.
And it is a massive problem, because the false positive marks you as a criminal - that means any legal recourse gets expensive and painful. This technology creates more problems for everyday people than the amount of issues it removes by tracking actual criminals.
Cutting them down with a reciprocating saw is absolutely justified. These things are an affront to privacy and accuse random citizens of being guilty of felonies.
If you have a million stolen licenses, but on the road remain very few, say 1k. While there are say 100m other cars. If you now have 99% accuracy, you will read 990 stolen cars, and misread 1m plates. Since 1% of plates are stolen, you randomly hit 10k of those misreads into a stolen plate.
This is generally alwayd an issue with rolling out mass tests. If the thing you search is even rarer than your failure rate, most detections are errors no matter how good your test.
This is why we don’t just test everyone for diseases all the time even when it would be very cheap and simple.
That’s probably exactly what they did, deployed a sbc + some off the shelf model and increased the price 300x by making a black plastic shell.
They’re actually badly secured android devices.
The trick isn’t the actual tech, it’s the connections and sales pitch.
I guess that they are implementing crappy cameras? Not sure about american plates, but in most jurisdictions licence plates are optimized for easy OCR:
Note how each letter is unique and can’t be transformed with a sharpie into another. That also makes OCR easy. If you use a crappy dashcam, you will get problems regardless. Esp. in low light situations at higher speeds.
So I would guess that Flock uses cheap sensors and cheap lenses. Which kind of makes sense when they are putting out thousands of scanners.
You must remember that these are people who don’t vary the size or colour of their money either.
They’re not clever people.
In some cases in the US, cops enter both O and 0 into the search databases. This probably compounds the number of false positives.
In the US, the states make their own license plate regulations and choices. So, it varies from state to state, like everything else in this hellhole.
The C could be definitely be turned into the G pretty flawlessly.
Not without some whiteout though, notice how the stroke gets thinner as it curves upwards?
Oh yes I see
Yeah I don’t think that is the real license plate typeface. I think that might be a font called Sauerkraut that is a riff on the official one.
Actually the tops of the C and G are different, but you’d have to be looking for it. Plater readers could probably still be fooled.
in my area I struggle to distinguish D, O, and 0. Idk if someone was paid to design that, but they did an awful job
Not sure about american plates, but in most jurisdictions licence plates are optimized for easy OCR:
Are they? Off the top of my head Germany is the only country I can think of that uses such a font (FE-Schrift).
German wiki mentiones 42 countries that also adapted the FE Schrift and 7 that use some adaptions:
I see. I’m in Europe and it’s not very common there so that’s why.
Never seen letters like that on license plates in Quebec.
You’re assuming that all people in the pipeline are competent
If you have a pi reading license plates, you are probably scanning a few dozen different plates a day, looking if they match a handful of possibilities. If the only plate that opens the gate is ABC123, nothing happens if it accidentally reads anything else, and I’ll probably get it right a second later.
When Flock scans plates, it reads thousand of them every second looking to read and match every possible plate variation from every state (which there are over 8000 of) to every reported stolen plate, in all weather and lighting conditions. It’s really no wonder if 71% of the times it reports it found a stolen plate it had got it wrong.
This has nothing to do with weather conditions or lighting. It has to do with basic statistics: if you are looking for a very small subset of identifiers in a very large pool, and you have a system that has a (pretty good btw) 96% chance to read an identifier correctly, you will get more and more false positives the more identifiers you scan.
That means expanding the Flock network also expands the amount of false positives.
This cannot be avoided, except if you are able to make the 96% rate better; but be reminded if this system would get upgraded from 96% to 99.6%, total precision would only increase about 1% (from 29.1% to 30.1%). reducing the number of false positives in this example by only 15 licence plates.
To actually reduce the amount of false positives, you would have to make the system less sensitive - which isn’t possible in this context.
I highly doubt a single flock camera would scan thousands of plates per second lol
There is no good reason for sending images of plates to the cloud for alpr when the camera can do it locally
I did not know there was only one single Flock camera in the city of 150 000 people, or in California, or the USA in general. lol.
it has cameras in more than 6,000 communities across the country, and it touts more than 20 billion vehicle reads each month.
Oh, kinda looks like they have more. 20 billion reads a month, huh. That’s 7610 scan a second. Fancy that.
Flock cameras process the images locally for plate recognition. It’s only after the image of the plate has been converted into text that the plate number (or in this instance the number it has misinterpreted) is sent to AWS for matches and alerts. The issue sounds like it’s at the individual camera level.
…yes?
The cameras around the US scan 7000 plates per second, trying to figure out what they say with 8000 different possible plate designs around the country, and those results are then matched to a database of stolen vehicles and other plates , which there’s roughly 600k reported stolen in the US each year so. That database is going to be massive.
That means every second, 7000 strings of letters and numbers OCR’d at unknown conditions and accuracy are being compared to hundreds of thousands of stolen plates. They are guaranteed to get a number of “matches” constantly, most of them from reads that were not accurate.
And specifically because they aren’t sent anywhere, nobody can double check the picture to see if the camera got it anywhere right at all.
Holy fuck you’re daft, good luck
What does any of this have to do with the cameras from one specific town being wrong a majority of the time? Not that it would be better if it wad the average across the nation. It would be much, much worse actually.
The cameras around the US scan 7000 plates per second
Yes the entire system saving that text, and maybe a still image handles that many… But the single camera taking each photo where the actual OCR processing is done ON THE CAMERA HARDWARE ITSELF only needs to handle maybe one or two each second… And only if it is looking at a busy road. At the distances these cameras can process, most are not handling a lot simultaneously. And very busy roads will have different hardware than the lower end camera systems used in most places.
You HAVE to purposely be trying NOT to understand any of the responses to you here. No one could possibly be that stupid unintentionally.
I don’t think you understand their point. The point is that the OCR can be done on a distributed system, because a distributed system is what the cameras already are.
My biggest contention with this claim is whether the computers onboard the license plate readers are actually capable of doing this with solar power and battery capacity, but considering I’d rather they didn’t exist anyway…
this technology is not safe to be deployed in communities
The main point for AI
On one hand that’s a good thing.
good thing for leo to profile people “errenously”, everyone knows its for people who are going to potentially protest in the future. and abused by DV-prone cops to stalk thier SO.
But on the other its not, as it can cause someone else with a similar plate number to be accused of a crime or infraction, etc that another committed.
Yup, just read an article about that the other day. They even acknowledged it was going to keep happening to the person whose vehicle was mistakenly flagged because of a partial ID match, and just advised him to keep the car at home until they figured out how to fix it. In his case it wasn’t as big a deal since it was a rental, but can you imagine being told to not drive your own fucking car or the cops will be automatically called to come pull you over as a carjacker??
not only that they track the actual drivers/passenger faces too now.
Wow, that is absolutely insane. I hate this current timeline so much.
Dozens of times so far. And the perpetrators point at each other, or simply refuse to acknowledge wrongdoing.
Edit: found the story I was thinking of! This guy was wrongfully added to these lists, and the sheriff only removed him locally after the news stated investigating. And that’s only for one county, so the problem will repeat if he drives to a different county.
Yep, I knew it was happening. Wasn’t aware this frequently already. Of course they point the finger at each other and refuse any wrongdoing, their lives aren’t the ones at stake when shit like that happens. And guarantee the ones falsely accused still face the penalties.
No. Its like drug dogs. Its just an excuse to do whatever they want. Infinite monkeys writing up probable cause forms.
Yay! Great investment. Totally worth the buck.
Get those criminals off the street.
I’m pretty sure I could do better stoned. I mean, I’m typing with a much higher accuracy than 71%, and I’m stoned right now.
and how many false positives in the other 29%?
For context: I once lived in roseville and worked in Sacramento. They shared radio and otaTV stations back when those were a thing. I have heard about (insane) people commuting to sf from there.
This isn’t bumfuck nowhere.
Side note: There’s still over the air television, it’s just digital instead of analog.
