Terrible detections accuracy #6627
Replies: 6 comments 25 replies
-
|
At 640x360, I wouldn't expect accurate detections on people that don't fill most of the frame. Do you have the ability to set a substream to 720p with a fps of 5? Unless you have very fast objects, there isn't a good reason to have your detect fps set to 10. |
Beta Was this translation helpful? Give feedback.
-
|
The substreams highest resolution is 640x360. In my second config I tried to use the primary stream resized to 1080p. I also tried 720p but it seemed to break everything. Constant crashes, loads of ffmpeg errors, no detections, live views broke, cpu usage through the roof (gpu usage doubled but only 10%). What am I doing wrong in my second config where I'm trying to resize my primary stream to a better resolution for detect? I've read all the docs and I think it's to spec. I must be missing something. |
Beta Was this translation helpful? Give feedback.
-
|
I've just remembered. I have the fps set to 10 so the recorded footage is at 10fps. 5 fps is fine for snapshots/detections but not so good for playing smoother video back. I think I read the detection clips are taken from the detect stream. Is that right? I'm wondering if it is now because I've got a record stream set that is a 25fps stream from the cameras. |
Beta Was this translation helpful? Give feedback.
-
|
The fps you have set for detect does not effect playback of recordings. |
Beta Was this translation helpful? Give feedback.
-
|
Can you post the ffprobe output of each stream? My guess is that your machine just can't handle decoding and resizing that many streams. I would try changing one camera at a time. |
Beta Was this translation helpful? Give feedback.
-
|
So I dropped the cameras down to 5 fps and now I'm getting a crash. I've changed nothing else. Here's a sample, its thousands and thousands of these: and these: |
Beta Was this translation helpful? Give feedback.
-
|
Hmm. Looks like it's running out of cache storage. That's probably a symptom of the actual problem. |
Beta Was this translation helpful? Give feedback.
-
|
So where in Frigate is the cache storage specified? Is that in the Docker compose? I configured that to be 256 after reading how to calculate it based on cameras and resolutions. The disk Frigate is running on is a 240GB SSD and recording storage is a TrueNAS iSCSI target. |
Beta Was this translation helpful? Give feedback.
-
|
Seems like you are referring to shared memory (shm) which is RAM. But there is also temp cache which is RAM. In this case we are referring to TMP cache |
Beta Was this translation helpful? Give feedback.
-
|
Ah, where is that specified? How can I check it? Ah it's in the docker compose. Any documentation on calculating size? |
Beta Was this translation helpful? Give feedback.
-
|
what is it set to now? it's not enough so I'd increase it. There is on calculation since it is generally not much until you download a clip or some other API calls |
Beta Was this translation helpful? Give feedback.
-
|
Good question, how do you view the compose file I originally used? I've never needed to do that before! I used the default compose and only changed bits the documentation covered so guess its what's specified there. Happy to check though |
Beta Was this translation helpful? Give feedback.
-
|
ok, I've deleted the container and started again. I'll see how this gets on. GPU is still sitting at 2.81% which seems low, ffmpeg at between 6-7% dependent on camera, Capture 0.3-1% and Detect 0.3-15.2% (15.2% is because detection is happening on that camera right now I think). Should I be using the i965 driver rather than renderD128 perhaps? preset-intel-qsv-h264 within my config file seemed to cause a lot of issues, vaapi appears to be stable. |
Beta Was this translation helpful? Give feedback.
-
|
I've also found my sub stream for the 4 Hikvisions can go slightly higher so I've done that to remove the resizing on CPU that seems to be happening. Running htop shows each camera spikes to ~20-25% when detecting otherwise just a couple of %. However go2rtc seems to be using 40-50% CPU constantly. GPU use still seems to be around 2%. Current config: mqtt:
enabled: true
host: 192.168.1.10
port: 1883
user: frigate_mqtt
password: passwordhere
detectors:
coral:
type: edgetpu
device: usb
objects:
track:
- person
- dog
- bird
- cat
- bicycle
- car
- motorcycle
- horse
snapshots:
enabled: true
timestamp: true
cameras:
Front_Door:
ffmpeg:
hwaccel_args: preset-vaapi
inputs:
- path: user://127.0.0.1:8554/Front_Door
roles:
- record
- path: user://127.0.0.1:8554/Front_Door_Sub
roles:
- detect
record:
enabled: true
events:
pre_capture: 15
post_capture: 15
detect:
width: 640
height: 480
fps: 5
Front_Garden:
ffmpeg:
hwaccel_args: preset-vaapi
inputs:
- path: user://127.0.0.1:8554/Front_Garden
roles:
- record
- path: user://127.0.0.1:8554/Front_Garden_Sub
roles:
- detect
record:
enabled: true
events:
pre_capture: 15
post_capture: 15
detect:
width: 640
height: 480
fps: 5
Utility_External:
ffmpeg:
hwaccel_args: preset-vaapi
inputs:
- path: user://127.0.0.1:8554/Utility_External
roles:
- record
- path: user://127.0.0.1:8554/Utility_External_Sub
roles:
- detect
record:
enabled: true
events:
pre_capture: 15
post_capture: 15
detect:
width: 640
height: 480
fps: 5
Office_External:
ffmpeg:
hwaccel_args: preset-vaapi
inputs:
- path: user://127.0.0.1:8554/Office_External
roles:
- record
- path: user://127.0.0.1:8554/Office_External_Sub
roles:
- detect
record:
enabled: true
events:
pre_capture: 15
post_capture: 15
detect:
width: 640
height: 480
fps: 5
motion:
mask:
- 440,0,392,360,640,360,640,45,640,0
Garage:
ffmpeg:
hwaccel_args: preset-vaapi
inputs:
- path: user://127.0.0.1:8554/Garage
roles:
- record
- path: user://127.0.0.1:8554/Garage_Sub
roles:
- detect
record:
enabled: true
events:
pre_capture: 15
post_capture: 15
detect:
width: 640
height: 360
fps: 5
Back_Door:
ffmpeg:
hwaccel_args: preset-vaapi
inputs:
- path: user://127.0.0.1:8554/Back_Door
roles:
- record
- path: user://127.0.0.1:8554/Back_Door_Sub
roles:
- detect
record:
enabled: true
retain:
days: 14
mode: motion
events:
retain:
default: 14
mode: active_objects
pre_capture: 15
post_capture: 15
detect:
width: 640
height: 360
fps: 5
go2rtc:
streams:
Front_Door:
- user://user:password@192.168.1.108:554/Streaming/channels/101
Front_Door_Sub:
- user://user:password@192.168.1.108:554/Streaming/channels/102
Front_Garden:
- user://user:password@192.168.1.107:554/Streaming/channels/101
Front_Garden_Sub:
- user://user:password@192.168.1.107:554/Streaming/channels/102
Utility_External:
- user://user:password@192.168.1.105:554/Streaming/channels/101
Utility_External_Sub:
- user://user:password@192.168.1.105:554/Streaming/channels/102
Office_External:
- user://user:password@192.168.1.106:554/Streaming/channels/101
Office_External_Sub:
- user://user:password@192.168.1.106:554/Streaming/channels/102
Garage:
- user://user:password@192.168.1.102/av_stream/ch0
Garage_Sub:
- user://user:password@192.168.1.102/av_stream/ch1
Back_Door:
- user://user:password@192.168.1.103/av_stream/ch0
Back_Door_Sub:
- user://user:password@192.168.1.103/av_stream/ch1
webrtc:
candidates:
- 192.168.1.4:8555
- stun:8555 |
Beta Was this translation helpful? Give feedback.
-
|
Can you show the ffprobe output from the main and sub streams for your cameras? You might also want to only pass streams through go2rtc when necessary rather than for all of your streams. You really only need the streams you want to use for the upgraded live views in there. |
Beta Was this translation helpful? Give feedback.
-
|
So on go2rtc I shouldn't pass through the detect streams (the streams with _sub in the config)? Just the HD streams for viewing and recording? These are the two camera types. [
{
"return_code": 0,
"stderr": "",
"stdout": {
"programs": [],
"streams": [
{
"avg_frame_rate": "25/1",
"codec_long_name": "H.264/AVC/MPEG-4AVC/MPEG-4part10",
"display_aspect_ratio": "16:9",
"height": 1520,
"width": 2688
}
]
}
},
{
"return_code": 0,
"stderr": "",
"stdout": {
"programs": [],
"streams": [
{
"avg_frame_rate": "10/1",
"codec_long_name": "H.264/AVC/MPEG-4AVC/MPEG-4part10",
"display_aspect_ratio": "16:9",
"height": 480,
"width": 640
}
]
}
}
][
{
"return_code": 0,
"stderr": "",
"stdout": {
"programs": [],
"streams": [
{
"avg_frame_rate": "10/1",
"codec_long_name": "H.264/AVC/MPEG-4AVC/MPEG-4part10",
"height": 1080,
"width": 1920
}
]
}
},
{
"return_code": 0,
"stderr": "",
"stdout": {
"programs": [],
"streams": [
{
"avg_frame_rate": "10/1",
"codec_long_name": "H.264/AVC/MPEG-4AVC/MPEG-4part10",
"height": 360,
"width": 640
}
]
}
}
] |
Beta Was this translation helpful? Give feedback.
-
That's what I would suggest, yes. The main stream on your Hikvision cameras is 2688x1520 at 25fps. That's a lot for your machine to decode, and I'm not sure it's realistic to use it for detection on this machine unless you are willing to lower the fps to at least 10, but maybe 5. The other option would be to lower the resolution of the main stream to be something manageable. |
Beta Was this translation helpful? Give feedback.
-
|
So I think I've misunderstood how things work then? The main stream I thought was just used for viewing and the decoding should be done by the cpu via quick sync/vaapi. The sub streams which are lower resolution are used for detection. Again decoding is done by the GPU and CPU just checks for motion on these streams then passes frames to Coral for detection. It looks like I need higher resolution sub streams for better detections which I was hoping I could resize the main stream on the GPU. Apart from detecting motion I didn't think the CPU should be doing much? |
Beta Was this translation helpful? Give feedback.
-
|
Spoke too soon, I seem to be having issues playing back clips from 12hrs+ in the past. The retain option in the config doesn't seem to affect whether the clips are available or not. Snapshots are available. |
Beta Was this translation helpful? Give feedback.
-
|
Check the logs to see why playback is failing |
Beta Was this translation helpful? Give feedback.
-
I've had a look at the masks (which I assume is what you mean by filters) but most of my false detections are Person detections in areas in which you could expect People (paths, garden, drive, doorways etc.). In this situation I can't use masks. If I were detecting horses indoors all the time then obviously masks can prevent that. Unless I'm missing something? |
Beta Was this translation helpful? Give feedback.
-
|
No, by filters I meant filters, not masks https://docs.frigate.video/guides/false_positives |
Beta Was this translation helpful? Give feedback.
-
|
Ah, I saw this and also saw Blake saying not to change them as the defaults were best in another thread. My cameras are covering areas where I want to capture people both up close and further away so reducing the detection area doesn't work for me so well. If I do that I either don't capture people up close to the camera or further away. I'll have a play with both motion and filters to see if I can get something working. The logs for playback failing are: |
Beta Was this translation helpful? Give feedback.
-
|
@StarMonkey1 I am having the same experience, detection accuracy is really, really bad. It detects some garbage as "person" with over 60% confidence, making the detection results pretty much unusable. I tried all of the available models from the Coral website and none of them was good enough, the quality especially drops at night. Regardless if during day or night, they to ignore persons that are a bit further away, despite being clearly visible and start reliably detecting an actual person as a person only at a closer range. The misdetection of random stuff as persons is the worst though. At this point I am almost considering to fine tune one of the models to my conditions, since I could not find a model that would perform well enough to be actually usable in my outdoor setup. |
Beta Was this translation helpful? Give feedback.
-
Hoping I can get some advice.
I've managed to get a config that seems pretty stable but my detections are terrible. Plant pots detected as people, people/dog/horse detections of a pile of things in a hall etc. This is annoying as I have Home Assistant setup to send me an alert when I'm away, I'm getting quite a few detections an hour of non-moving objects as people/various other things. I've had to turn the notifications off so its essentially useless.
I've seen that the default model may not be the best model for CCTV but I also think the sub stream of my cameras may be too low a resolution. I tried to use the record stream for the detect stream but resized the resolution to 1080p after reading that resizing should be carried out by my Intel GPU with vaapi configured. However the GPU usage goes up from 5 to 10% and CPU per camera goes from 14% to 60%. It also resulted in huge amounts of errors and regular crashes with the cameras page not showing live images anymore. Perhaps my Intel GPU isn't being used as I thought it was?
I'm running Frigate in Docker on Ubuntu Server on a Dell Wyse 5070 with USB Coral (Intel J4105 CPU), 8GB RAM. I have 2 cheap 1080P PTZ cameras and the rest are 4MP Hikvisions. What am I doing wrong/what can I do to improve the detection accuracy?
I used a lightly modified version of the default Docker compose in the Frigate documentation.
My stable config file but with poor accuracy in detections is:
The config file where I tried to raise the detection resolution to 1080P but it broke everything was:
Beta Was this translation helpful? Give feedback.
All reactions