Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Config Support]: v12.x possibly causing HA core to crash #6485

Closed
xbmcnut opened this issue May 14, 2023 · 26 comments
Closed

[Config Support]: v12.x possibly causing HA core to crash #6485

xbmcnut opened this issue May 14, 2023 · 26 comments
Labels
stale support triage

Comments

@xbmcnut
Copy link

xbmcnut commented May 14, 2023 •
edited
Loading

Describe the problem you are having

It appears as though the Frigate add-on crashes and then for some reason, that crashes the deCONZ add-on and then HA core crashes too all the way to the point where I can’t even ping HA any more so the ability to SSH and check or restart anything has gone.

This has happened about 5~6 times over the last 12 months on two different hardware platforms, with the last three events happening in the past 7 days on my new 12th gen Core i5 Beelink SEi12. In typically ‘smoke detector battery warning’ fashion, the last two episodes happened at 0400 and 0200. 🙁 The previous hardware was an Intel NUC with a 5th gen core i7.

The only way to recover HA is to force shutdown the hardware and reboot. Arghh!!!

I have an Uptime Kuma monitor on my Synology NAS that polls the MQTT topic frigate/available and checks for the payload online which is the only way I get notified something has failed. I've since added another monitor to ping HA itself.

I have disabled Frigate for now as I have things like my hot water managed via HA and when HA fails and we have no water, I start walking with a limp!

Version

12

Frigate config file

## https://docs.frigate.video/configuration/index#full-configuration-reference ##
## Full configuration File ##
logger:
  # Optional: default log level (default: shown below)
  default: info
  # Optional: module by module log level configuration
  logs:
    frigate.app: debug

mqtt:
  host: 10.0.1.100

detectors:
  coral:
    type: edgetpu
    device: usb

rtmp:
  # Optional: Enable the RTMP stream (default: True)
  enabled: false

birdseye:
  enabled: True
  width: 1280  
  height: 720
  quality: 8
  mode: objects # motion
 
ffmpeg:
  hwaccel_args: preset-vaapi #-hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p

detect:
  width: 1280 # <---- update for your camera's resolution
  height: 720 # <---- update for your camera's resolution
  fps: 6

record: # <----- Enable recording
  enabled: True
  events:
    #max_seconds: 300 # default value = 300
    pre_capture: 2 # default = 5
    post_capture: 5
    objects:
      - person
      - cat
    required_zones: []
    retain:
      default: 7 # default = 10

snapshots: # <----- Enable snapshots
  enabled: True
  bounding_box: True 
  # Optional: crop the snapshot (default: shown below)
  crop: False
  # Optional: height to resize the snapshot to (default: original size)
  #height:    

# Optional: in-feed timestamp style configuration
# NOTE: Can be overridden at the camera level
timestamp_style:
  # Optional: Position of the timestamp (default: shown below)
  #           "tl" (top left), "tr" (top right), "bl" (bottom left), "br" (bottom right)
  position: "bl"
  # Optional: Format specifier conform to the Python package "datetime" (default: shown below)
  #           Additional Examples:
  #             german: "%d.%m.%Y %H:%M:%S"
  format: "%d.%m.%Y %H:%M:%S"
  # Optional: Color of font
  color:
    # All Required when color is specified (default: shown below)
    red: 255 # default = 255
    green: 255 # default = 255
    blue: 255 # default = 255
  # Optional: Line thickness of font (default: shown below)
  thickness: 2 # default = 2
  # Optional: Effect of lettering (default: shown below)
  #           "solid" (solid background in inverse color of font)
  #           "shadow" (shadow for font)
  effect: solid

go2rtc:
  streams:
    driveway_frigate:
      - rtsp://username:password@10.0.30.14:554/Streaming/Channels/103 #(3rd stream h.264, 720p, 6fps)
    entry_frigate:
      - rtsp://username:password@10.0.30.12:554/Streaming/Channels/103
    path_frigate:
      - rtsp://username:password@10.0.30.13:554/cam/realmonitor?channel=1&subtype=02 #(3rd stream h.264, 720p, 6fps)
    deck_frigate:
      - rtsp://username:password@10.0.30.11:554/cam/realmonitor?channel=1&subtype=02
    spa_frigate:
      - rtsp://username:password@10.0.30.15:554/Streaming/Channels/103
    backyard_frigate:
      - rtsp://username:password@10.0.30.7:554/Streaming/Channels/103

##################    
#### DRIVEWAY ####
##################  

cameras:
  driveway_frigate: # <------ Name the camera    
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/driveway_frigate # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - detect
            - record           
    motion:
      # Increasing this value will make motion detection less sensitive
      # threshold: 30
      # default 25
      # Minimum size in pixels in the resized motion image that counts as motion.
      # Increasing this value will prevent smaller areas of motion from being detected. Decreasing will
      # make motion detection more sensitive to smaller moving objects.
      # As a rule of thumb:
      #  - 15 - high sensitivity
      #  - 30 - medium sensitivity
      #  - 50 - low sensitivity
      # contour_area: 35
      # default: 30    
      mask:
        - 1280,282,1219,234,1144,210,982,159,886,117,819,132,592,720,1280,720
        - 601,0,510,73,0,103,0,0
    objects:
      # Optional: list of objects to track from labelmap.txt (default: shown below)
      track:
        - person
        - cat
      filters:
        person:
          # Optional: minimum width*height of the bounding box for the detected object (default: 0)
          min_area: 5000
          # Optional: maximum width*height of the bounding box for the detected object (default: 24000000)
          max_area: 250000
          # Optional: minimum score for the object to initiate tracking (default: shown below)
          min_score: 0.5
          # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
          threshold: 0.8 
    # Optional: Configuration for the jpg snapshots published via MQTT
    mqtt:
      # Optional: Enable publishing snapshot via mqtt for camera (default: shown below)
      # NOTE: Only applies to publishing image data to MQTT via 'frigate/<camera_name>/<object_name>/snapshot'.
      # All other messages will still be published.
      enabled: True
      # Optional: print a timestamp on the snapshots (default: shown below)
      timestamp: True
      # Optional: draw bounding box on the snapshots (default: shown below)
      bounding_box: True
      # Optional: crop the snapshot (default: shown below)
      crop: false
      # Optional: height to resize the snapshot to (default: shown below)
      height: 540
      # Optional: jpeg encode quality (default: shown below)
      quality: 90
      # Optional: Restrict mqtt messages to objects that entered any of the listed zones (default: no required zones)
      required_zones: []

  entry_frigate: # <------ Name the camera    
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/entry_frigate # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - detect
            - record             
    motion:
      # Increasing this value will make motion detection less sensitive
      # threshold: 30
      # default 25
      # Minimum size in pixels in the resized motion image that counts as motion.
      # Increasing this value will prevent smaller areas of motion from being detected. Decreasing will
      # make motion detection more sensitive to smaller moving objects.
      # As a rule of thumb:
      #  - 15 - high sensitivity
      #  - 30 - medium sensitivity
      #  - 50 - low sensitivity
      # contour_area: 35
      # default: 30        
      mask:    
        - 0,575,733,325,669,0,0,0
        - 709,361,969,347,947,131,691,141
        - 1280,264,1280,720,1053,720,1001,290              
    objects:
      # Optional: list of objects to track from labelmap.txt (default: shown below)
      track:
        - person
        - cat
      filters:
        person:
          # Optional: minimum width*height of the bounding box for the detected object (default: 0)
          min_area: 10000
          # Optional: maximum width*height of the bounding box for the detected object (default: 24000000)
          max_area: 225000
          # Optional: minimum score for the object to initiate tracking (default: shown below)
          min_score: 0.5
          # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
          threshold: 0.8
    # Optional: Configuration for the jpg snapshots published via MQTT
    mqtt:
      # Optional: Enable publishing snapshot via mqtt for camera (default: shown below)
      # NOTE: Only applies to publishing image data to MQTT via 'frigate/<camera_name>/<object_name>/snapshot'.
      # All other messages will still be published.
      enabled: True
      # Optional: print a timestamp on the snapshots (default: shown below)
      timestamp: True
      # Optional: draw bounding box on the snapshots (default: shown below)
      bounding_box: True
      # Optional: crop the snapshot (default: shown below)
      crop: false
      # Optional: height to resize the snapshot to (default: shown below)
      height: 540
      # Optional: jpeg encode quality (default: shown below)
      quality: 90
      # Optional: Restrict mqtt messages to objects that entered any of the listed zones (default: no required zones)
      required_zones: []

  path_frigate: # <------ Name the camera    
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/path_frigate # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - detect
            - record             
    motion:
      # Increasing this value will make motion detection less sensitive
      # threshold: 30
      # default 25
      # Minimum size in pixels in the resized motion image that counts as motion.
      # Increasing this value will prevent smaller areas of motion from being detected. Decreasing will
      # make motion detection more sensitive to smaller moving objects.
      # As a rule of thumb:
      #  - 15 - high sensitivity
      #  - 30 - medium sensitivity
      #  - 50 - low sensitivity
      # contour_area: 35
      # default: 30        
      mask:    
        - 1035,0,985,569,617,687,385,291,369,0
        - 1243,70,1246,31,961,29,959,69         
    objects:
      # Optional: list of objects to track from labelmap.txt (default: shown below)
      track:
        - person
        - cat
      filters:
        person:
          # Optional: minimum width*height of the bounding box for the detected object (default: 0)
          min_area: 5000
          # Optional: maximum width*height of the bounding box for the detected object (default: 24000000)
          max_area: 150000
          # Optional: minimum score for the object to initiate tracking (default: shown below)
          min_score: 0.5
          # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
          threshold: 0.8
    # Optional: Configuration for the jpg snapshots published via MQTT
    mqtt:
      # Optional: Enable publishing snapshot via mqtt for camera (default: shown below)
      # NOTE: Only applies to publishing image data to MQTT via 'frigate/<camera_name>/<object_name>/snapshot'.
      # All other messages will still be published.
      enabled: True
      # Optional: print a timestamp on the snapshots (default: shown below)
      timestamp: True
      # Optional: draw bounding box on the snapshots (default: shown below)
      bounding_box: True
      # Optional: crop the snapshot (default: shown below)
      crop: false
      # Optional: height to resize the snapshot to (default: shown below)
      height: 540
      # Optional: jpeg encode quality (default: shown below)
      quality: 90
      # Optional: Restrict mqtt messages to objects that entered any of the listed zones (default: no required zones)
      required_zones: []

  deck_frigate: # <------ Name the camera    
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/deck_frigate # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - detect
            - record             
    motion:
      # Increasing this value will make motion detection less sensitive
      # threshold: 30
      # default 25
      # Minimum size in pixels in the resized motion image that counts as motion.
      # Increasing this value will prevent smaller areas of motion from being detected. Decreasing will
      # make motion detection more sensitive to smaller moving objects.
      # As a rule of thumb:
      #  - 15 - high sensitivity
      #  - 30 - medium sensitivity
      #  - 50 - low sensitivity
      # contour_area: 35
      # default: 30        
      mask:  
        - 724,0,1280,0,1280,402,834,202,802,172,673,135,536,66,329,78,300,0
        - 1280,720,1280,590,1186,621,1156,720   
    objects:
      # Optional: list of objects to track from labelmap.txt (default: shown below)
      track:
        - person
      filters:
        person:
          # Optional: minimum width*height of the bounding box for the detected object (default: 0)
          min_area: 5000
          # Optional: maximum width*height of the bounding box for the detected object (default: 24000000)
          max_area: 150000
          # Optional: minimum score for the object to initiate tracking (default: shown below)
          min_score: 0.5
          # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
          threshold: 0.8
    # Optional: Configuration for the jpg snapshots published via MQTT
    mqtt:
      # Optional: Enable publishing snapshot via mqtt for camera (default: shown below)
      # NOTE: Only applies to publishing image data to MQTT via 'frigate/<camera_name>/<object_name>/snapshot'.
      # All other messages will still be published.
      enabled: True
      # Optional: print a timestamp on the snapshots (default: shown below)
      timestamp: True
      # Optional: draw bounding box on the snapshots (default: shown below)
      bounding_box: True
      # Optional: crop the snapshot (default: shown below)
      crop: false
      # Optional: height to resize the snapshot to (default: shown below)
      height: 540
      # Optional: jpeg encode quality (default: shown below)
      quality: 90
      # Optional: Restrict mqtt messages to objects that entered any of the listed zones (default: no required zones)
      required_zones: []

  spa_frigate: # <------ Name the camera    
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/spa_frigate # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - detect
            - record             
    motion:
      # Increasing this value will make motion detection less sensitive
      # threshold: 30
      # default 25
      # Minimum size in pixels in the resized motion image that counts as motion.
      # Increasing this value will prevent smaller areas of motion from being detected. Decreasing will
      # make motion detection more sensitive to smaller moving objects.
      # As a rule of thumb:
      #  - 15 - high sensitivity
      #  - 30 - medium sensitivity
      #  - 50 - low sensitivity
      # contour_area: 35
      # default: 30        
      mask:
        - 666,94,492,117,330,152,0,254,0,169,0,93,0,0,63,0,158,0,332,0,942,0,897,140
        - 1220,0,1216,150,1068,124,1073,0
    objects:
      # Optional: list of objects to track from labelmap.txt (default: shown below)
      track:
        - person
      filters:
        person:
          # Optional: minimum width*height of the bounding box for the detected object (default: 0)
          min_area: 5000
          # Optional: maximum width*height of the bounding box for the detected object (default: 24000000)
          max_area: 150000
          # Optional: minimum score for the object to initiate tracking (default: shown below)
          min_score: 0.7
          # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
          threshold: 0.85
    # Optional: Configuration for the jpg snapshots published via MQTT
    mqtt:
      # Optional: Enable publishing snapshot via mqtt for camera (default: shown below)
      # NOTE: Only applies to publishing image data to MQTT via 'frigate/<camera_name>/<object_name>/snapshot'.
      # All other messages will still be published.
      enabled: True
      # Optional: print a timestamp on the snapshots (default: shown below)
      timestamp: True
      # Optional: draw bounding box on the snapshots (default: shown below)
      bounding_box: True
      # Optional: crop the snapshot (default: shown below)
      crop: false
      # Optional: height to resize the snapshot to (default: shown below)
      height: 540
      # Optional: jpeg encode quality (default: shown below)
      quality: 90
      # Optional: Restrict mqtt messages to objects that entered any of the listed zones (default: no required zones)
      required_zones: []

  backyard_frigate: # <------ Name the camera    
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/backyard_frigate # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - detect
            - record             
    motion:
      # Increasing this value will make motion detection less sensitive
      # threshold: 30
      # default 25
      # Minimum size in pixels in the resized motion image that counts as motion.
      # Increasing this value will prevent smaller areas of motion from being detected. Decreasing will
      # make motion detection more sensitive to smaller moving objects.
      # As a rule of thumb:
      #  - 15 - high sensitivity
      #  - 30 - medium sensitivity
      #  - 50 - low sensitivity
      # contour_area: 35
      # default: 30        
      mask:
        - 1280,0,1280,190,1198,185,1173,202,1023,193,742,181,521,251,507,476,259,561,0,620,0,0,783,0
        - 46,720,424,720,427,674,45,675
    objects:
      # Optional: list of objects to track from labelmap.txt (default: shown below)
      track:
        - person
        - cat
      filters:
        person:
          # Optional: minimum width*height of the bounding box for the detected object (default: 0)
          min_area: 5000
          # Optional: maximum width*height of the bounding box for the detected object (default: 24000000)
          max_area: 225000
          # Optional: minimum score for the object to initiate tracking (default: shown below)
          min_score: 0.5
          # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
          threshold: 0.8
    # Optional: Configuration for the jpg snapshots published via MQTT
    mqtt:
      # Optional: Enable publishing snapshot via mqtt for camera (default: shown below)
      # NOTE: Only applies to publishing image data to MQTT via 'frigate/<camera_name>/<object_name>/snapshot'.
      # All other messages will still be published.
      enabled: True
      # Optional: print a timestamp on the snapshots (default: shown below)
      timestamp: True
      # Optional: draw bounding box on the snapshots (default: shown below)
      bounding_box: True
      # Optional: crop the snapshot (default: shown below)
      crop: false
      # Optional: height to resize the snapshot to (default: shown below)
      height: 540
      # Optional: jpeg encode quality (default: shown below)
      quality: 90
      # Optional: Restrict mqtt messages to objects that entered any of the listed zones (default: no required zones)
      required_zones: []

Relevant log output

Nothing relevant. Here are the last few entries of the HA log before it stopped working.
11/05/2023	14:06:29	Error	homeassistant	user	homeassistant	�[36m2023-05-11 14:06:28.876 DEBUG (MainThread) [custom_components.frigate] Finished fetching frigate data in 3.708 seconds (success: True)�[0m
11/05/2023	14:06:21	Error	homeassistant	user	homeassistant	�[36m2023-05-11 14:06:20.876 DEBUG (MainThread) [custom_components.frigate] Finished fetching frigate data in 3.708 seconds (success: True)�[0m
11/05/2023	14:06:13	Error	homeassistant	user	homeassistant	�[36m2023-05-11 14:06:12.878 DEBUG (MainThread) [custom_components.frigate] Finished fetching frigate data in 3.710 seconds (success: True)�[0m

Frigate stats

No response

Operating system

HassOS

Install method

HassOS Addon

Coral version

USB

Any other information that may be helpful

No response
@xbmcnut
Copy link
Author

xbmcnut commented May 14, 2023

Seems related to #6477 and #6469

@NickM-27
Copy link
Collaborator

also related to #5976

there any other instances as well, and there are many different root causes. Some were storage configuration issues and others seem to be related to go2rtc being used with cheap cameras (wyze, tapo, yi hack, etc.) which have poor rtsp implementation

@xbmcnut
Copy link
Author

xbmcnut commented May 14, 2023

FYI, I only have two brands across my 6 cameras, Hikvision and Dahua.

@xbmcnut xbmcnut mentioned this issue May 16, 2023
Closed
@NickM-27
Copy link
Collaborator

If you're not able to try different combinations then it will be difficult to troubleshoot and find the cause. Could try disabling hwaccel as recommended in the other threads or you could try using the i965 driver instead of iHD

@xbmcnut
Copy link
Author

xbmcnut commented May 17, 2023 •
edited
Loading

If you're not able to try different combinations then it will be difficult to troubleshoot and find the cause. Could try disabling hwaccel as recommended in the other threads or you could try using the i965 driver instead of iHD

I'm happy to try some combinations but it's risky for me with so many automations depending on HA. FYI, I upgraded to v12.x while I still had my old hardware which was an Intel 5th gen core i7 with 16GB RAM also under HASSOS. I was using hwaccel_args: preset-vaapi with that too but under v11.x, it was -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p. After upgrading to an Intel 12th gen core i5 also with 16GB RAM, I set it to hwaccel_args: preset-vaapi also.

It crashed on both hardware platforms.

I'll try without hwaccel and report back.

And in regards your last comment, does the 12th gen Intel i5-1235U with Intel XE graphics support i965?

@NickM-27
Copy link
Collaborator

If you are working with 12th gen then the i965 driver is not relevant. In any case you have less control of your system with HA OS so there are less options to try regarding hwaccel drivers. I know HA OS 10 includes new drivers for iHD so this may be affecting things as well.

@xbmcnut
Copy link
Author

xbmcnut commented May 17, 2023

What's the best way to capture the relevant logs persistently for the hwaccel errors? I want to help but not sure how to set up the environment to provide what you need. At the moment, I've disabled hwaccel altogether and I'm monitoring HassOS free system memory to see if that's being impacted. Not sure if that's the right thing to watch.

With hwaccel disabled and my 6 cameras capturing 1280 x 720 @ 6fps (not recording as I have Synology SS for that), my CPU is hardly breaking a sweat at around 11%. On the older 5th gen i7, it used to sit at 35~40% with hwaccel on!

If I can get some advice on the correct logging parameters for both Frigate and HA (if needed), I can turn hwaccel back on and see what happens.

N.B My current logging setup for Frigate is shown in the frigate.yaml file above.

@xbmcnut
Copy link
Author

xbmcnut commented May 18, 2023 •
edited
Loading

Just checked the logs and have some errors even with hwaccel off. Not sure if relevant.

2023-05-17 01:34:09.408449154  [INFO] Starting Frigate...
2023-05-17 01:34:10.029551390  [2023-05-17 13:34:10] frigate.app                    INFO    : Starting Frigate (0.12.0-da3e197)
2023-05-17 01:34:10.057982063  [2023-05-17 13:34:10] frigate.app                    INFO    : Creating directory: /tmp/cache
2023-05-17 01:34:10.058672893  [2023-05-17 13:34:10] peewee_migrate                 INFO    : Starting migrations
2023-05-17 01:34:10.060741545  [2023-05-17 13:34:10] peewee_migrate                 INFO    : There is nothing to migrate
2023-05-17 01:34:10.069873522  [2023-05-17 13:34:10] detector.coral                 INFO    : Starting detection process: 587
2023-05-17 01:34:13.927588564  [2023-05-17 13:34:10] frigate.app                    INFO    : Output process started: 589
2023-05-17 01:34:13.927790357  [2023-05-17 13:34:10] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as usb
2023-05-17 01:34:13.927973539  [2023-05-17 13:34:10] frigate.app                    INFO    : Camera processor started for driveway_frigate: 595
2023-05-17 01:34:13.928146797  [2023-05-17 13:34:10] frigate.app                    INFO    : Camera processor started for entry_frigate: 597
2023-05-17 01:34:13.928290184  [2023-05-17 13:34:10] frigate.app                    INFO    : Camera processor started for path_frigate: 599
2023-05-17 01:34:13.928427008  [2023-05-17 13:34:10] frigate.app                    INFO    : Camera processor started for deck_frigate: 600
2023-05-17 01:34:13.928582834  [2023-05-17 13:34:10] frigate.app                    INFO    : Camera processor started for spa_frigate: 602
2023-05-17 01:34:13.928722282  [2023-05-17 13:34:10] frigate.app                    INFO    : Camera processor started for backyard_frigate: 604
2023-05-17 01:34:13.928851165  [2023-05-17 13:34:10] frigate.app                    INFO    : Capture process started for driveway_frigate: 605
2023-05-17 01:34:13.928987293  [2023-05-17 13:34:10] frigate.app                    INFO    : Capture process started for entry_frigate: 609
2023-05-17 01:34:13.929132967  [2023-05-17 13:34:10] frigate.app                    INFO    : Capture process started for path_frigate: 614
2023-05-17 01:34:13.929247261  [2023-05-17 13:34:10] frigate.app                    INFO    : Capture process started for deck_frigate: 620
2023-05-17 01:34:13.929366412  [2023-05-17 13:34:10] frigate.app                    INFO    : Capture process started for spa_frigate: 631
2023-05-17 01:34:13.929489968  [2023-05-17 13:34:10] frigate.app                    INFO    : Capture process started for backyard_frigate: 638
2023-05-17 01:34:13.934901699  [2023-05-17 13:34:13] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found
2023-05-17 16:53:24.552476131  [2023-05-18 04:53:24] frigate.video                  ERROR   : deck_frigate: Unable to read frames from ffmpeg process.
2023-05-17 16:53:24.552615628  [2023-05-18 04:53:24] frigate.video                  ERROR   : deck_frigate: ffmpeg process is not running. exiting capture thread...
2023-05-17 16:53:31.267803613  [2023-05-18 04:53:31] watchdog.deck_frigate          ERROR   : Ffmpeg process crashed unexpectedly for deck_frigate.
2023-05-17 16:53:31.267923802  [2023-05-18 04:53:31] watchdog.deck_frigate          ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-05-17 16:53:31.271039157  [2023-05-18 04:53:31] ffmpeg.deck_frigate.detect     ERROR   : [segment @ 0x5634bdc67000] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2023-05-17 16:53:31.271143121  [2023-05-18 04:53:31] ffmpeg.deck_frigate.detect     ERROR   : [swscaler @ 0x5634bdd3c880] [swscaler @ 0x5634bdd49840] deprecated pixel format used, make sure you did set range correctly
2023-05-17 16:53:31.271213546  [2023-05-18 04:53:31] ffmpeg.deck_frigate.detect     ERROR   : [swscaler @ 0x5634bdd3c880] [swscaler @ 0x5634be125080] deprecated pixel format used, make sure you did set range correctly
2023-05-17 16:53:31.271299411  [2023-05-18 04:53:31] ffmpeg.deck_frigate.detect     ERROR   : [swscaler @ 0x5634be122dc0] [swscaler @ 0x5634be12fdc0] deprecated pixel format used, make sure you did set range correctly
2023-05-17 16:53:31.271382522  [2023-05-18 04:53:31] ffmpeg.deck_frigate.detect     ERROR   : [swscaler @ 0x5634be122dc0] [swscaler @ 0x5634bdd52380] deprecated pixel format used, make sure you did set range correctly
2023-05-17 16:53:31.271469377  [2023-05-18 04:53:31] ffmpeg.deck_frigate.detect     ERROR   : [swscaler @ 0x5634be122dc0] [swscaler @ 0x5634be12fdc0] deprecated pixel format used, make sure you did set range correctly
2023-05-17 16:53:31.271538617  [2023-05-18 04:53:31] ffmpeg.deck_frigate.detect     ERROR   : [swscaler @ 0x5634be122dc0] [swscaler @ 0x5634bdd537c0] deprecated pixel format used, make sure you did set range correctly
2023-05-17 16:53:31.271612664  [2023-05-18 04:53:31] ffmpeg.deck_frigate.detect     ERROR   : [swscaler @ 0x5634be122dc0] [swscaler @ 0x5634be12fdc0] deprecated pixel format used, make sure you did set range correctly
2023-05-17 16:53:31.271686129  [2023-05-18 04:53:31] ffmpeg.deck_frigate.detect     ERROR   : [swscaler @ 0x5634be122dc0] [swscaler @ 0x5634bdd4a000] deprecated pixel format used, make sure you did set range correctly
2023-05-17 16:53:31.271765868  [2023-05-18 04:53:31] ffmpeg.deck_frigate.detect     ERROR   : [segment @ 0x5634bdc67000] Non-monotonous DTS in output stream 0:0; previous: 0, current: 0; changing to 1. This may result in incorrect timestamps in the output file.
2023-05-17 16:53:31.271842822  [2023-05-18 04:53:31] ffmpeg.deck_frigate.detect     ERROR   : rtsp://127.0.0.1:8554/deck_frigate: Connection timed out
2023-05-17 16:53:31.271921368  [2023-05-18 04:53:31] ffmpeg.deck_frigate.detect     ERROR   :     Last message repeated 1 times
2023-05-17 16:53:36.389267278  [2023-05-18 04:53:36] frigate.video                  ERROR   : deck_frigate: Unable to read frames from ffmpeg process.
2023-05-17 16:53:36.389472697  [2023-05-18 04:53:36] frigate.video                  ERROR   : deck_frigate: ffmpeg process is not running. exiting capture thread...
2023-05-17 16:53:41.274548446  [2023-05-18 04:53:41] watchdog.deck_frigate          ERROR   : Ffmpeg process crashed unexpectedly for deck_frigate.
2023-05-17 16:53:41.274620696  [2023-05-18 04:53:41] watchdog.deck_frigate          ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-05-17 16:53:41.274741237  [2023-05-18 04:53:41] ffmpeg.deck_frigate.detect     ERROR   : rtsp://127.0.0.1:8554/deck_frigate: Invalid data found when processing input
2023-05-17 16:53:46.401344415  [2023-05-18 04:53:46] frigate.video                  ERROR   : deck_frigate: Unable to read frames from ffmpeg process.
2023-05-17 16:53:46.401517751  [2023-05-18 04:53:46] frigate.video                  ERROR   : deck_frigate: ffmpeg process is not running. exiting capture thread...
2023-05-17 16:53:51.284760640  [2023-05-18 04:53:51] watchdog.deck_frigate          ERROR   : Ffmpeg process crashed unexpectedly for deck_frigate.
2023-05-17 16:53:51.284950648  [2023-05-18 04:53:51] watchdog.deck_frigate          ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-05-17 16:53:51.285132050  [2023-05-18 04:53:51] ffmpeg.deck_frigate.detect     ERROR   : rtsp://127.0.0.1:8554/deck_frigate: Invalid data found when processing input
2023-05-17 16:53:56.406672477  [2023-05-18 04:53:56] frigate.video                  ERROR   : deck_frigate: Unable to read frames from ffmpeg process.
2023-05-17 16:53:56.407129614  [2023-05-18 04:53:56] frigate.video                  ERROR   : deck_frigate: ffmpeg process is not running. exiting capture thread...
2023-05-17 16:54:01.291743241  [2023-05-18 04:54:01] watchdog.deck_frigate          ERROR   : Ffmpeg process crashed unexpectedly for deck_frigate.
2023-05-17 16:54:01.291773441  [2023-05-18 04:54:01] watchdog.deck_frigate          ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-05-17 16:54:01.291892245  [2023-05-18 04:54:01] ffmpeg.deck_frigate.detect     ERROR   : rtsp://127.0.0.1:8554/deck_frigate: Invalid data found when processing input
2023-05-17 16:54:03.924284484  [2023-05-18 04:54:03] frigate.video                  ERROR   : deck_frigate: Unable to read frames from ffmpeg process.
2023-05-17 16:54:03.924425999  [2023-05-18 04:54:03] frigate.video                  ERROR   : deck_frigate: ffmpeg process is not running. exiting capture thread...
2023-05-17 16:54:11.301583687  [2023-05-18 04:54:11] watchdog.deck_frigate          ERROR   : Ffmpeg process crashed unexpectedly for deck_frigate.
2023-05-17 16:54:11.301890354  [2023-05-18 04:54:11] watchdog.deck_frigate          ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-05-17 16:54:11.301931837  [2023-05-18 04:54:11] ffmpeg.deck_frigate.detect     ERROR   : [rtsp @ 0x55b373a1af40] method DESCRIBE failed: 404 Not Found
2023-05-17 16:54:11.302152609  [2023-05-18 04:54:11] ffmpeg.deck_frigate.detect     ERROR   : rtsp://127.0.0.1:8554/deck_frigate: Server returned 404 Not Found
2023-05-17 23:32:33.179873607  [2023-05-18 11:32:33] frigate.watchdog               INFO    : Detection appears to be stuck. Restarting detection process...
2023-05-17 23:32:33.179877012  [2023-05-18 11:32:33] root                           INFO    : Waiting for detection process to exit gracefully...
2023-05-17 23:33:03.190852961  [2023-05-18 11:33:03] root                           INFO    : Detection process didnt exit. Force killing...
2023-05-17 23:33:03.196956144  [2023-05-18 11:33:03] root                           INFO    : Detection process has exited...
2023-05-17 23:33:03.209455307  [2023-05-18 11:33:03] detector.coral                 INFO    : Starting detection process: 97261
2023-05-17 23:33:05.824043479  [2023-05-18 11:33:03] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as usb
2023-05-17 23:33:05.828417079  [2023-05-18 11:33:05] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found

FFMPEG Probe for the named camera

[{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"6/1","codec_long_name":"H.264/AVC/MPEG-4AVC/MPEG-4part10","display_aspect_ratio":"168:95","height":720,"width":1280}]}}]

@NickM-27
Copy link
Collaborator

Connection timed out, looks like go2rtc crashed probably. You could try updating to a newer version of go2rtc

@xbmcnut
Copy link
Author

xbmcnut commented May 18, 2023

Connection timed out, looks like go2rtc crashed probably. You could try updating to a newer version of go2rtc

Thanks, but not sure how to do that under HassOS/Frigate add-on.

@NickM-27
Copy link
Collaborator

https://docs.frigate.video/configuration/advanced#custom-go2rtc-version

@xbmcnut
Copy link
Author

xbmcnut commented May 18, 2023

Thank you. Is this the file I need? https://github.com/AlexxIT/go2rtc/releases/download/v1.5.0/go2rtc_linux_amd64

How do I set the permissions in step 3 under HassOS?

@NickM-27
Copy link
Collaborator

Not 100% sure but most likely ssh or terminal addon could be used, just need to run chmod 777 go2rtc

@xbmcnut
Copy link
Author

xbmcnut commented May 18, 2023

I bit above my pay grade. 😟 I'll jump on Discord and see if anyone has done it already.

@xbmcnut
Copy link
Author

xbmcnut commented May 18, 2023

Connection timed out, looks like go2rtc crashed probably. You could try updating to a newer version of go2rtc

Would it work to install go2rtc as on add-on or would Frigate not know about/integrate with that version?
https://github.com/AlexxIT/go2rtc#go2rtc-home-assistant-add-on

@xbmcnut
Copy link
Author

xbmcnut commented May 23, 2023

FYI, I disabled hwaccel 5 days ago now and not a single crash yet so that's promising.

@NickM-27
Copy link
Collaborator

Would it work to install go2rtc as on add-on or would Frigate not know about/integrate with that version?

Frigate would not be able to use that for its features

@xbmcnut
Copy link
Author

xbmcnut commented May 23, 2023

Thanks for letting me know. I didn't thing it would. So do we have to wait for the HA team to update the video driver inside HA OS?

@NickM-27
Copy link
Collaborator

You could always try adjusting BIOS settings to see if something improves the situation, but if you are on HA OS then yes that is currently the case

@xbmcnut
Copy link
Author

xbmcnut commented May 23, 2023

I know it's not your realm, but have you got any idea on how the heck I would report that as a HA bug report so it gets some serious consideration? I'll try looking into the BIOS in the meantime but as it crashed on two different hardware platforms, I'm not liking my chances.

@NickM-27
Copy link
Collaborator

I don't know and I'm not sure if they would accept it as a bug unless there are more occurrences from other users running HA OS and some application that uses the GPU showing that it is indeed the driver that is the issue

@xbmcnut
Copy link
Author

xbmcnut commented May 23, 2023

Yeah, thought so. I don't know how many peeps would be 'wasting' a core i7 or i5 bare metal box on just HA like I am. Most would be using Proxmox I'd assume. I tried building a standalone Debian 11 based Frigate server on my old NUC but so far I haven't been able to get Frigate to start but that's a whole other issue 😒

@xbmcnut
Copy link
Author

xbmcnut commented May 31, 2023

FYI, as Home Assistant has just dropped 10.2 with the 6.1.29 kernel, I've just re-enabled hwaccel to see what happens. Like others, the impact on my gen 12 core i5 with hwaccel off is negligible anyway so if I have to leave it off, no big deal.

@xbmcnut
Copy link
Author

xbmcnut commented Jun 2, 2023 •
edited
Loading

Verdict is in. hwaccel is definitely the problem. Only lasted 3 days then crashed this morning. Reboot the hardware and it crashed again within the hour! As there is minimal impact with hwaccel disabled, I'll just leave it off.

image

@github-actions
Copy link

github-actions bot commented Jul 4, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale label Jul 4, 2023
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Jul 7, 2023
@xbmcnut
Copy link
Author

xbmcnut commented Jul 7, 2023 •
edited
Loading

FYI, I've managed to update Go2rtc inside Home assistant and I'm now monitoring again. This video made the process super easy.

https://youtu.be/upXyBVMR4RM?t=233

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
stale support triage
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@xbmcnut @NickM-27