Feat/video

[abrichr]

This PR

• implements video recording with ffmpeg via pyav.
• fixes a performance bug affecting the usage of mss
• fixes some other bugs (e.g. types, remove_invalid_keyboard_events, get_timestamp)

config.RECORD_IMAGES == True and config.RECORD_VIDEO == False before mss bugfix:

config.RECORD_IMAGES == True and config.RECORD_VIDEO == False after mss bugfix:

config.RECORD_IMAGES == False and config.RECORD_VIDEO == True after mss bugfix:

[0dm]

What are the advantages of using FFmpeg to record instead of openadapt.capture (screen-recorder-sdk on windows, AVFoundation on Mac)? I'd guess that AVFoundation is probably more light-weight.

[abrichr]

@0dm good question! The idea here is ultimately to replace the Screenshot model by extracting frames from the recorded video. My initial thought was that this would be easier to do with ffmpeg, but I'm not sure.

Do you have any thoughts on how to extract high precision frames from the video recorded by openadapt.capture? Here's what I get when I run ffprobe on the output of capture:

(openadapt-py3.10) abrichr@MacBook-Pro-4 OpenAdapt % ffprobe captures/2024-02-19-10-43-33.mov  
ffprobe version 6.1.1 Copyright (c) 2007-2023 the FFmpeg developers
  built with Apple clang version 15.0.0 (clang-1500.1.0.2.5)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/6.1.1_3 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags='-Wl,-ld_classic' --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libharfbuzz --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopenvino --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd88a704b40] moov atom not found
captures/2024-02-19-10-43-33.mov: Invalid data found when processing input