BirdNET-Pi behind a reverse proxy

[Svardsten53]

Is it possible to disable or bypass the .local function? I guess this is the cause of the problem I have. I have several other web servers in my network and therefore use reverse proxy in my Synology NAS to direct calls from the Internet correctly, because in the router you can only control calls on one port to a specific server. This function requires that you enter an IP address and port number on the local network, a regular URL is not accepted. I have tried to enter both IP address and regular URL in the BirdNET-Pi configuration, but nothing works. So what I want is for the web server to respond to the local IP address instead of birdnetpi.local. I hope there is a solution to this.

[mcguirepr89]

So what I want is for the web server to respond to the local IP address instead of birdnetpi.local. I hope there is a solution to this.

Hello, @Svardsten53 and thank you so much for you interest in BirdNET-Pi and for your great question. I actually was just helping another user with a similar scenario to allow for several BirdNET-Pis on the same local network. Please take a look through that conversation if you'd like to see how that worked out, though that is just for reference and not me pointing you elsewhere :)

Is it possible to disable or bypass the .local function? I guess this is the cause of the problem

BirdNET-Pi's announce themselves on the network via an avahi-alias for "birdnetpi.local" that allow mDNS allowed networks to find it easily. It can easily be disabled by issuing sudo systemctl disable --now avahi-alias@birdnetpi.local.service. With that said, however, that's not really going to be causing any issues and can safely be left alone.

The "problem" is actually a great security feature of Caddy's wherein caddy only listens and responds to its site-blocks and ignores requests to the underlying host's open ports when the requests don't match the site block. This means we just need to tell Caddy that is should listen for requests to the host's local IP. The birdnet.conf file is the place to set this, and you were SO close to having this resolved yourself.

I have tried to enter both IP address and regular URL in the BirdNET-Pi configuration, but nothing works.

It seems you came quite close to the solution and had precisely the right idea on how to achieve what you need, with one crucial step missing: updating BirdNET-Pi after altering the birdnet.conf file. After you edit the custom URL section of birdnet.conf to resemble my example below:

BIRDNETPI_URL=http://192.168.1.41
EXTRACTIONLOG_URL=
BIRDNETLOG_URL=

then issue update_birdnet.sh in the command line to reconfigure BirdNET-Pi to respond to the local IP address instead of birdnetpi.local.

Be sure to clear your web browser's cache so that it does not load old, broken links.

Please let me know how this works for you and if you have further questions or would like any further assistance as I am more than happy to help!

My best regards,
Patrick

[davidmam]

Thanks for this. I have to run with the IP as my ethernet extender does not wish to behave. Unfortunately the recordings links seem to be hardwired to birdnetpi.local. Is there a way in the config to get them to point to an IP rather than the hostname? When setting the URLS to the IP and running update_birdnet.sh I get lots of:
sudo: unable to resolve host birdnetpi: Name or service not known

I've checked and this a 'feature' of the powerline extender in use. It enables it's own DHCP server and breaks the Bonjour service on devices connected to it.

[mcguirepr89]

Hello, Dr. Martin, and thank you very much for your interest in BirdNET-Pi. Thank you also for utilizing these discussions and the wiki to get up to speed on things -- it is very helpful and very much appreciated. I'm 100% certain we can get this working for you, though I will try to guide you through the method that BirdNET-Pi expects. From there, if your hardware continues to fuss, we can definitely figure out a more tailored solution to accommodate your setup.

The best documentation on configuring the links to use IP addresses can be found here in the Custom URLs section. Please give that a read over before continuing below.

The basic idea will have you put the desired IP address into the BIRDNETPI_URL variable in birdnet.conf and keep the other custom URLs empty. Running update_birdnet.sh at that point should reconfigure all of the birdnetpi.local links to the IP address you use.

When setting the URLS to the IP and running update_birdnet.sh I get lots of:
sudo: unable to resolve host birdnetpi: Name or service not known

I imagine this is due to setting all of the URLs to the IP address, which will improperly reconfigure the Caddyfile and throw many errors. My notes above to Mr. Svardsten highlight how Caddy will only respond to exactly what is configured within each of its site-blocks. For this reason, the Caddyfile will need to be expecting requests to the IP address only. (That's just for clarification -- no extra steps are needed to configure the Caddyfile as the birdnet.conf and update_birdnet.sh will handle that.)

I've checked and this [is] a 'feature' of the powerline extender in use. It enables it's own DHCP server and breaks the Bonjour service on devices connected to it.

That is very helpful information, but I'm hopeful that getting things properly configured will mitigate any of those peculiarities.

When you are able, please adjust your birdnet.conf file's "Custom URLs" section to resemble what I have below:
Example IP address=192.168.1.68

BIRDNETPI_URL=http://192.168.1.68
EXTRACTIONLOG_URL=
BIRDNETLOG_URL=

When you've made that adjustment and have saved your changes, issue update_birdnet.sh again.

Please keep the terminal opened and the output from update_birdnet.sh available so that you can post its complete output here for further troubleshooting in case the steps above fail for any reason.

My best regards,
Patrick

PS -- I will not be able to respond much today, but will certainly respond later this evening and will be available and eager to help tomorrow through Sunday.

[davidmam]

Thanks for such a speedy response. It is now up and happily listening for the birds. So far not many as afternoons are quiet in winter, but should be plenty around dawn tomorrow.

It is a great project. I hope to encourage some student projects based on this (and the Audiomoth for bats)

[mcguirepr89]

I'm very glad to hear that you're up and running. Am I correct in assuming that your BirdNET-Pi will be listening in Scotland? Also, if you would be so kind as to star the project (once it has earned your seal of approval), I would be very grateful for the endorsement.

[Svardsten53]

Patrick, thanks for your quick reply. I'm much closer the solution now I hope.
I have managed to reach the BirdNET-Pi web server on the local network by typing the IP address 192.168.0.106 in the browser.

I have still problem when typing the URL which is defined in the reversed proxy to port 80 on the Pi, the only thing I get is a white page. The strange thing is if i point the proxy to port 8000 I can see the Icecast2 status page. I have not modified anything in the .local settings. Update is done after every change.

Settings in config file:

BIRDNETPI_URL=http://192.168.0.106
EXTRACTIONLOG_URL=
BIRDNETLOG_URL=

The caddy file looks like this:

http://localhost http://birdnetpi.local http://192.168.0.106 {
  root * /home/pi/BirdSongs/Extracted
  file_server browse
  basicauth /Processed* {
    birdnet xxx
  }
  basicauth /scripts* {
    birdnet xxx
  }
  basicauth /stream {
    birdnet xxx
  }
  basicauth /phpsysinfo* {
    birdnet xxx
  }
  reverse_proxy /stream localhost:8000
  php_fastcgi unix//run/php/php7.3-fpm.sock
}

The caddy log from the last update:

dec 05 19:52:55 birdnetpi systemd[1]: Reloading Caddy.
dec 05 19:52:55 birdnetpi caddy[6717]: {"level":"info","ts":1638730375.1315625,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":""}
dec 05 19:52:55 birdnetpi caddy[6717]: {"level":"warn","ts":1638730375.1362822,"msg":"input is not formatted with 'caddy fmt'","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
dec 05 19:52:55 birdnetpi caddy[630]: {"level":"info","ts":1638730375.1513507,"logger":"admin.api","msg":"received request","method":"POST","host":"localhost:2019","uri":"/load","remote_addr":"127.0.0.1:41944","headers":{"Accept-Encoding":["gzip"],"Content-Length":["2136"],"Content-Type":["application/json"],"Origin":["localhost:2019"],"User-Agent":["Go-http-client/1.1"]}}
dec 05 19:52:55 birdnetpi caddy[630]: {"level":"info","ts":1638730375.1541495,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["localhost:2019","[::1]:2019","127.0.0.1:2019"]}
dec 05 19:52:55 birdnetpi caddy[630]: {"level":"info","ts":1638730375.154579,"logger":"http","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv0","http_port":80}
dec 05 19:52:55 birdnetpi caddy[630]: {"level":"info","ts":1638730375.15463,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x4000224b60"}
dec 05 19:53:05 birdnetpi caddy[630]: {"level":"info","ts":1638730385.152588,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0x400019e620"}
dec 05 19:53:05 birdnetpi caddy[630]: {"level":"info","ts":1638730385.1531591,"msg":"autosaved config (load with --resume flag)","file":"/var/lib/caddy/.config/caddy/autosave.json"}
dec 05 19:53:05 birdnetpi caddy[630]: {"level":"error","ts":1638730385.1545076,"logger":"admin","msg":"stopping current admin endpoint","error":"shutting down admin server: context deadline exceeded"}
dec 05 19:53:05 birdnetpi caddy[630]: {"level":"info","ts":1638730385.1545749,"logger":"admin.api","msg":"load complete"}
dec 05 19:53:05 birdnetpi systemd[1]: Reloaded Caddy.

[mcguirepr89]

Hello again!

I have still problem when typing the URL which is defined in the reversed proxy to port 80 on the Pi, the only thing I get is a white page.

This bit seems to point to the remaining issue, which is again tied to the aforementioned Caddy listening rules wherein it will only respond to requests that match its site-block definitions. This means that you'll need to add whatever URL is defined in your reverse proxy to the Caddyfile's main site-block. Also, you will want to change your CADDY_PWD since it was posted here (though it was hashed, it was not encrypted -- I did however, remove the original comment with the password).

1. You can change the password easily by changing it in birdnet.conf and running another update, either through the command update_birdnet.sh or through the web-interface's "System Links" --> "Tools" --> "Update BirdNET-Pi".

2. After the update, you will want to manually edit /etc/caddy/Caddyfile to look something like this::

   http://localhost http://birdnetpi.local http://192.168.0.106 http://WHATEVERISDEFINEDINYOURREVERSE.proxy {
   root * /home/pi/BirdSongs/Extracted
   file_server browse
   basicauth /Processed* {
    birdnet xxx
   }
   basicauth /scripts* {
     birdnet xxx
   }
   basicauth /stream {
     birdnet xxx
   }
   basicauth /phpsysinfo* {
    birdnet xxx
   }
   reverse_proxy /stream localhost:8000
   php_fastcgi unix//run/php/php7.3-fpm.sock
   }
   

3. Then you would reload caddy and try again. The command below will reload caddy and then will "follow" the caddy log :

   sudo systemctl reload caddy && journalctl -fu caddy
   

   (Ctrl+c to quit)

I suspect that will work, but without knowing more specifics (that you should not post here) about how you're trying to route the traffic, and more specifically, what the request headers contain, I can only say that is my best guess as to what will work. Unfortunately, the caddy logs you've provided only show a successful caddy reload, which is a good thing and indicates that there are no issues on Caddy's end, but doesn't contain any requests/responses.

If you'd like more help, please consider emailing me privately so as not to share sensitive networking information here.

My fingers are crossed for you and I hope it works 👍

[Svardsten53]

A big thank you to Patrick for the quick response and clear and informative instructions and of course for a very nice program.

Now we have made it work! In the end, it was pretty simple.
The first line of the Caddyfile should look like this with the IP address of the Pi computer:

http://localhost http://birdnetpi.local http://192.168.0.106 http://192.168.0.106.proxy {

In birdnet.conf:

BIRDNETPI_URL=http://birdnet.yourdomain.com
EXTRACTIONLOGURL=http://birdnet.yourdomain.com
BIRDNETLOG_URL=http://birdnet.yourdomain.com

In my reverse proxy I forwarded incoming HTTPS traffic on port 443 for the current subdomain to HTTP 192.168.0.106 on port 80.
I also needed to adjust some links in the menu file. Those who have ports 8080 and 8888 can only be seen on the local network, otherwise you have to open these ports in the router, which feels unnecessary and insecure.

Case closed and I hope that others can also benefit from these instructions in the future.

Best regards from a retired mainframe guy who has now entered the Linux world.

[mcguirepr89]

I'm very glad to help.

You may be interested in this Wiki article I put together that lays out how to share your installation, creating free hostnames and free domain registrations for those hostnames, which can be used to address the log links and can add TLS through SSL certificates automatically handled and obtained by Caddy.

May I ask in what part of the world your BirdNET-Pi will be listening??

[Svardsten53]

I read that article a few days ago and it really is a good help to get started. I have had some domains for several years now and enjoy creating things on the Internet.

I live 20 km outside Stockholm in Sweden. Right now it's not so much birds because it's winter. My plan is to mix two microphones, a omnidirectional and a shotgun microphone to cover both the ground and the sky to also detect migrating birds.

[mcguirepr89]

I read that article a few days ago and it really is a good help to get started. I have had some domains for several years now and enjoy creating things on the Internet.

Knowing that you're familiar with registering domain records and that you own a domain and are using subdomains to reach your installation, I would love to use this as an opportunity to better explain why the "Custom URL" section of birdnet.conf contains three variables for you and others. This will allow Caddy to handle everything over HTTPS for you, including obtaining and updating SSL certificates (from LetsEncrypt). Otherwise, as you know, your Basic Auth username and password are being sent in plain-text over HTTP.

It will also address:

I also needed to adjust some links in the menu file. Those who have ports 8080 and 8888 can only be seen on the local network, otherwise you have to open these ports in the router, which feels unnecessary and insecure.

Using the EXTRACTIONLOG_URL and BIRDNETLOG_URL variables will add two additional site-blocks to /etc/caddy/Caddyfile that will reverse-proxy requests to those URLs back to the gotty web servers, allowing you to add SSL certificates for those small web-servers as well. It will also trigger parts of the installation/update scripts to fix those links in homepage/menu.html for you. This is explained in more detail below.

*Note: It is assumed (and required for things to work) that you run an update after changing "Custom URL" settings. Do this either with update_birdnet.sh at the command line, or using "System Links"--->"Tools"--->"Update BirdNET-Pi" on the web-interface.

With BirdNET-Pi's default settings

The default birdnet.conf:

...
BIRDNETPI_URL=
EXTRACTIONLOG_URL=
BIRDNETLOG_URL=
...

The /etc/caddy/Caddyfile that is created from the birdnet.conf above:

http://localhost http://birdnetpi.local  {
  root * /home/pi/BirdSongs/Extracted
  file_server browse
  basicauth /Processed* {
    birdnet HASHEDPASSWORD
  }
  basicauth /scripts* {
    birdnet HASHEDPASSWORD
  }
  basicauth /stream {
    birdnet HASHEDPASSWORD
  }
  basicauth /phpsysinfo* {
    birdnet HASHEDPASSWORD
  }
  reverse_proxy /stream localhost:8000
  php_fastcgi unix//run/php/php7.3-fpm.sock
}

With these defaults, the links that are found in homepage/menu.html for the "BirdNET-Lite Log" and "Extraction Log" are not altered, and therefore will simply be "http://birdnetpi.local:8080" and "http://birdnetpi.local:8888," respectively.

Ports 8080 and 8888 are independent gotty webservers, small, TTY terminal-emulators built in Go.

So in a default installation, everthing is local only, so I just refer visitors directly to the gotty servers.

Things are a bit different when you utilize the "Custom URLs" in birdnet.conf . . . .

Customizing the BIRDNETPI_URL only

birdnet.conf with a custom BIRDNETPI_URL so that BirdNET-Pi can be reached from its LAN IP address, "http://192.168.1.68"

BIRDNETPI_URL=http://192.168.1.68
EXTRACTIONLOG_URL=
BIRDNETLOG_URL=

The /etc/caddy/Caddyfile that is created from the above birdnet.conf:

http://localhost http://birdnetpi.local http://192.168.1.68  {
  root * /home/pi/BirdSongs/Extracted
  file_server browse
  basicauth /Processed* {
    birdnet HASHEDPASSWORD
  }
  basicauth /scripts* {
    birdnet HASHEDPASSWORD
  }
  basicauth /stream {
    birdnet HASHEDPASSWORD
  }
  basicauth /phpsysinfo* {
    birdnet HASHEDPASSWORD
  }
  reverse_proxy /stream localhost:8000
  php_fastcgi unix//run/php/php7.3-fpm.sock
}

With BIRDNETPI_URL=http://192.168.1.68, the links that are found in homepage/menu.html for the "BirdNET-Lite Log" and "Extraction Log" are altered, changed to be "http://192.168.1.68:8080" and "http://192.168.1.68:8888," respectively.

Since this installation is still local only, I still refer visitors directly to the gotty servers, which, unlike caddy, will respond to any request to it's open ports.

Customizing all of the URLs, i.e., a publicly accessible installation

Domain Records:
Type:A---Domain:https://birdnetpi.pmcgui.xyz--->Address:BirdNET-Pi's Public IP Address
Type:CNAME---Domain:https://birdnetlog.pmcgui.xyz--->Cannonical Name:birdnetpi.pmcgui.xyz
Type:CNAME---Domain:https://extractionlog.pmcgui.xyz--->Cannonical Name:birdnetpi.pmcgui.xyz

birdnet.conf with a custom BIRDNETPI_URL so that BirdNET-Pi can be reached from a domain name, "https://birdnetpi.pmcgui.xyz," and custom EXTRACTIONLOG_URL and BIRDNETLOG_URLs so that the terminal web-servers can be reached at port 443 instead of ports 8080 and 8888:

BIRDNETPI_URL=https://birdnetpi.pmcgui.xyz
EXTRACTIONLOGURL=https://extractionlog.pmcgui.xyz
BIRDNETLOG_URL=https://birdnetlog.pmcgui.xyz

The /etc/caddy/Caddyfile that is created from the above birdnet.conf:

http://localhost http://birdnetpi.local https://birdnetpi.pmcgui.xyz {
  root * /home/pi/BirdSongs/Extracted
  file_server browse
  basicauth /Processed* {
    birdnet HASHEDPASSWORD
  }
  basicauth /scripts* {
    birdnet HASHEDPASSWORD
  }
  basicauth /stream {
    birdnet HASHEDPASSWORD
  }
  basicauth /phpsysinfo* {
    birdnet HASHEDPASSWORD
  }
  reverse_proxy /stream localhost:8000
  php_fastcgi unix//run/php/php7.3-fpm.sock
}

https://extractionlog.pmcgui.xyz {
  reverse_proxy localhost:8888
}

https://birdnetlog.pmcgui.xyz {
  reverse_proxy localhost:8080
}

With BIRDNETPI_URL=https://birdnetpi.pmcgui.xyz, the links that are found in homepage/menu.html for the "BirdNET-Lite Log" and "Extraction Log" are altered, this time changed to "https://birdnetlog.pmcgui.xyz" and "https://extractionlog.pmcgui.xyz," respectively.

Since this installation is public, you'll notice that two new site-blocks have been added which will handle reverse-proxying requests to the gotty webservers using the encrypted and already-opened port 443.

Conclusion

I hope that is a better overview of the variables, why there are three URLs, and how simply adding the 's' to the site-block will automatically cause caddy to reach out to Certificate Authorities, run the appropriate challenges, and will then obtain a (free) SSL certificate to encrypt all of your BirdNET-Pi traffic. caddy will also update those certificates when they expire, and will force requests to (in the case of my example) http://birdnetpi.pmcgui.xyz to be redirected to port 443 (https://birdnetpi.pmcgui.xyz) and will encrypt all of its traffic. This means all of your various "Tools," (e.g., Clear ALL Data, Shutdown/Reboot, Update BirdNET-Pi), "System Info," and "Live Audio Stream" will be for your eyes (and ears) only.

For thoroughness, would you mind please posting here your complete /etc/caddy/Caddyfile (with HASHEDPASSWORD obscured, of course)? I am very curious to see it based on your last post.

I live 20 km outside Stockholm in Sweden. Right now it's not so much birds because it's winter. My plan is to mix two microphones, a omnidirectional and a shotgun microphone to cover both the ground and the sky to also detect migrating birds.

This is so awesome!!!! If you find any migratory bird / night-migration prediction models, I'd be happy to do my best at integrating another model for other uses. I know they exist, but I don't know how they would compare to just using BirdNET-Lite for that as well. Please keep me posted on your progress and please also consider joining in on some of our Discussions as I know some of the others would be interested in that idea.

My best regards

[Svardsten53]

Thank you Patrick for the detailed description. Caddy seems to know a lot and I will gladly study it in the future. Right now I just want to make it work and use the solution I already have.

When I rebooted my Pi I was back to the white page in my browser, but since I now know from your description that Caddy uses parameters in the config file, it was possible to fix, as long as you did things in the right order.

This is what the URL configuration looks like now and the result in the Caddy file.

BIRDNETPI_URL=http://birdnet.mydomain.se
EXTRACTIONLOG_URL=
BIRDNETLOG_URL=


http://localhost http://birdnetpi.local http://birdnet.mydomain.se {
  root * /home/pi/BirdSongs/Extracted
  file_server browse
  basicauth /Processed* {
    birdnet xxx
  }
  basicauth /scripts* {
    birdnet xxx
  }
  basicauth /stream {
    birdnet xxx
  }
  basicauth /phpsysinfo* {
    birdnet xxx
  }
  reverse_proxy /stream localhost:8000
  php_fastcgi unix//run/php/php7.3-fpm.sock
}

The microphone is now connected, the sound card configured and I get spectrograms. However, I have an error message, but open a separate ticket for it to keep things separate, unless I find the solution myself.

Thanks a lot for all your help!

[mcguirepr89]

Glad to help!

The microphone is now connected, the sound card configured and I get spectrograms. However, I have an error message, but open a separate ticket for it to keep things separate, unless I find the solution myself.

I'm sticking around and will be ready to address this as soon as I see the new issue.

My best!

[davidmam]

yes, Dundee. I do get some bizarre annotations, eg a great bittern which is very rare and not found in Scotland at all. Looking forward to what tomorrow’s dawn chorus brings. I’m hoping to persuade the University to put some in the botanic gardens. ..d Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: Patrick McGuire ***@***.***> Sent: Thursday, December 30, 2021 6:37:42 PM To: mcguirepr89/BirdNET-Pi ***@***.***> Cc: David Martin (Staff) ***@***.***>; Comment ***@***.***> Subject: Re: [mcguirepr89/BirdNET-Pi] BirdNET-Pi behind a reverse proxy (Discussion #92) CAUTION: This email originated from outside the University of Dundee. Do not click links or open attachments unless you recognise the sender's email address and know the content is safe. I'm very glad to hear that you're up and running. Am I safe in assuming that your BirdNET-Pi will be listening in Scotland? — Reply to this email directly, view it on GitHub<#92 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAOH7JQ7VAG6B2DVJM6MQCTUTSRHNANCNFSM5JOZOWGQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***> The University of Dundee is a registered Scottish Charity, No: SC015096

[mcguirepr89]

yes, Dundee.

Very cool! I'm glad to have Scotland represented!

I do get some bizarre annotations, eg a great bittern which is very rare and not found in Scotland at all. Looking forward to what tomorrow’s dawn chorus brings.

You will definitely encounters some false positives, but I would encourage you to take a look at this comment and the discussion around it for some erudite methods of tuning the BirdNET-Pi.

Within birdnet.conf, you will find a few parameters that you can adjust to mitigate false positives. These settings will alter your detections significantly. The variables that you may be interested in are CONFIDENCE and SENSITIVITY.

To quote a wonderful avian acoustic expert, Frank @DD4WH:

". . . if you would like to lower the number of false positives, it would be much better to reduce SENSITIVITY (and thus increase specificity) of the classifier model by lowering the sensitivity parameter. If there are still too many false positives after that, you can then increase the minimum CONFIDENCE score.

Making those adjustments and utilizing the data visualization tools on the BirdNET-Pi webpage will certainly help with discerning false positives, but will also allow for detecting very rare occurrences of species that are deemed atypical.

Another tools you may utilize is a custom_species_list.txt. Using the labels.txt file found in /home/pi/BirdNET-Pi/model, you can curate your own list of species for monitoring. Simply selecting the species from labels.txt and pasting them into another file called custom_species_list.txt that you keep in the /home/pi/BirdNET-Pi directory will only report detection results from species found therein. If I'm not mistaken, Frank @DD4WH has put together a list of European species he may be able to share with you -- (sorry, Frank, for not keeping up with that!!!)

I'm hopeful that the info above will allow you to adjust your system enough to meet your needs.

One last important note on mitigating false positives: BirdNET-Pi is fully integrated with BirdWeather (you can even see how the integration took place here), which uses e-Bird grid data during post-processing to drastically lower confidence scores for rare and unlikely species, while increasing the confidence score for common and likely species.

To see how the same data is reported differently on the BirdNET-Pi and within BirdWeather, take a look at @CaiusX's installation here, and then locate his installation in Johannesburg, South Africa on the BirdWeather Map.

With all of that said, I hope you will be willing to reach out to Tim ( hisname@birdweatherDOTcom ) <--- trying to obfuscate the actual email address from internet crawlers, but replace the DOT with a . and hisname with Tim. I hope you will reach out to him with your Latitude, Longitude, and a general location for the station label so that he can provide you with a unique BIRDWEATHER_ID. When you receive your BirdWeather ID, simply input it as the BIRDWEATHER_ID within birdnet.conf (no quotes or anything, just paste it next to the = sign) and your subsequent detections will be shared with the rest of the world!

And finally,

I’m hoping to persuade the University to put some in the botanic gardens.

This is very exciting news. PLEASE allow me to help in any way that I can. BirdNET-Pis install NoMachine remote desktop software by default, so if any scenario comes up for which you would like another set of eyes, or need to recover valuable data, please don't hesitate to let me know as I am eager to help and equally want to persuade the University to put some in the botanical gardens!

My very best regards,
Patrick

[davidmam]

Dear Patrick, Thank you so much for the detailed info. I'll definitely tweak the settings and see what I can do with them. I'll take a look through the species labels and build a regional list. There are great regional variations within UK, let alone Europe so I will need to do some curation there. I have some experience with using ML approaches in bioinformatics, though little in actually implementing tensorflow models. Our University is big on impact and we are partnering with the Eden Project in bringing their first site outside Cornwall to Dundee. As part of that there is lots of opportunity for engagement and student learning, and this fits in very nicely. Best wishes ..d Dr David Martin Senior Lecturer in Bioinformatics College of Life Sciences University of Dundee

…

________________________________ From: Patrick McGuire ***@***.***> Sent: 31 December 2021 14:47 To: mcguirepr89/BirdNET-Pi ***@***.***> Cc: David Martin (Staff) ***@***.***>; Comment ***@***.***> Subject: Re: [mcguirepr89/BirdNET-Pi] BirdNET-Pi behind a reverse proxy (Discussion #92) CAUTION: This email originated from outside the University of Dundee. Do not click links or open attachments unless you recognise the sender's email address and know the content is safe. yes, Dundee. Very cool! I'm glad to have Scotland represented! I do get some bizarre annotations, eg a great bittern which is very rare and not found in Scotland at all. Looking forward to what tomorrow’s dawn chorus brings. You will definitely encounters some false positives, but I would encourage you to take a look at this comment<#11 (comment)> and the discussion around it for some erudite methods of tuning the BirdNET-Pi. Within birdnet.conf, you will find a few parameters that you can adjust to mitigate false positives. These settings will alter your detections significantly. The variables that you may be interested in are CONFIDENCE and SENSITIVITY. To quote a wonderful avian acoustic expert, Frank @DD4WH<https://github.com/DD4WH>: ". . . if you would like to lower the number of false positives, it would be much better to reduce SENSITIVITY (and thus increase specificity) of the classifier model by lowering the sensitivity parameter. If there are still too many false positives after that, you can then increase the minimum CONFIDENCE score. Making those adjustments and utilizing the data visualization tools on the BirdNET-Pi webpage will certainly help with discerning false positives, but will also allow for detecting very rare occurrences of species that are deemed atypical. Another tools you may utilize is a custom_species_list.txt. Using the labels.txt file found in /home/pi/BirdNET-Pi/model, you can curate your own list of species for monitoring. Simply selecting the species from labels.txt and pasting them into another file called custom_species_list.txt that you keep in the /home/pi/BirdNET-Pi directory will only report detection results from species found therein. If I'm not mistaken, Frank @DD4WH<https://github.com/DD4WH> has put together a list of European species he may be able to share with you -- (sorry, Frank, for not keeping up with that!!!) I'm hopeful that the info above will allow you to adjust your system enough to meet your needs. One last important note on mitigating false positives: BirdNET-Pi is fully integrated with BirdWeather<https://app.birdweather.com> (you can even see how the integration took place here<#82>), which uses e-Bird grid data during post-processing to drastically lower confidence scores for rare and unlikely species, while increasing the confidence score for common and likely species. To see how the same data is reported differently on the BirdNET-Pi and within BirdWeather, take a look at @CaiusX<https://github.com/CaiusX>'s installation here<https://birds.naturestation.net>, and then locate his installation in Johannesburg, South Africa on the BirdWeather Map<https://app.birdweather.com>. With all of that said, I hope you will be willing to reach out to Tim ( ***@***.*** ) <--- trying to obfuscate the actual email address from internet crawlers, but replace the DOT with a . and hisname with Tim. I hope you will reach out to him with your Latitude, Longitude, and a general location for the station label so that he can provide you with a unique BIRDWEATHER_ID. When you receive your BirdWeather ID, simply input it as the BIRDWEATHER_ID within birdnet.conf (no quotes or anything, just paste it next to the = sign) and your subsequent detections will be shared with the rest of the world! And finally, I’m hoping to persuade the University to put some in the botanic gardens. This is very exciting news. PLEASE allow me to help in any way that I can. BirdNET-Pis install NoMachine remote desktop software by default, so if any scenario comes up for which you would like another set of eyes, or need to recover valuable data, please don't hesitate to let me know as I am eager to help and equally want to persuade the University to put some in the botanical gardens! My very best regards, Patrick — Reply to this email directly, view it on GitHub<#92 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAOH7JUB22RBIZEC4B2GHU3UTW67XANCNFSM5JOZOWGQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***> The University of Dundee is a registered Scottish Charity, No: SC015096

[mcguirepr89]

This is very exciting and wonderful to know. I do hope you will pass along my willingness to help to any students interested in using BirdNET-Pi for their projects. I have helped a few students at the Technical University in Denmark and would be honored to continue making contributions to anyone else eager to learn. As a selfish reminder, please do Star the project should you decide to use it, as I would be very, very grateful and it is my best indication of how many github users are using a BirdNET-Pi.

My best regards as always!
Patrick

[davidmam]

Your star count has gone up by 1! I have started the wheels in motion on this. I might be able to get an honours student next September to work on it as well - I'll have to push the Machine Learning in biology side and treat it as a particular type of thesis project. ..d Dr David Martin Senior Lecturer in Bioinformatics College of Life Sciences University of Dundee

…

________________________________ From: Patrick McGuire ***@***.***> Sent: 31 December 2021 15:10 To: mcguirepr89/BirdNET-Pi ***@***.***> Cc: David Martin (Staff) ***@***.***>; Comment ***@***.***> Subject: Re: [mcguirepr89/BirdNET-Pi] BirdNET-Pi behind a reverse proxy (Discussion #92) CAUTION: This email originated from outside the University of Dundee. Do not click links or open attachments unless you recognise the sender's email address and know the content is safe. This is very exciting and wonderful to know. I do hope you will pass along my willingness to help to any students interested in using BirdNET-Pi for their projects. I have helped a few students at the Technical University in Denmark and would be honored to continue making contributions to anyone else eager to learn. As a selfish reminder, please do Star the project should you decide to use it, as I would be very, very grateful and its my best indication of how many github users are using a BirdNET-Pi. My best regards as always! Patrick — Reply to this email directly, view it on GitHub<#92 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAOH7JV3TKAKKLWLCWBUCLLUTXBWNANCNFSM5JOZOWGQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***> The University of Dundee is a registered Scottish Charity, No: SC015096

[mcguirepr89]

Thank you very much! My fingers are crossed that you are able to come up with just the right angle in presenting your ideas and visions for the University. I'm positive you've seen these, but my due diligence requires I point out the Related publications section for great papers on BirdNET's use-cases and lots of the other stuff academia requires to be persuaded towards an endeavor.

[mcguirepr89]

Greetings, @davidmam! I hope this finds you well.

I just wanted to check in with you to see how you are doing and see what your thoughts are after running your BirdNET-Pi for a little over a week. I have enjoyed checking in on your station via BirdWeather :)

My best regards and a belated Happy New Year!
Patrick

[davidmam]

It seems to be running just fine, though my powerline network has been a bit flaky. There are a small number of false positives – I have the settings at a threshold 0.75 and the sensitivity at 1.25, so mostly out of the box. Quite happy with it so far. I have a few others keen to set one up but there are supply issues with Raspberry Pi at the moment. ..d From: Patrick McGuire ***@***.***> Sent: 11 January 2022 13:04 To: mcguirepr89/BirdNET-Pi ***@***.***> Cc: David Martin (Staff) ***@***.***>; Mention ***@***.***> Subject: Re: [mcguirepr89/BirdNET-Pi] BirdNET-Pi behind a reverse proxy (Discussion #92) CAUTION: This email originated from outside the University of Dundee. Do not click links or open attachments unless you recognise the sender's email address and know the content is safe. Greetings, @davidmam<https://github.com/davidmam>! I hope this finds you well. I just wanted to check in with you to see how you are doing and see what your thoughts are after running your BirdNET-Pi for a little over a week. I have enjoyed checking in on your station via BirdWeather<https://app.birdweather.com> :) My best regards and a belated Happy New Year! Patrick — Reply to this email directly, view it on GitHub<#92 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAOH7JXBGCMTRVGLVOYHKZLUVQTEBANCNFSM5JOZOWGQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>> The University of Dundee is a registered Scottish Charity, No: SC015096

[mcguirepr89]

I'm very glad to hear that you are happy with (mostly) default settings. Learning what false positives your installation will trigger is part of the fun!! and will help you in tuning settings to your needs in Dundee.

I've heard from others that the RPi shortage is a real barrier for Europe right now. My fingers are crossed that the supply will start to reflect the demand and that you and others will have more ease getting your hands on these great tools sooner than later.

My kindest regards,
Patrick

[davidmam]

I was listening back to the Great Bittern calls. Sounds like a small heron with a 4 litre diesel engine 🙂 I have the microphones on the back of my garage, right next to a busy road and traffic lights so get a lot of that diesel vehicle rumbling. It would be helpful to have an exclude list rather than an include list. ..d Dr David Martin Senior Lecturer in Bioinformatics College of Life Sciences University of Dundee

…

________________________________ From: Patrick McGuire ***@***.***> Sent: 11 January 2022 14:14 To: mcguirepr89/BirdNET-Pi ***@***.***> Cc: David Martin (Staff) ***@***.***>; Mention ***@***.***> Subject: Re: [mcguirepr89/BirdNET-Pi] BirdNET-Pi behind a reverse proxy (Discussion #92) CAUTION: This email originated from outside the University of Dundee. Do not click links or open attachments unless you recognise the sender's email address and know the content is safe. I'm very glad to hear that you are happy with (mostly) default settings. Learning what false positives your installation will trigger is part of the fun!! and will help you in tuning settings to your needs in Dundee. I've heard from others that the RPi shortage is a real barrier for Europe right now. My fingers are crossed that the supply will start to reflect the demand and that you and others will have more ease getting your hands on these great tools sooner than later. My kindest regards, Patrick — Reply to this email directly, view it on GitHub<#92 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAOH7JRN5GO5Z22L4LP2DUDUVQ3KVANCNFSM5JOZOWGQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***> The University of Dundee is a registered Scottish Charity, No: SC015096

[mcguirepr89]

It would be helpful to have an exclude list rather than an include list.

It's interesting to me that you say this, as I had the same suggestion. @CaiusX was badass enough to make it a reality, but it simply hasn't made its way into the main project yet --

This is incentive to move this back up on my development list. 🤔