24/7 Raspberry Pi/birdNET monitoring (1 Viewer)

[LittleBitOfBreadNoCheese]

This video shows someone setting up a Raspberry Pi with the lite version of birdNET so you can have continuous monitoring.

I had no idea so many people were already doing this, sharing data via sites like BirdWeather:

BirdWeather - a living library of bird vocalizations.

Join a growing community of citizen scientists with PUC, our AI powered bioacoustics platform.

www.birdweather.com

Anyone tried or want to give it a go?

 

[LittleBitOfBreadNoCheese]

It sounds like BirdWeather will launch their own ready to go device Q1 2023, which links you straight into BirdWeather.

"
We’re excited to introduce PUC (Physical Universe Codec) - our AI powered bioacoustics platform. Packed with dual microphones, WiFi/BLE, GPS, environmental sensors, and a built-in neural engine, all in a weatherproof enclosure, PUC is ready to capture all that nature can throw at it!


(Shipping Q1 2023)
"

BirdWeather - a living library of bird vocalizations.

Join a growing community of citizen scientists with PUC, our AI powered bioacoustics platform.

www.birdweather.com

 

[Jon.Bryant]

I never knew BirdWeather existed. I am quite amazed but also a little dazed and confused.

If you explore the the map on the BirdWeather Website, the latest detection across the map flash up on the screen. There are also live sound streams from some stations including one in the UK, the USA and Europe. I am currently listening to somewhere in Florida, hearing aircraft noise, boat noise, someone talking and some indistinct bird vocals - every now and then the map says that Yellow-rumped Warbler has been detected from the recording (although I can't seem to see anything special on the live sonogram).

By the look of it the PUC is a monitoring station with built-in mics and sensors, which has wi-fi and bluetooth technology. I presume that the intention is that [stop press Caroline Wren now calling in at the Florida station] you create a permanent station and add to the network.

My main issue is how is the AI engine fed? Are the analysed sounds positively ID'd by the autonomous stations fed back into the AI system, and if so how are false ID's weeded out? There have been at least two threads on how Merlin quite regularly makes clangers and from my experience of BirdNet it is far from infallible. If these errors were automatically submitted, I can see that the integrity of the BirdNet system could be degraded rather than improved over time.

The pics of the device do not make it look very secure for a permanent install, so perhaps one for a private and secure location - I think I would prefer a fixing plate for screwing in place or for tethering the device in place using a cable lock (as per WildLife Acoustics recording devices). I am not sure I like the tripod mount option, which seems more in keeping with something meant to be lugged around rather than a permanent station.

Unfortunately my garden backs on to a main road and is very noisy, so I don't think I will be setting up one at home - and to be honest, I don't think anyone would get much please or amusement listening to my back yard!

If I had a better location, then before changing my mind, I would also like to resolve my concerns over potential false ID's, as I would be nervous of automatically uploading potential junk to an AI system - Looking at the website there were 11 detection of Long-eared Owl from one site in the UK in the middle of the day, which seems rather unlikely and perhaps a case in point.

That said, It would be pretty cool if you could look at a station at an eco-lodge and see that a target species has recently been vocalizing each day, and whether the species tends to vocalize at a particular time.

 

[LittleBitOfBreadNoCheese]

The neural network or AI is what does the ID. The question is whether some versions of the client have their own NN. It looks to me like the lite version github redirects here

GitHub - kahst/BirdNET-Analyzer: BirdNET analyzer for scientific audio data processing.

BirdNET analyzer for scientific audio data processing. - kahst/BirdNET-Analyzer

github.com

and there is a checkpoint folder with NN updates, last was just August 2022.

History for checkpoints - kahst/BirdNET-Analyzer

BirdNET analyzer for scientific audio data processing. - History for checkpoints - kahst/BirdNET-Analyzer

github.com

So this version is self-contained and BirdWeather and/or birdNET are only acting as collators. I don't know if the app enforces NN update, if not that is a bit of an issue in terms of accuracy.

The live data page shows the accuracy scores for each submission.

Data Explorer :: BirdWeather

This would then be quite different from the traditional birdNET app which send recordings to the central server for ID'ing.

 

[LittleBitOfBreadNoCheese]

Incidentally it seems the BirdWeather app, so far only available for iPhone/iPad, is also a self-contained client and server for birdNET although not everyone is perhaps savvy enough to understand the distinction. I've bolded the key bits.

"
Our BirdWeather Mobile App is now available in the App Store for iPhone/iPad! With the BirdWeather app, you’ve got the full power of the BirdNET AI in your pocket. Featuring:

• Continuously listens for bird sounds with on-board AI processing!
• Post your detections/soundscapes to the BirdWeather site (optional)
• Cached detections/soundscapes when you’re out of coverage

"
So if you aren't online it will detect the species on-board and save for when you reconnect and are ready to submit your IDs.

 

[Jon.Bryant]

Our BirdWeather Mobile App is now available in the App Store for iPhone/iPad! With the BirdWeather app, you’ve got the full power of the BirdNET AI in your pocket. Featuring:

• Continuously listens for bird sounds with on-board AI processing!
• Post your detections/soundscapes to the BirdWeather site (optional)
• Cached detections/soundscapes when you’re out of coverage

I read this as well, but I have downloaded the BirdWeather app and to be honest it is a pretty simple looking - A 'tap to listen' button and a settings button. The setting button leads to a screen with the following options

1. Detections
   1. Probability %
   2. Confidence %
2. Privacy
   1. Policy text
   2. Post to Server
   3. Listen in Background
3. Notifications
   1. Show Notifications
4. Audio Input
   1. Audio Input Source
   2. MIC gain
5. About
   1. App version

It is interesting that the top of the settings page states 'Station will be automatically created when first species are detected' which reads as if you will be shown as a station on the web page when a bird is detected whether you like it or not.

The whole app only uses 112.2MB of memory, which is surely too small for the NN to be downloaded to your phone. There are no options to download regional packages, and no options for 'off-line' and 'on-line' working.

I suspect that the 'cached detection', may not necessarily be identified species, but purely clips of sound where a potential bird has been detected. You can do this (perhaps a bit more manually) with the BirdNet app (i.e. save a sound for later upload and analysis).

I also think it would be a bit odd if the BirdNet app needs to hook up to the internet for an ID (which it does), but BirdWeather (using the same analyser) doesn't.

Looking at the three apps, I personally think that Merlin and BirdNet are better. As I say BirdWeather seems simpler, with fewer options - I also found their policy text a little scary (it reads as if they may obtain a lot of info about you). I suppose it all comes down to do you want to share your detections and location online or not.

The question is whether some versions of the client have their own NN. It looks to me like the lite version github redirects here

I am note sure that github includes the actual NN. I suspect that the 'BirdNet Analyser' published on GitHub is really just a package to allow developers to access and harness the NN. The analyser package could easily do this through facilitating a hook up to a server. Again I think the files on github are too small to contain the complete solution. Also if the purpose of placing BirdNet on github was to make it truly open source, I would suspect that they would then have full details of how to programme the NN, so you could train your own analyser from scratch for specific bird acoustics. The guidance notes read to me as if they are just a set of instructions to utilize the analyser in your own application.

 

[LittleBitOfBreadNoCheese]

The analyzer uses TensorFlowLite, the model is in the checkpoints folder (*.tflite). I've run it with and without internet connection and it produces exactly the same output. The species list in the 'Global 3k model update' has 3337 species listed. If you download the prepackaged Windows version though it still seems to have the previous version supporting 2k species only.

D:\Birds\birdNET-analyzer>BirdNET-Analyzer.exe --i example\ --o example\
Species list contains 2434 species
Found 1 files to analyze
Analyzing example\soundscape.wav
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Finished example\soundscape.wav in 23.34 seconds

D:\Birds\birdNET-analyzer>pause
Press any key to continue . . .

Output:
Selection View Channel Begin Time (s) End Time (s) Low Freq (Hz) High Freq (Hz) Species Code Common Name Confidence
1 Spectrogram 1 1 0 3.0 150 12000 bkcchi Black-capped Chickadee 0.7889
2 Spectrogram 1 1 0 3.0 150 12000 tuftit Tufted Titmouse 0.3439
3 Spectrogram 1 1 111.0 114.0 150 12000 engine Engine 0.1875
4 Spectrogram 1 1 117.0 120.0 150 12000 amegfi American Goldfinch 0.2864
5 Spectrogram 1 1 21.0 24.0 150 12000 blujay Blue Jay 0.2268
6 Spectrogram 1 1 30.0 33.0 150 12000 houfin House Finch 0.1527
7 Spectrogram 1 1 36.0 39.0 150 12000 daejun Dark-eyed Junco 0.4943
8 Spectrogram 1 1 42.0 45.0 150 12000 daejun Dark-eyed Junco 0.3797
9 Spectrogram 1 1 42.0 45.0 150 12000 merlin Merlin 0.1177
10 Spectrogram 1 1 54.0 57.0 150 12000 houfin House Finch 0.1250
11 Spectrogram 1 1 6.0 9.0 150 12000 tuftit Tufted Titmouse 0.4345
12 Spectrogram 1 1 6.0 9.0 150 12000 blctit4 Black-crested Titmouse 0.2278
13 Spectrogram 1 1 60.0 63.0 150 12000 daejun Dark-eyed Junco 0.3530
14 Spectrogram 1 1 60.0 63.0 150 12000 rusbla Rusty Blackbird 0.2101
15 Spectrogram 1 1 69.0 72.0 150 12000 houfin House Finch 0.2131
16 Spectrogram 1 1 72.0 75.0 150 12000 houfin House Finch 0.2301
17 Spectrogram 1 1 9.0 12.0 150 12000 houfin House Finch 0.1554
18 Spectrogram 1 1 90.0 93.0 150 12000 amegfi American Goldfinch 0.5028
19 Spectrogram 1 1 93.0 96.0 150 12000 houfin House Finch 0.2107
20 Spectrogram 1 1 96.0 99.0 150 12000 amegfi American Goldfinch 0.1396

 

[LittleBitOfBreadNoCheese]

I also ran it over some random birdNET files from my phone and got plausible output eg

Selection View Channel Begin Time (s) End Time (s) Low Freq (Hz) High Freq (Hz) Species Code Common Name Confidence
1 Spectrogram 1 1 0 3.0 150 12000 pifgoo Pink-footed Goose 0.8884
2 Spectrogram 1 1 3.0 6.0 150 12000 pifgoo Pink-footed Goose 0.7777

 

[Jon.Bryant]

I also ran it over some random birdNET files from my phone and got plausible output eg

Selection View Channel Begin Time (s) End Time (s) Low Freq (Hz) High Freq (Hz) Species Code Common Name Confidence
1 Spectrogram 1 1 0 3.0 150 12000 pifgoo Pink-footed Goose 0.8884
2 Spectrogram 1 1 3.0 6.0 150 12000 pifgoo Pink-footed Goose 0.7777

Very interesting. Did you try analysing these without an internet connection? If that works and the footprint is so small, why doesn't the BirdNet app work offline?

 

[Jon.Bryant]

why doesn't the BirdNet app work offline

Selection View Channel Begin Time (s) End Time (s) Low Freq (Hz) High Freq (Hz) Species Code Common Name Confidence
1 Spectrogram 1 1 0 3.0 150 12000 pifgoo Pink-footed Goose 0.8884
2 Spectrogram 1 1 3.0 6.0 150 12000 pifgoo Pink-footed Goose 0.7777

This is test from the BirdNet website regarding the BirdNet app.

'This app lets you record a file using the internal microphone of your Android or iOS device and an artificial neural network will tell you the most probable bird species present in your recording. We use the native sound recording feature of smartphones and tablets as well as the GPS-service to make predictions based on location and date. Give it a try! Please note: We need to transfer the audio recordings to our servers in order to process the files. Recording quality may vary depending on your device. External microphones will probably increase the recording quality.'

 

[LittleBitOfBreadNoCheese]

Very interesting. Did you try analysing these without an internet connection? If that works and the footprint is so small, why doesn't the BirdNet app work offline?

I don't know why the original birdNET chose to stay with a client server model especially as I find it sometimes takes quite a few seconds to respond. But it does have the advantage they stay in charge of the updates, if you follow their twitter they add a new species just about every day I think.
eg today

https://twitter.com/x/status/1612086750452613123

By contrast the github just says:

• Keep BirdNET updated by running git pull for BirdNET-Analyzer folder occasionally

 

[timsterc]

Hi Everyone!

I'm Tim from BirdWeather - stumbled across this thread and thought I'd introduce myself.
Happy to answer any questions about the system - PUC, etc.

Cheers!
-Tim

 

[LittleBitOfBreadNoCheese]

Hi, thanks for asking.
1. Will the android app be made available to Huawei Android users like me? (need an alternative source to the Google store). I run birdNET fine, luckily it doesn't seem to rely on GMS.
2. Why does your map report dogs, cars and gunshots (!)? I can filter in entries including Dog but not filter out? Also sometimes I'd like to ignore Robin as it drowns out the rest of the data. For example the other night there was a dog endlessly barking on Crete.
3. I'd like to monitor a small loch that's basically in a marsh. Any ideas on how one might do this? It is going to need super level waterproofing.

 

[timsterc]

@LittleBitOfBreadNoCheese
In answer to your questions ...
1 - We don't at present have an Android app, and it may remain that way. As we're running the BirdNET model on the mobile, it does require a fair bit of (processing) resources to keep up with a live stream. The variance across Android models makes me nervous trying to support a free app.
2 - The newer version of the AI reports a number of additional species and other (noise pollution level) sounds. At present we don't have a user-settable exclusion list. I'll have a think about that one.
3 - You could check out the BirdNET-Pi site - Discussions · mcguirepr89/BirdNET-Pi - people have deployed those into some pretty rough locations.

 

[LittleBitOfBreadNoCheese]

Thanks, I'm looking out for the release of the PUC!

 

[timsterc]

Me too! My first off-the-line production samples should be here next week. Got slowed down with covid rolling through China, but should have volume sometime in mid/late February.

 

[MiddleRiver]

Any idea what cost for PUC will be?

 

[Jon.Bryant]

I'm Tim from BirdWeather - stumbled across this thread and thought I'd introduce myself.
Happy to answer any questions about the system - PUC, etc.

Hi Tim,

I am not sure if you noticed the conversation in the tread above, but I am confused about whether BirdWeather (which I understand utilises BirdNET Analyser) can ‘recognise’ species without an internet connection. BirdNET suggests that sounds must be uploaded to the server for analysis and recognition. I also note that the PUC has Wifi and Bluetooth connection, so wonder if the unit relies on being tethered to a mobile phone or being connected to a wifi network - this would seem to make sense as the BirdWeathers website seems to be a live/near live collation of recording station data.

If the PUC does rely on an internet connection, I presume that the intention is also to review results through the web interface - if not how does the PUC broadcast the results of analysis to the operator?

I have tried asking BirdNET to clarify wether BirdNET Abalyser relies an internet collection - either for processing and breakdown of the sound in component parts or for subsequent pattern matching, but have not received a response.

Regards

Jon Bryant

 

[timsterc]

Hi Jon,

BirdWeather.com at present is basically a visualization/repository for BirdNET (generated) detections/soundscapes.
There are currently 4 different sources, all of which work slightly differently.

• BirdNET-Pi stations run the older 6K-species BirdNET model locally on a Raspberry Pi, posting their detections to BirdWeather.
• BirdWeather App stations are running the BirdWeather iOS app, which runs the 3K-species BirdNET-Analyzer model on the mobile phone, again posting detections/soundscapes to BirdWeather.
• The Red and Blue station markers are YouTube and LocusSonus stations respectively. I basically run a load of concurrent versions of BirdNET-Analyzer on a Mac Mini M1 on my desk, listening to the streams, then posting real-time detections to BirdWeather.

While capable of a number of modes of operation, the primary use-case for the PUC would be as a portable drop-off style recorder, with the built-in SD card storing all of the sensor and audio data. From there you would be able to run BirdNET-Analyzer on the recorded audio files.

Hope that helps.

Cheers,
-Tim

 

[Jon.Bryant]

BirdWeather.com at present is basically a visualization/repository for BirdNET (generated) detections/soundscapes.
There are currently 4 different sources, all of which work slightly differently.

• BirdNET-Pi stations run the older 6K-species BirdNET model locally on a Raspberry Pi, posting their detections to BirdWeather.
• BirdWeather App stations are running the BirdWeather iOS app, which runs the 3K-species BirdNET-Analyzer model on the mobile phone, again posting detections/soundscapes to BirdWeather.
• The Red and Blue station markers are YouTube and LocusSonus stations respectively. I basically run a load of concurrent versions of BirdNET-Analyzer on a Mac Mini M1 on my desk, listening to the streams, then posting real-time detections to BirdWeather.

While capable of a number of modes of operation, the primary use-case for the PUC would be as a portable drop-off style recorder, with the built-in SD card storing all of the sensor and audio data. From there you would be able to run BirdNET-Analyzer on the recorded audio files.

Hope that helps.

Thanks for this explanation. However, I am still a bit fuzzy about how the different stations complete the analysis. The reason I say this because there has been some excitement as to whether you can take a device into the field, where there is no wi-fi or mobile phone coverage and to still be able to automatically identify species. The confusion stems from the fact that the BirdNet state on their web page that sounds must be uploaded to their server for analysis and identification, and this is even the case for the mobile app. This seems to be potentially at odds with BirdNet-Pi stations and devices using BirdNet-Analyser, that seem to be able to be running autonomous identifications without relying on an internet connection.

I note that the BirdNet-Pi and BirdWeather App devices live post to BirdWeather, so obviously require an internet connection for this purpose. But I am unclear as to whether this connection is also being used surreptitiously by the BirdNet model or BirdNet-Analyser. In particular the BirdNet-Analyser seems to have a very small footprint if it is indeed a standalone package able to detect, analyse and identify 3000 species.

It is interesting that you state that the PUC is 'a portable drop-off style recorder, with the built-in SD card storing all of the sensor and audio data. From there you would be able to run BirdNET-Analyzer on the recorded audio files.' So I understand this is not a recorder with built in analyser, that runs autonomously and can be interrogated to determine what sounds have been detected and what species have been identified?

My questions really relate to how the BirdNet software works, and whether it relies on an internet connection, so it is a bit frustrating that BirdNet have not responded to me email. As you are obviously very conversant with stations running the BirdNet model and BirdNet-Analyser, any comments on how the underlying analysers work would be greatly appreciated.