Join for FREE
It only takes a minute!
Zeiss - Always on the lookout for something special – Shop now

Welcome to BirdForum.
BirdForum is the net's largest birding community, dedicated to wild birds and birding, and is absolutely FREE! You are most welcome to register for an account, which allows you to take part in lively discussions in the forum, post your pictures in the gallery and more.

Reliability of identifications: Some research

Reply
 
Thread Tools Rate Thread
Old Wednesday 22nd January 2020, 16:35   #1
Mono
Hi!
 
Mono's Avatar

 
Join Date: Dec 2005
Location: Lake District,UK
Posts: 1,979
Reliability of identifications: Some research

I stumbled across this paper from back in Feb 2019.

"Seeing rare birds where there are none: self-rated expertise predicts correct species identification, but also more false rarities"

Nils Bouillard, Rachel L. White, Hazel A. Jackson, Gail E. Austen, Julia Schroeder

https://ecoevorxiv.org/9z63v/

The authors tested 3000 people on their bird id skills. Whilst the self proclaimed experts got more correct, they were also more likely to misidentify birds as rarities.

The test they used is here

https://docs.google.com/forms/d/e/1F...2aIrA/viewform
Mono is offline  
Reply With Quote
Old Wednesday 22nd January 2020, 18:47   #2
Paul Chapman
Registered User
 
Paul Chapman's Avatar

 
Join Date: Sep 2008
Location: Clevedon
Posts: 9,480
Quote:
Originally Posted by Mono View Post
I stumbled across this paper from back in Feb 2019.

"Seeing rare birds where there are none: self-rated expertise predicts correct species identification, but also more false rarities"

Nils Bouillard, Rachel L. White, Hazel A. Jackson, Gail E. Austen, Julia Schroeder

https://ecoevorxiv.org/9z63v/

The authors tested 3000 people on their bird id skills. Whilst the self proclaimed experts got more correct, they were also more likely to misidentify birds as rarities.

The test they used is here

https://docs.google.com/forms/d/e/1F...2aIrA/viewform
It seems pretty obvious having looked at the paper and the test - some people were having a laugh......

All the best
__________________
Paul Chapman

I've decided to start a self-find list as self-help to reduce my level of self-harm.
Paul Chapman is offline  
Reply With Quote
Old Wednesday 22nd January 2020, 20:38   #3
opisska
Jan Ebr
 
opisska's Avatar

 
Join Date: Mar 2012
Location: Warszawa
Posts: 1,456
Oh yeah, the effect of "advanced birders" finding way too many "rarities" is quite obvious. One has to sometimes wonder why such a large fraction of vagrants are birds that are very similar to something locally common while simultaneously being also very in avoiding having any pictures of them taken.
__________________
Birds: world 2160, WP 561, gWP 598 (#1 Czech WP and gWP birder!*), bird photos
Mammals: 257, mammal photos
* and my wife is #3 in both
opisska is offline  
Reply With Quote
Old Thursday 23rd January 2020, 06:48   #4
jalid
Registered User

 
Join Date: Nov 2008
Location: Finland
Posts: 243
The questions about expertise are in fact quite good and the answers probably rather objective. The main result is that more experienced birdwatchers can identify birds better than less experienced. I find this hardly surprising. More experienced people named more rarities probably because they knew about existence of such birds, and inability to read instructions is distributed evenly in all expertise groups. But it is a good recommendation that information about the experience of observer should be filed together with the observations. However, this study does not go very deep into the interesting problem about the reliability of data of "citizen science" projects.
jalid is offline  
Reply With Quote
Old Thursday 23rd January 2020, 08:18   #5
Farnboro John
Registered User

 
Join Date: Mar 2006
Location: Farnborough
Posts: 13,775
I am immediately reminded of Odd Billie's treatise on submitting rarities and how inclusion of "observer familiar with the species" might be met with "committee familiar with the observer!"

Everybody makes mistakes. Experienced birders are consulted on more difficult questions (because of their experience, doh) and therefore have the opportunity to make bigger and more public mistakes....

I'm not sure why mention is made of data in citizen science projects. Is there some evidence that scientists don't make the same kind of mistakes as experienced birders in the field at the same frequency for given observing conditions? I'd actually suspect the opposite might be true, that a good birder will outmatch a project-oriented scientist in the field.

John

Last edited by Farnboro John : Thursday 23rd January 2020 at 10:50.
Farnboro John is online now  
Reply With Quote
Old Thursday 23rd January 2020, 08:47   #6
Paul Chapman
Registered User
 
Paul Chapman's Avatar

 
Join Date: Sep 2008
Location: Clevedon
Posts: 9,480
Have the other people that have commented looked at the test? I remain firmly of the view that the misidentifications were deliberate and people having a laugh! I doubt the Yellow Bunting identification was even a Yellow Bunting identification. More likely the use of the old name for Yellowhammer.

All the best
__________________
Paul Chapman

I've decided to start a self-find list as self-help to reduce my level of self-harm.
Paul Chapman is offline  
Reply With Quote
Old Thursday 23rd January 2020, 09:00   #7
Paul Chapman
Registered User
 
Paul Chapman's Avatar

 
Join Date: Sep 2008
Location: Clevedon
Posts: 9,480
The survey is nonsense. The over elaborate interpretation of the results is laughable.

No one thought that the Starlings were Brown-headed Cowbirds or the European Robins were Red-flanked Bluetails. Look at the underlying paper and survey. It is funny!

All the best
__________________
Paul Chapman

I've decided to start a self-find list as self-help to reduce my level of self-harm.

Last edited by Paul Chapman : Thursday 23rd January 2020 at 09:16.
Paul Chapman is offline  
Reply With Quote
Old Thursday 23rd January 2020, 10:52   #8
Farnboro John
Registered User

 
Join Date: Mar 2006
Location: Farnborough
Posts: 13,775
Quote:
Originally Posted by Paul Chapman View Post
I doubt the Yellow Bunting identification was even a Yellow Bunting identification. More likely the use of the old name for Yellowhammer.

All the best
That certainly caused some consternation on Scilly when someone persuaded Cardboard Box to use it instead of Yellowhammer!

John
Farnboro John is online now  
Reply With Quote
Old Thursday 23rd January 2020, 10:52   #9
opisska
Jan Ebr
 
opisska's Avatar

 
Join Date: Mar 2012
Location: Warszawa
Posts: 1,456
Quote:
Originally Posted by Paul Chapman View Post
Have the other people that have commented looked at the test? I remain firmly of the view that the misidentifications were deliberate and people having a laugh! I doubt the Yellow Bunting identification was even a Yellow Bunting identification. More likely the use of the old name for Yellowhammer.

All the best
I have looked at the test and I openly admit that I am not sure with many of the birds, in particular the drawings - how do you even ID a drawing??? - and some of the weird angles. It is also not made extremely clear in the formulation where should be these birds be assumed to be observed.
__________________
Birds: world 2160, WP 561, gWP 598 (#1 Czech WP and gWP birder!*), bird photos
Mammals: 257, mammal photos
* and my wife is #3 in both
opisska is offline  
Reply With Quote
Old Thursday 23rd January 2020, 11:25   #10
Mike C
Emeritus President at Burnage Rugby Club
 
Mike C's Avatar

 
Join Date: Dec 2002
Location: Stockport
Posts: 1,357
Quote:
Originally Posted by Farnboro John View Post
That certainly caused some consternation on Scilly when someone persuaded Cardboard Box to use it instead of Yellowhammer!

John


good old Cardboard Box
Mike C is offline  
Reply With Quote
Old Thursday 23rd January 2020, 11:29   #11
Nutcracker
Stop Brexit!
 
Nutcracker's Avatar

 
Join Date: Dec 2004
Location: UK
Posts: 19,566
What's 'cardboard box' in this context?
Nutcracker is offline  
Reply With Quote
Old Thursday 23rd January 2020, 13:18   #12
Mike C
Emeritus President at Burnage Rugby Club
 
Mike C's Avatar

 
Join Date: Dec 2002
Location: Stockport
Posts: 1,357
Quote:
Originally Posted by Nutcracker View Post
What's 'cardboard box' in this context?
Completely irrelevant to this thread but...
In the 80's and 90's the October birding scene on Scilly was extremely active.
As an efficient method of spreading news, birders adopted handheld CB radios.
People adopted nicknames (or handles as CB aficionados would say).

A retired resident on Scilly, living high up in the area called Telegraph, was an avid CB fan. The story I know is that he followed the birders conversations from early October and enjoyed the virtual company, although he knew nothing about birds other than a countryman's familiarity of the birds around him.

At the peak of CB use there was often a manned (personed really, as there was often a woman's voice) powerful mains powered CB broadcasting from the Porthcressa Hotel (a pub beloved by birders), this was later replicated at Old Town Café and Longstones Café.

However, one rarity rich day, folks were calling up trying to get information.
All to no avail, people were watching rare birds !!
However, dear old Cardboard Box stepped in to the void and started relaying messages. He was located in possibly the best spot and had a very powerful CB.
Thus was born the legend of Cardboard Box
We had a whip round for a bird guide for him but there are many amusing anecdotes around his mispronunciation and general "getting mixed up" but he was a lovely old bloke.
Sadly, he died a few years ago and is sadly missed.
Mike C is offline  
Reply With Quote
Old Thursday 23rd January 2020, 13:36   #13
Nutcracker
Stop Brexit!
 
Nutcracker's Avatar

 
Join Date: Dec 2004
Location: UK
Posts: 19,566
Thanks!
Nutcracker is offline  
Reply With Quote
Old Friday 24th January 2020, 05:56   #14
Paul Chapman
Registered User
 
Paul Chapman's Avatar

 
Join Date: Sep 2008
Location: Clevedon
Posts: 9,480
Quote:
Originally Posted by opisska View Post
I have looked at the test and I openly admit that I am not sure with many of the birds, in particular the drawings - how do you even ID a drawing??? - and some of the weird angles. It is also not made extremely clear in the formulation where should be these birds be assumed to be observed.
Jan

Don't overthink them. The drawings are quite 'strong' but maybe I am used to Mike Langman's illustrations. I enjoyed a mural of his at a local nature reserve (Ham Wall) on Sunday.

I believe that they are all very obvious identifications really.

All the best
__________________
Paul Chapman

I've decided to start a self-find list as self-help to reduce my level of self-harm.
Paul Chapman is offline  
Reply With Quote
Old Saturday 25th January 2020, 07:57   #15
nartreb
Speak softly and carry a long lens
 
nartreb's Avatar

 
Join Date: Jan 2013
Location: United States
Posts: 1,393
Quote:
Originally Posted by Paul Chapman View Post
The survey is nonsense. The over elaborate interpretation of the results is laughable.

No one thought that the Starlings were Brown-headed Cowbirds
or the European Robins were Red-flanked Bluetails. Look at the underlying paper and survey. It is funny!

All the best
I don't know about that. From one of the figures in the paper, the brown-headed cowbird mistake seems to be on this image: https://lh4.googleusercontent.com/2i...2Yl4B0QQ_=w740

This does have a reasonable resemblance to a female or juvenile brown-headed cowbird. Of course, the beak shape is all wrong and there are other problems, but I have seen far worse misidentifications. This looks like a reasonable fit for the hypothesis "didn't recognize the species right away [slightly tough angle and/or not familiar with juveniles], so looked at a North American website". The authors state that mistaken ID of exotics correlates with use of reference material.

It's also worth noting that the "identified rare or exotic birds" trend is not linear. It peaks at a self-rated expertise of 4 ("familiar with most British species, especially common birds"). I don't think this is "experts having a laugh", I think it's mostly sincere mistakes. Most of the "trend" is simply due to novice birders not ever considering exotics, so their rate is near zero. Statistically, that makes correlation of exotics with expertise almost inevitable.

To me the authors are naive in being surprised that their respondents would consider exotics. They never stated that all birds shown were native to Britain. Many respondents may have assumed so, but the fact that (at least) 4% did not is hardly surprising.
nartreb is offline  
Reply With Quote
Old Saturday 25th January 2020, 23:38   #16
Paul Chapman
Registered User
 
Paul Chapman's Avatar

 
Join Date: Sep 2008
Location: Clevedon
Posts: 9,480
Natreb

Many thanks. All noted but it is entitled - "Common British Birds Identification Quiz". I suppose that it is possible but my views remain unaltered. (I have seen Brown-headed Cowbird in Britain but 5 records is not common.)

Can you tell whether it was a Starling, Chaffinch, House Sparrow, Greenfinch, Blue Tit or Robin that was identified as Cream-coloured Courser?!?

Edit - just to add a personal Cowbird pic from New England in 2016.

All the best

Paul
Attached Thumbnails
Click image for larger version

Name:	Brown-headed Cowbird (reduced).jpg
Views:	25
Size:	461.0 KB
ID:	716359  
__________________
Paul Chapman

I've decided to start a self-find list as self-help to reduce my level of self-harm.

Last edited by Paul Chapman : Sunday 26th January 2020 at 18:24.
Paul Chapman is offline  
Reply With Quote
Old Tuesday 28th January 2020, 12:11   #17
opisska
Jan Ebr
 
opisska's Avatar

 
Join Date: Mar 2012
Location: Warszawa
Posts: 1,456
I am sorry, maybe I am just not the brightest person in the world, but to me it was not clear immediately whether the birds are in Britain or not. Maybe it's written in a title or somewhere, but not where I looked at first glance and I was not there to give it a thorough criminal investigation. Is is such a stretch to assume that other random visitors may have had the same experience?

Also I am a member of many "beginner friendly" groups on FB where I routinely help laypeople ID birds. There is a huge population of people who "like birds" but have a zero idea that different birds live in different areas and they constantly put forward absurdly out-of-range IDs, because they have put "small green bird" into google and the second hit looked pretty much like what they just saw - who cares that it was taken in Singapore ...
__________________
Birds: world 2160, WP 561, gWP 598 (#1 Czech WP and gWP birder!*), bird photos
Mammals: 257, mammal photos
* and my wife is #3 in both
opisska is offline  
Reply With Quote
Old Tuesday 28th January 2020, 13:16   #18
Farnboro John
Registered User

 
Join Date: Mar 2006
Location: Farnborough
Posts: 13,775
Quote:
Originally Posted by opisska View Post
Also I am a member of many "beginner friendly" groups on FB where I routinely help laypeople ID birds. There is a huge population of people who "like birds" but have a zero idea that different birds live in different areas and they constantly put forward absurdly out-of-range IDs, because they have put "small green bird" into google and the second hit looked pretty much like what they just saw - who cares that it was taken in Singapore ...
Just occasionally it's right, as well: Aunty Emily's escaped Cockatiel/Budgerigar/Java Sparrow.... I was freezing my bits off at Farlington Marshes one January and found an Eclectus Parrot in the bushes. Go figure!

Notwithstanding which I think Paul makes a good case for the frivolous answer theory.

John
Farnboro John is online now  
Reply With Quote
Old Tuesday 28th January 2020, 13:24   #19
Paul Chapman
Registered User
 
Paul Chapman's Avatar

 
Join Date: Sep 2008
Location: Clevedon
Posts: 9,480
Quote:
Originally Posted by opisska View Post
I am sorry, maybe I am just not the brightest person in the world, but to me it was not clear immediately whether the birds are in Britain or not. Maybe it's written in a title or somewhere, but not where I looked at first glance and I was not there to give it a thorough criminal investigation. Is is such a stretch to assume that other random visitors may have had the same experience?

Also I am a member of many "beginner friendly" groups on FB where I routinely help laypeople ID birds. There is a huge population of people who "like birds" but have a zero idea that different birds live in different areas and they constantly put forward absurdly out-of-range IDs, because they have put "small green bird" into google and the second hit looked pretty much like what they just saw - who cares that it was taken in Singapore ...
Opisska

I think that if anything, you are too bright and are overthinking this. It really is as simple as looking at the title of the quiz:-

"Common British Birds Identification Quiz"

House Sparrow (drawing)
Robin
Chaffinch
Starling
Starling
Blue Tit
Greenfinch
Starling (drawing)
Blue Tit
Robin
Blue Tit (drawing)
Chaffinch
Chaffinch (drawing)
Greenfinch
Greenfinch (drawing)
House Sparrow
Greenfinch
Starling
Robin (drawing)
House Sparrow
House Sparrow
Chaffinch
Blue Tit
Robin

6 species occur 4 times each with no doubt an attempt to have similar difficulties in different orders eg a drawing, a non-adult male plumage, obscuring some characters, etc.

This is a fabricated ill-thought through survey designed to produce a particular result. It has many flaws including as you have already pointed out the stylised nature of the drawings which does not even appear in the last question as a reason why any of the birds were difficult to identify.

All the best
__________________
Paul Chapman

I've decided to start a self-find list as self-help to reduce my level of self-harm.

Last edited by Paul Chapman : Tuesday 28th January 2020 at 13:33.
Paul Chapman is offline  
Reply With Quote
Old Tuesday 28th January 2020, 13:37   #20
Paul Chapman
Registered User
 
Paul Chapman's Avatar

 
Join Date: Sep 2008
Location: Clevedon
Posts: 9,480
Quote:
Originally Posted by Farnboro John View Post
Just occasionally it's right, as well: Aunty Emily's escaped Cockatiel/Budgerigar/Java Sparrow.... I was freezing my bits off at Farlington Marshes one January and found an Eclectus Parrot in the bushes. Go figure!

Notwithstanding which I think Paul makes a good case for the frivolous answer theory.

John
Who said Cream-coloured Courser and to which image?!?!

A friend studied anthropology in the 80's at university. He spent a year with an African tribe earnestly studying them. Virtually all of his 80's text books have been debunked a quarter of a century later and the conclusion he has reached is that with the language barrier and his naivety the tribe just spent the year having a laugh at his expense and taking the mickey out of him by doing stuff that they would not otherwise have done. He said to me - "lets face it, it is what we would do to a naïve student!"

All the best
__________________
Paul Chapman

I've decided to start a self-find list as self-help to reduce my level of self-harm.

Last edited by Paul Chapman : Tuesday 28th January 2020 at 13:41.
Paul Chapman is offline  
Reply With Quote
Old Tuesday 28th January 2020, 13:41   #21
dantheman
Bah humbug
 
dantheman's Avatar

 
Join Date: Dec 2005
Location: Cornwall
Posts: 12,502
Blog Entries: 2
Reasonably early on I twigged that it was a simple repetition of the same species, which kinda spoilt it for me. (I read the title properly as 'Common' and 'British' btw).

Maybe would have required a trickier analysis, but think throwing in a few other species or a similar test with some other actual id challenges (think eg a linnet or twite, a dunnock, or even an Azure Tit for the relevant sections)



I think it's interesting and a start - obviously more can be done, and with increasing citizen science and interest in wildlife a worthwhile area to double-check on how it is all going by attempting to research what people's motivations/reasonings/shorcomings are (cf recent ebird thread).
__________________
stithiansreservoirbirding.blogspot.co.uk/ - last update 10/11/15 - really rather remarkable still!!!
dantheman is online now  
Reply With Quote
Old Tuesday 28th January 2020, 13:56   #22
Paul Chapman
Registered User
 
Paul Chapman's Avatar

 
Join Date: Sep 2008
Location: Clevedon
Posts: 9,480
Quote:
Originally Posted by dantheman View Post
Reasonably early on I twigged that it was a simple repetition of the same species, which kinda spoilt it for me. (I read the title properly as 'Common' and 'British' btw).

Maybe would have required a trickier analysis, but think throwing in a few other species or a similar test with some other actual id challenges (think eg a linnet or twite, a dunnock, or even an Azure Tit for the relevant sections)

I think it's interesting and a start - obviously more can be done, and with increasing citizen science and interest in wildlife a worthwhile area to double-check on how it is all going by attempting to research what people's motivations/reasonings/shorcomings are (cf recent ebird thread).
Dan

The eBird quiz can be quite addictive when you set it to more challenging locations and produces a byproduct of rating photos, inputting information that assists on their database and correcting errors:-

https://ebird.org/quiz/

Some of the photos are unidentifiable though.

I flagged a Lesser Black-backed Gull photo labelled as a Herring Gull yesterday from Somerset for this year. (Possibly just mislabelling as it was an adult.)

There was a German one where you got a score and it was timed which I messed around with in the 90's. Not sure if that is still around.

All the best
__________________
Paul Chapman

I've decided to start a self-find list as self-help to reduce my level of self-harm.
Paul Chapman is offline  
Reply With Quote
Old Tuesday 28th January 2020, 14:04   #23
dantheman
Bah humbug
 
dantheman's Avatar

 
Join Date: Dec 2005
Location: Cornwall
Posts: 12,502
Blog Entries: 2
Cheers Paul, will have a look.

Whilst on quizzes (as a tool for advancement), there are two others ...

One was a counting birds in flocks quiz - you were shown a flock of birds quickly passing across the screen and had a few seconds only to try and guesstimate/estimate numbers (funnily enough bumped into the German programmer who made it some years back in Israel whilst birding).

Second was here on BF - a user (from Scandinavia I think) had a quiz in their bottom signature line with a random selection of 10 or so (N Europe) bird songs/calls to identify.


Both pretty useful, anyone know if either still around?
__________________
stithiansreservoirbirding.blogspot.co.uk/ - last update 10/11/15 - really rather remarkable still!!!
dantheman is online now  
Reply With Quote
Old Tuesday 28th January 2020, 19:04   #24
foresttwitcher
Registered User

 
Join Date: May 2011
Location: The Chilterns
Posts: 1,825
Probably Bavarian Bird Quiz as I knew it, Paul, now Computerbirding: http://www.computerbirding.com/what/cbirding_2018_e.php
__________________
Pete.

Can't see the birds for the trees!

Last edited by foresttwitcher : Tuesday 28th January 2020 at 19:05. Reason: Addition.
foresttwitcher is offline  
Reply With Quote
Old Tuesday 28th January 2020, 19:38   #25
Paul Chapman
Registered User
 
Paul Chapman's Avatar

 
Join Date: Sep 2008
Location: Clevedon
Posts: 9,480
Quote:
Originally Posted by foresttwitcher View Post
Probably Bavarian Bird Quiz as I knew it, Paul, now Computerbirding: http://www.computerbirding.com/what/cbirding_2018_e.php
Many thanks. I think so.

This also seems quite interesting:-

https://www.birdid.no/bird/training.php

All the best
__________________
Paul Chapman

I've decided to start a self-find list as self-help to reduce my level of self-harm.
Paul Chapman is offline  
Reply With Quote
Advertisement
Reply


Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Durability and reliability of Canon IS binoculars Hermann Canon 2 Friday 20th December 2019 11:43
alexandertrent reliability question SJC Canon 13 Saturday 19th May 2012 09:01
Swarovski reliability Philip Johnson Swarovski 51 Thursday 21st July 2011 15:49
mechanical reliability of binoculars John Dracon Binoculars 26 Tuesday 1st March 2011 01:04
Reliability of Nikon Monarch SteveClark Nikon 24 Saturday 20th December 2008 20:12

{googleads}

Fatbirder's Top 1000 Birding Websites

Help support BirdForum

Page generated in 0.22677803 seconds with 40 queries
All times are GMT. The time now is 09:43.