• Welcome to BirdForum, the internet's largest birding community with thousands of members from all over the world. The forums are dedicated to wild birds, birding, binoculars and equipment and all that goes with it.

    Please register for an account to take part in the discussions in the forum, post your pictures in the gallery and more.
ZEISS DTI thermal imaging cameras. For more discoveries at night, and during the day.

Reliability of identifications: Some research (1 Viewer)

Mono

Hi!
Staff member
Supporter
Europe
I stumbled across this paper from back in Feb 2019.

"Seeing rare birds where there are none: self-rated expertise predicts correct species identification, but also more false rarities"

Nils Bouillard, Rachel L. White, Hazel A. Jackson, Gail E. Austen, Julia Schroeder

https://ecoevorxiv.org/9z63v/

The authors tested 3000 people on their bird id skills. Whilst the self proclaimed experts got more correct, they were also more likely to misidentify birds as rarities.

The test they used is here

https://docs.google.com/forms/d/e/1FAIpQLSckLu7Lq0ljje8aIfleot2iVX0ExkP-WKMPU6zbWn25E2aIrA/viewform
 
I stumbled across this paper from back in Feb 2019.

"Seeing rare birds where there are none: self-rated expertise predicts correct species identification, but also more false rarities"

Nils Bouillard, Rachel L. White, Hazel A. Jackson, Gail E. Austen, Julia Schroeder

https://ecoevorxiv.org/9z63v/

The authors tested 3000 people on their bird id skills. Whilst the self proclaimed experts got more correct, they were also more likely to misidentify birds as rarities.

The test they used is here

https://docs.google.com/forms/d/e/1FAIpQLSckLu7Lq0ljje8aIfleot2iVX0ExkP-WKMPU6zbWn25E2aIrA/viewform

It seems pretty obvious having looked at the paper and the test - some people were having a laugh......

All the best
 
Oh yeah, the effect of "advanced birders" finding way too many "rarities" is quite obvious. One has to sometimes wonder why such a large fraction of vagrants are birds that are very similar to something locally common while simultaneously being also very in avoiding having any pictures of them taken.
 
The questions about expertise are in fact quite good and the answers probably rather objective. The main result is that more experienced birdwatchers can identify birds better than less experienced. I find this hardly surprising. More experienced people named more rarities probably because they knew about existence of such birds, and inability to read instructions is distributed evenly in all expertise groups. But it is a good recommendation that information about the experience of observer should be filed together with the observations. However, this study does not go very deep into the interesting problem about the reliability of data of "citizen science" projects.
 
I am immediately reminded of Odd Billie's treatise on submitting rarities and how inclusion of "observer familiar with the species" might be met with "committee familiar with the observer!" :eek!:

Everybody makes mistakes. Experienced birders are consulted on more difficult questions (because of their experience, doh) and therefore have the opportunity to make bigger and more public mistakes.... 8-P

I'm not sure why mention is made of data in citizen science projects. Is there some evidence that scientists don't make the same kind of mistakes as experienced birders in the field at the same frequency for given observing conditions? I'd actually suspect the opposite might be true, that a good birder will outmatch a project-oriented scientist in the field. ;)

John
 
Last edited:
Have the other people that have commented looked at the test? I remain firmly of the view that the misidentifications were deliberate and people having a laugh! I doubt the Yellow Bunting identification was even a Yellow Bunting identification. More likely the use of the old name for Yellowhammer.

All the best
 
The survey is nonsense. The over elaborate interpretation of the results is laughable.

No one thought that the Starlings were Brown-headed Cowbirds or the European Robins were Red-flanked Bluetails. Look at the underlying paper and survey. It is funny!

All the best
 
Last edited:
I doubt the Yellow Bunting identification was even a Yellow Bunting identification. More likely the use of the old name for Yellowhammer.

All the best

That certainly caused some consternation on Scilly when someone persuaded Cardboard Box to use it instead of Yellowhammer!

John
 
Have the other people that have commented looked at the test? I remain firmly of the view that the misidentifications were deliberate and people having a laugh! I doubt the Yellow Bunting identification was even a Yellow Bunting identification. More likely the use of the old name for Yellowhammer.

All the best

I have looked at the test and I openly admit that I am not sure with many of the birds, in particular the drawings - how do you even ID a drawing??? - and some of the weird angles. It is also not made extremely clear in the formulation where should be these birds be assumed to be observed.
 
What's 'cardboard box' in this context?

Completely irrelevant to this thread but...
In the 80's and 90's the October birding scene on Scilly was extremely active.
As an efficient method of spreading news, birders adopted handheld CB radios.
People adopted nicknames (or handles as CB aficionados would say).

A retired resident on Scilly, living high up in the area called Telegraph, was an avid CB fan. The story I know is that he followed the birders conversations from early October and enjoyed the virtual company, although he knew nothing about birds other than a countryman's familiarity of the birds around him.

At the peak of CB use there was often a manned (personed really, as there was often a woman's voice) powerful mains powered CB broadcasting from the Porthcressa Hotel (a pub beloved by birders), this was later replicated at Old Town Café and Longstones Café.

However, one rarity rich day, folks were calling up trying to get information.
All to no avail, people were watching rare birds !!
However, dear old Cardboard Box stepped in to the void and started relaying messages. He was located in possibly the best spot and had a very powerful CB.
Thus was born the legend of Cardboard Box
We had a whip round for a bird guide for him but there are many amusing anecdotes around his mispronunciation and general "getting mixed up" but he was a lovely old bloke.
Sadly, he died a few years ago and is sadly missed.
 
I have looked at the test and I openly admit that I am not sure with many of the birds, in particular the drawings - how do you even ID a drawing??? - and some of the weird angles. It is also not made extremely clear in the formulation where should be these birds be assumed to be observed.

Jan

Don't overthink them. The drawings are quite 'strong' but maybe I am used to Mike Langman's illustrations. I enjoyed a mural of his at a local nature reserve (Ham Wall) on Sunday.

I believe that they are all very obvious identifications really.

All the best
 
The survey is nonsense. The over elaborate interpretation of the results is laughable.

No one thought that the Starlings were Brown-headed Cowbirds
or the European Robins were Red-flanked Bluetails. Look at the underlying paper and survey. It is funny!

All the best

I don't know about that. From one of the figures in the paper, the brown-headed cowbird mistake seems to be on this image: https://lh4.googleusercontent.com/2...77_RNS5pY2maxSvdjxiiN8paOnYU8TZ2Yl4B0QQ_=w740

This does have a reasonable resemblance to a female or juvenile brown-headed cowbird. Of course, the beak shape is all wrong and there are other problems, but I have seen far worse misidentifications. This looks like a reasonable fit for the hypothesis "didn't recognize the species right away [slightly tough angle and/or not familiar with juveniles], so looked at a North American website". The authors state that mistaken ID of exotics correlates with use of reference material.

It's also worth noting that the "identified rare or exotic birds" trend is not linear. It peaks at a self-rated expertise of 4 ("familiar with most British species, especially common birds"). I don't think this is "experts having a laugh", I think it's mostly sincere mistakes. Most of the "trend" is simply due to novice birders not ever considering exotics, so their rate is near zero. Statistically, that makes correlation of exotics with expertise almost inevitable.

To me the authors are naive in being surprised that their respondents would consider exotics. They never stated that all birds shown were native to Britain. Many respondents may have assumed so, but the fact that (at least) 4% did not is hardly surprising.
 
Natreb

Many thanks. All noted but it is entitled - "Common British Birds Identification Quiz". I suppose that it is possible but my views remain unaltered. (I have seen Brown-headed Cowbird in Britain but 5 records is not common.)

Can you tell whether it was a Starling, Chaffinch, House Sparrow, Greenfinch, Blue Tit or Robin that was identified as Cream-coloured Courser?!?

Edit - just to add a personal Cowbird pic from New England in 2016.

All the best

Paul
 

Attachments

  • Brown-headed Cowbird (reduced).jpg
    Brown-headed Cowbird (reduced).jpg
    461 KB · Views: 29
Last edited:
I am sorry, maybe I am just not the brightest person in the world, but to me it was not clear immediately whether the birds are in Britain or not. Maybe it's written in a title or somewhere, but not where I looked at first glance and I was not there to give it a thorough criminal investigation. Is is such a stretch to assume that other random visitors may have had the same experience?

Also I am a member of many "beginner friendly" groups on FB where I routinely help laypeople ID birds. There is a huge population of people who "like birds" but have a zero idea that different birds live in different areas and they constantly put forward absurdly out-of-range IDs, because they have put "small green bird" into google and the second hit looked pretty much like what they just saw - who cares that it was taken in Singapore ...
 
Also I am a member of many "beginner friendly" groups on FB where I routinely help laypeople ID birds. There is a huge population of people who "like birds" but have a zero idea that different birds live in different areas and they constantly put forward absurdly out-of-range IDs, because they have put "small green bird" into google and the second hit looked pretty much like what they just saw - who cares that it was taken in Singapore ...

Just occasionally it's right, as well: Aunty Emily's escaped Cockatiel/Budgerigar/Java Sparrow.... I was freezing my bits off at Farlington Marshes one January and found an Eclectus Parrot in the bushes. Go figure!

Notwithstanding which I think Paul makes a good case for the frivolous answer theory.

John
 
I am sorry, maybe I am just not the brightest person in the world, but to me it was not clear immediately whether the birds are in Britain or not. Maybe it's written in a title or somewhere, but not where I looked at first glance and I was not there to give it a thorough criminal investigation. Is is such a stretch to assume that other random visitors may have had the same experience?

Also I am a member of many "beginner friendly" groups on FB where I routinely help laypeople ID birds. There is a huge population of people who "like birds" but have a zero idea that different birds live in different areas and they constantly put forward absurdly out-of-range IDs, because they have put "small green bird" into google and the second hit looked pretty much like what they just saw - who cares that it was taken in Singapore ...

Opisska

I think that if anything, you are too bright and are overthinking this. It really is as simple as looking at the title of the quiz:-

"Common British Birds Identification Quiz"

House Sparrow (drawing)
Robin
Chaffinch
Starling
Starling
Blue Tit
Greenfinch
Starling (drawing)
Blue Tit
Robin
Blue Tit (drawing)
Chaffinch
Chaffinch (drawing)
Greenfinch
Greenfinch (drawing)
House Sparrow
Greenfinch
Starling
Robin (drawing)
House Sparrow
House Sparrow
Chaffinch
Blue Tit
Robin

6 species occur 4 times each with no doubt an attempt to have similar difficulties in different orders eg a drawing, a non-adult male plumage, obscuring some characters, etc.

This is a fabricated ill-thought through survey designed to produce a particular result. It has many flaws including as you have already pointed out the stylised nature of the drawings which does not even appear in the last question as a reason why any of the birds were difficult to identify.

All the best
 
Last edited:
Just occasionally it's right, as well: Aunty Emily's escaped Cockatiel/Budgerigar/Java Sparrow.... I was freezing my bits off at Farlington Marshes one January and found an Eclectus Parrot in the bushes. Go figure!

Notwithstanding which I think Paul makes a good case for the frivolous answer theory.

John

Who said Cream-coloured Courser and to which image?!?!

A friend studied anthropology in the 80's at university. He spent a year with an African tribe earnestly studying them. Virtually all of his 80's text books have been debunked a quarter of a century later and the conclusion he has reached is that with the language barrier and his naivety the tribe just spent the year having a laugh at his expense and taking the mickey out of him by doing stuff that they would not otherwise have done. He said to me - "lets face it, it is what we would do to a naïve student!" :t:

All the best
 
Last edited:
Warning! This thread is more than 4 years ago old.
It's likely that no further discussion is required, in which case we recommend starting a new thread. If however you feel your response is required you can still do so.

Users who are viewing this thread

Back
Top