New unified list of birds - Avilist (2 Viewers)

[Jon.Bryant]

On Green-winged Teal, it is understood that there was a vote which originally was in favour of a continued split but that was re-tabled following the paper relating to gene flow across the Bering Strait & the unanimous vote by NACC not to split them. The second vote ended up with a lump.

Yes, I had heard something similar... and I am fine if it ends up being a lump.

Interesting though, that I wouldn't have thought that gene flow in ducks was to be that unexpected, after all isn't hybridization in ducks 'a thing'. In fact I am slightly concerned about the future of Baer's Pochard for this reason - apparent Baer's now appear to be breeding much further south (including in Henan where my wife's family live). Hybrids with Ferruginous Duck are not uncommon in winter- and to be honest this is only considering obvious hybrids (I don't think there is a definite article on how to ID more subtle bird from pure Baer's - I think this is probably an important topic to resolve - The Chinese do annual surveys of the critically endangered Baer's Pochard, but if the full extent of hybridization is not apparent, the situation may be more perilous than surveys suggest. Perhaps I am being alarmist, but there seems a possible Ruddy Duck-White-headed Duck nightmare scenario - but in this case with the added difficulty that hybridization is with a naturally occurring rather than introduced species.

In other threads we have debated on Birdforum how species are defined - I am mo no expect, but a school of thought is that you look at two sympatric species of the same genus, work our what 'separates' them, then apply the same rules to allopatric species of the same genus. This means there are no hard rules over % difference in DNA, gene flow etc etc. Which is probably good - Large gulls apparently do not show much genetic difference, and birds such as Pallid and Common Swift apparently show reasonable gene flow! If the experts lumped everything based on one set of rules we may have an impoverished species list.

 

[Jon.Bryant]

Second, as I mentioned above in another comment, I don't see clements and birdlife lists going away, as I think they have purposes each is suited before, namely organizing ebird and having a convenient list to refer to status updates and changes. For those reasons I don't think maintenance of those lists is much of an extra burden. NACC and SACC certainly are not going to give up the ghost, nor do I see them passively caving to any change made in Avilist.

When I said the source lists may eventually be scrapped, I was only referring to old comments made in personal correspondence with one of the organisations - I have been working on my own three-way database and this work normally flags up any errors in the various spreadsheets, that crop up from time to time, so I have had occasional correspondence to flag up these issues. In fact I first heard about WGAC through the organisations, when I created a three-way mapping and wrote to the three organisations highlighting the differences between their taxonomies... I can't take credit for the WGAC... my plea for unification and my route map for picking low-lying fruit first, seems to have come several months after the WGAC started taking shape.... but I still think the lowest handing fruit is unification of Latin names - I get particularly peeved with different spellings of Latin names!!! Pedantry or taxonomy?

AviList is proposed to be updated yearly, and if all the planets do eventually align, is there a reason for three organisations to publish three separate MS Excel spreadsheets that say the same thing (although perhaps slightly out of sync).

I concede that the base information with need to be adapted, so that we have species groups in EBird etc. but this is slightly different.

Regarding NACC and SACC, as we have both said before, they will not go away and will retain their taxonomic opinion, but if we are all storing and submitting our sightings on platforms based on a unified list, then submission based on different regional lists will become a ball ache, which many birdwatchers and onithologists may not wish to waste their time on... if regional committees want to spend there time 'converting' records based on a unified database, to a regional taxonomic database, then that's their perogative, but I wont be wasting my time doing it for them.

 

[Paul Chapman]

Regarding NACC and SACC, as we have both said before, they will not go away and will retain their taxonomic opinion, but if we are all storing and submitting our sightings on platforms based on a unified list, then submission based on different regional lists will become a ball ache, which many birdwatchers and onithologists may not wish to waste their time on... if regional committees want to spend there time 'converting' records based on a unified database, to a regional taxonomic database, then that's their perogative, but I wont be wasting my time doing it for them.

I record in eBird which is a different taxonomic arrangement to my local birding committee. However, I record to subspecific level where relevant so my records would not need to be converted.

All the best

Paul

 

[Jon.Bryant]

I record in eBird which is a different taxonomic arrangement to my local birding committee. However, I record to subspecific level where relevant so my records would not need to be converted.

In Surrey the I believe that the county recorder automatically extracts records from BirdTrack (the BTO recording app based on the IOC checklist). I understand that quite a few counties in the UK now have the capabilities to automatically sweep up records from this program, and this is negating the need to submit records separately.

In the Surrey Bird Club newsletter we have be provided with some quite laborious instructions in how to extract EBird records to Excel, parse the information to correct species and place names and then submit our records as an Excel workbook. I presume that the parsed workbooks can then be automatically read by whatever database the county recorders are using - I think they were using a database also developed by the BTO - the days of record cards are long gone! I mention place names, but as a side point have you noticed that the character encoding for EBird exports seem incorrect (or at least was when I last tried it), so that foreign place names with special characters get mangled? Not sure if this also happen's for Rüppell's warbler etc, but I suspect that these names will also be corrupted.

So although you say your data does not have to be converted, I suspect that somewhere along the line perhaps it does... although of course the import function for the local database may do this automatically (i.e. someone has written a bit of code that says take all X's and make them Y's). The problem with all this is that somewhere behind the scenes there are a lot of databases that need to be maintained and an equal amount of import functions that need to be kept up to date to allow easy processing of desperate data... Although AI will perhaps change things, to a database 'Green-winged Teal', and 'Common Teal (Green-winged)' are different, and it takes a human eye to realize they are the same.

 

[Jon.Bryant]

I record in eBird which is a different taxonomic arrangement to my local birding committee. However, I record to subspecific level where relevant so my records would not need to be converted.

Forgot to ask. Does anyone know if any EBird records are swept up by regional recorders, when compiling regional reports.

For example a few years ago, I found a Lazuli Bunting in the Lower Rio Grand Valley, Texas, which is apparently scarce (1 or 2 records a year). I submitted to EBird and the local adjudicator asked me to provide further evidence. I then attached a photo for their review, and go a thank you message.... But I have never bothered to find out if there is a local recorder for the Rio Grande area or tried to submit the record to them. Was my submission in a way waste, and merely sits in the EBird database, or do local recorders also access and extract data from the EBird database. The Lazuli Bunting is probably a bad example as records on extralimital birds may not be that meaningful, but I suspect that it is important that local recorders access other records, as they are most likely to understand and significant changes in status of the birds on their 'patch'. We then need people with the insight to join the dots - I noticed a very local decline of Lesser Pecker on my old Cheshire patch over 40 years ago, which I suspected was due to forest management, but I never would have imagined that this may have been a precursor to their now rapid decline across the whole country.

I suspect that like most birders visiting oversees destinations, we keep records on EBird, but don't actually formally submit any records to overseas recorders.

... I still think one taxonomy cuts an awful amount of work out, and allows the easy dissemination and review of records across the globe... That said, I suspect that we are still a long time away from a complete unified list. The WGAC started their species voting work in February 2021, and with this work now reportedly complete, I understand they are now turning their hand to subspecies. Understandably, I think we can expect several years to resolve differences here - subspecies definitions are undoubtedly subtler, there are a lot more subspecies than species, and looking at existing lists there are more significant differences of opinion in the numbers of recognized subspecies, what constitutes a junior synonym, correct scientific names and assignment to species. Perhaps we can expect a good few years, if not a decade or so before the stars truly align. It could be argued that subspecies are not so important, but I would point out that the current IOC, Birdlife and Clements species lists do hide a few problems - alignment at species level, does not preclude that at a lower level subspecies are not assigned to different species by different authorities - I haven't checked the latest lists, but it was definitely fairly recently the case that one authorities subspecies of Pallas's Reed Bunting was another authorities subspecies of Common Reed Bunting...and I don't think this is something the EBird groups deals with adequately.

 

[Jon.Bryant]

I record in eBird which is a different taxonomic arrangement to my local birding committee. However, I record to subspecific level where relevant so my records would not need to be converted.

A last question, and off track a bit from the thread, but do you find the EBird groups easy to use, especially when overseas and your knowledge of regional subspecies may not be perfect. I find the groups a bit challenging - a strange mix of names sometimes using descriptive text (e.g. Black-faced Dacnis [Yellow-tufted], sometimes using regional descriptions (e.g. Great Spotted Woodpecker [Canarian]) and sometimes referring to the subspecies Latin name (e.g. Golden Tanager [arthus]). My wife sometimes asks me should I be submitting this as such and such a group, but even when I know and positively identify the subspecies, I can find it difficult to correctly align with the given EBird name - who knows off the top of their head that alboides is Hodgson's White Wagtail... I think Himalayan White Wagtail seems more intuitive and has been used by others... and field guides are not much help... most good ones include Latin names of distinct subspecies, but I have never seen a field reference that includes the Ebird group names (but perhaps they should).

And then we have the problem with missing groups. The late Martin Garner provided us guidance on how to identify Northern Great Spotted Woodpeckers (subspecies major), but there does not appear to be a way with Ebird groups to record an scarce Northern bird as opposed to a bog standard British one.

 

[Paul Chapman]

In answer to your last three:-

1. I am aware some areas extract data from eBird for their reports even locally where the eBird geographical boundaries do not match the recording boundaries despite promised changes. There is a script that is used for this purpose;
2. I am aware that this can include scraping for regional rarities but I have been asked to submit national rarities separately by reviewers; &
3. The subspecies groups are at best challenging at times and in many circumstances merge with the unidentifiable so they can be random & when I have asked I have not got clear answers or guidance.

All the best

Paul

 

[Steve Lister]

Jon - Following up Paul's response to your query about local recorders accessing eBird records, some do it very well, good examples being Cornwall, Lincolnshire, Norfolk and Yorkshire. My name is regularly listed in the Norfolk report contributors despite me only having used eBird for Norfolk birding for several years. Lincolnshire acknowledge all eBird contributors in their list, with a code. The data handling experts in some counties have very kindly offered to share their methods with others.
On the other hand some counties don't, my own included. As an eBird local reviewer I have decided to cherry-pick eBirdlists for anything I think won't get to the county database otherwise and enter it into the county records as 'J Bloggs per eBird' and this works OK, plus it saves the already overworked county recorder wading through thousands of records that are just Blue Tit x, Dunnock x, Blackbird x and so on in an enormous download.

 

[TicoTyler]

Forgot to ask. Does anyone know if any EBird records are swept up by regional recorders, when compiling regional reports.

In the case of Costa Rica, they are. Almost all members of the checklist committee for the country are also eBird reviewers, so anything new for the country from eBird gets noticed and added to the official country list.

 

[Mysticete]

When I said the source lists may eventually be scrapped, I was only referring to old comments made in personal correspondence with one of the organisations - I have been working on my own three-way database and this work normally flags up any errors in the various spreadsheets, that crop up from time to time, so I have had occasional correspondence to flag up these issues. In fact I first heard about WGAC through the organisations, when I created a three-way mapping and wrote to the three organisations highlighting the differences between their taxonomies... I can't take credit for the WGAC... my plea for unification and my route map for picking low-lying fruit first, seems to have come several months after the WGAC started taking shape.... but I still think the lowest handing fruit is unification of Latin names - I get particularly peeved with different spellings of Latin names!!! Pedantry or taxonomy?

AviList is proposed to be updated yearly, and if all the planets do eventually align, is there a reason for three organisations to publish three separate MS Excel spreadsheets that say the same thing (although perhaps slightly out of sync).

I concede that the base information with need to be adapted, so that we have species groups in EBird etc. but this is slightly different.

Regarding NACC and SACC, as we have both said before, they will not go away and will retain their taxonomic opinion, but if we are all storing and submitting our sightings on platforms based on a unified list, then submission based on different regional lists will become a ball ache, which many birdwatchers and onithologists may not wish to waste their time on... if regional committees want to spend there time 'converting' records based on a unified database, to a regional taxonomic database, then that's their perogative, but I wont be wasting my time doing it for them.

NACC and SACC decisions usually make it into the next checklist. realistically, differences will end up being minor. I don't believe, other than the list itself, that they manage any sort of "database" they would need to deal with. Really NACC and SACC will be more inspiring new changes for the global checklist, than the other way around, even if the reconciliation process makes it look like the opposite right now.

As far as formatting goes, as others have said the three checklists don't say the same thing. Birdlife includes conservation status, while there are subspecies groups and similar "taxonomic" entities in the ebird/Clements checklist. Also from the sound of things the Avilist will be much more taxonomic heavy than the existing checklist, and may include information that a casual birder doesn't want or need. SO imho there will be a place still for at least these two checklists.

 

[Kirk Roth]

'Over the next few years, BirdLife, along with Cornell and others, will be aligning fully with AviList (i.e. using it as the taxonomic basis for our work) so that in future the species displayed in Birds of the World (and used in eBird) will largely match those on the BirdLife DataZone and the IUCN Red List. However, this alignment will take some time because every taxonomic change requires considerable further work, particularly in assessing the IUCN Red List status of newly ‘split’, ‘lumped’ or revised species. BirdLife will continue to make the list of bird species assessed in each annual Red List update available in spreadsheet format.'

...as highlighted it is perhaps a bit odd to say it 'will be aligned', but will only 'largely match'.... it is either aligned or it isn't! Or perhaps they are just referring to the fact that time frames will prevent BoW and BirdLife Taxonomic List (and Datazone) from being perfectly up to date and synced as all times.

This is easy to understand when we consider that Birds of the World is comprised of full articles which take a great deal of time to edit, especially when that edit is a wholesale change regarding which species the article is about. It is inconceivable that all of these reviewed articles could be aligned completely and immediately with the speed that Birdlife, Cornell, and AviList can update their checklists.

Also, I might have detected some confusion in some other posts regarding the checklist entities - AviList is officially the "AviList Committee of the International Ornithologists Union." So I'm expecting the checklists of "AviList" and "IOC" to be the same thing. Also SACC is now an arm of IOC, so its not accurate to imply they are "doing their own thing" apart from IOC/AviList. I'll concede that NACC is a bit rogue to the process, but irrelevant to the whole bit about converting checklists - which are "refereed" in that portion of the world by the American Birding Association, which follows Cornell which follows AviList.

I expect Cornell and BirdLife to update their own databases in accordance with updates and information from the so-called "other" checklist... just like they have always done, except easier due to the conformity.

 

[jts1882]

There's also H&M, which is currently working on an update, although they've gone quiet and pulled the updates from their site. The concerns of the Trust that runs it over their intellectual property rights means it's unlikely to adopt a unified list.

 

[Jon.Bryant]

There's also H&M, which is currently working on an update, although they've gone quiet and pulled the updates from their site. The concerns of the Trust that runs it over their intellectual property rights means it's unlikely to adopt a unified list.

Interesting... I had heard that they had decided not to do another update. Assuming that they would charge again for their list, It would literally be a very difficult sell - especially as the majority have already settled on a 'free' taxonomy.

But this comes back to another previous thread, what gives any person the authority to define a taxonomy? (Peter's, Sibley & Monroe, Howard & Moore etc., etc.), and if no birdwatchers or scientists adopt the taxonomy what is the purpose?

 

[Paul Clapham]

I mention place names, but as a side point have you noticed that the character encoding for EBird exports seem incorrect (or at least was when I last tried it), so that foreign place names with special characters get mangled? Not sure if this also happen's for Rüppell's warbler etc, but I suspect that these names will also be corrupted.

I just tried downloading one of my eBird checklists which includes the ♀ and ♂ symbols in the species comments and they arrived in my computer safely. It's using UTF-8, as it should. When I read the download with the old text editor which I used to like, they show up as ? because the editor doesn't use UTF-8. I tried again using a checklist from Monfragüe PN in Spain and also had no problem reading the download into an editor which understands UTF-8.

 

[Jon.Bryant]

Also, I might have detected some confusion in some other posts regarding the checklist entities - AviList is officially the "AviList Committee of the International Ornithologists Union." So I'm expecting the checklists of "AviList" and "IOC" to be the same thing. Also SACC is now an arm of IOC, so its not accurate to imply they are "doing their own thing" apart from IOC/AviList. I'll concede that NACC is a bit rogue to the process, but irrelevant to the whole bit about converting checklists - which are "refereed" in that portion of the world by the American Birding Association, which follows Cornell which follows AviList.

Interesting.

The WGAC website states 'Heading the project are Les Christidis, WGAC Chair and former co-editor of the passerine volume of the Howard & Moore Complete Checklist of the Birds of the World edition 4; Frank Rheindt, WGAC deputy chair and ICZN Commissioner; and David Donsker from the IOC World Bird List, who will coordinate the bibliographic entry program. Members of checklist committees of individual continents are also included, such as the NACC and SACC.'

I have seen statements now from IOC, BirdLife and Clements that they are committed to aligning with Avibase. I haven't seen a similar statement from NACC. But why is the NACC on the WGAC if they follow ABA, which follows Cornell, which is committed to AviList? This would be a bit like inviting the British Ornithological Union, who used to maintain their own taxonomic opinion, but decided some time ago to follow IOC.

 

[Jon.Bryant]

I just tried downloading one of my eBird checklists which includes the ♀ and ♂ symbols in the species comments and they arrived in my computer safely. It's using UTF-8, as it should. When I read the download with the old text editor which I used to like, they show up as ? because the editor doesn't use UTF-8. I tried again using a checklist from Monfragüe PN in Spain and also had no problem reading the download into an editor which understands UTF-8.

Thanks. I think MS Excel uses UTF-8 as standard, but I see online that there are options to change to encoding... the problems I had where when reading the files in Excel, so perhaps I can find a solution. Strange that I have never had a similar issue with encoding when creating MS workbooks using Python and Pandas - I am sure there must be something like that running on the EBird Server to package and email the data.

I will have a play.

 

[Jon.Bryant]

As far as formatting goes, as others have said the three checklists don't say the same thing.

Well perhaps - yes any no. I would say that the base data could be the same (and hopefully eventually will be) and that each party then embellishes this core data. I have no problem with that, but it would make sense the 'authorities' to then export core data from the base AviList and add their own additional fields. So for Ebird, this could be groups and for BirdLife it could be conservation status. We don't know what format the AviList will take, but if it can be downloaded as an Excel spreadsheet, there is no issue with including all defined fields - if it is too much for the average user, we can always delete a few columns - I do this already.

It will be interesting with BirdLife as they are accountable for defining and updating Conservation Status. As this varies from year to year, it presumably will not form part of the AviList, unless it can be built into the annual update cycle - and somehow this discharges the obligation of BirdLife. But as they only define conservation status for species, I can't see that there will be a need to publish a full taxonomy to subspecies level (as they do now).

 

[johnallcock]

A last question, and off track a bit from the thread, but do you find the EBird groups easy to use, especially when overseas and your knowledge of regional subspecies may not be perfect. I find the groups a bit challenging - a strange mix of names sometimes using descriptive text (e.g. Black-faced Dacnis [Yellow-tufted], sometimes using regional descriptions (e.g. Great Spotted Woodpecker [Canarian]) and sometimes referring to the subspecies Latin name (e.g. Golden Tanager [arthus]). My wife sometimes asks me should I be submitting this as such and such a group, but even when I know and positively identify the subspecies, I can find it difficult to correctly align with the given EBird name - who knows off the top of their head that alboides is Hodgson's White Wagtail... I think Himalayan White Wagtail seems more intuitive and has been used by others... and field guides are not much help... most good ones include Latin names of distinct subspecies, but I have never seen a field reference that includes the Ebird group names (but perhaps they should).

And then we have the problem with missing groups. The late Martin Garner provided us guidance on how to identify Northern Great Spotted Woodpeckers (subspecies major), but there does not appear to be a way with Ebird groups to record an scarce Northern bird as opposed to a bog standard British one.

One thing I'll say here is - if in doubt, don't record the subspecies group. Don't guess which subspecies is correct. Recording the species level is fine and still provides data on that species. You don't have to enter something as a subspecies just because the option is available. But if you are sure of the subspecies, this data is very useful, especially where multiple subspecies co-occur geographically.

One of the problems I sometimes have as an eBird reviewer is people entering a common species but using a subspecies group that doesn't occur here. This is fine if they understand that they're reporting something unusual and provide details to back up the subspecies ID, but this is often not the case. This wastes my time following up on the record, and it means that the record is treated as unproven in the system unless they change the subspecies.

For the example you give with Great Spotted Woodpecker (or similar examples if you have them), it might be worth contacting eBird, either directly or through your local reviewer. If there is evidence that a group is distinct and can be identified reliably in the field they may be willing to add the group.

 

[Jon.Bryant]

Interesting... I had heard that they had decided not to do another update.

Looking at there website, looks like I was wrong. They suggest that they are now looking to publish an online version of their avian checklist.

They mention providing the checklist as free issue, but also mention subscriptions (and of course you can donate on their website). They mention an Excel version, but then also say '“Publishing on-line” is not a valid action in the eyes of the International Commission for Zoological Nomenclature because the html format is not protected. Where needed we will use e-publication with task-appropriate PDFs that are valid.' I only seem to be able to find on their website an online version of the 2014 checklist that cannot be downloaded and needs to be viewed family by family.

As I said previously, I think most parties have now settled on other taxonomies, and a new version of H&M (possibly in a difficult format to use as a database), will mean uptake will be limited.

 

[Jon.Bryant]

Jon - Following up Paul's response to your query about local recorders accessing eBird records, some do it very well, good examples being Cornwall, Lincolnshire, Norfolk and Yorkshire. My name is regularly listed in the Norfolk report contributors despite me only having used eBird for Norfolk birding for several years. Lincolnshire acknowledge all eBird contributors in their list, with a code. The data handling experts in some counties have very kindly offered to share their methods with others.

In the case of Costa Rica, they are. Almost all members of the checklist committee for the country are also eBird reviewers, so anything new for the country from eBird gets noticed and added to the official country list.

This is interesting, but are you both saying that reviewers who are sent 'unusual' records, merely pass these on to local recorders? Steve, you mention your name popping up in the Norfolk report 'automatically', but you only get your name against a scarce or rare species, so your route into the reports could still be through reviewer access to your data.

I think that with the BTOs BirdTrack it is different and that county recorders are granted a kind of superuser access, so that can pull all records for their area.