As birdwatchers, birders and fans of the popular board game Wingspan know, feathered friends are found worldwide in a variety of different habitats, from frosty Antarctic penguins to desert-dwelling cactus wrens to Midwest favorite cardinals. BirdSAT, a new tool for classification and ecological mapping of bird species, enables the creation of species distribution maps across any geographic region to provide important information to assist ecologists who monitor and protect endangered species.
Nathan Jacobs, professor of computer science & engineering in the McKelvey School of Engineering at Washington University in St. Louis, leads the research team that developed BirdSAT. The framework combines ground-level imagery of birds, location and time metadata, and corresponding satellite imagery to classify different species of birds and map their habitation anywhere in the world. First author Srikumar Sastry, a doctoral student in Jacobs’ lab, presented BirdSAT Jan. 6 at the Winter Conference on Applications of Computer Vision in Waikoloa, Hawaii.
“The two tasks – species classification and distribution mapping – are distinct and often challenging for computer models because we need to make use of subtle differences to categorize species into fine-grained classes,” Sastry said. “With BirdSAT, we’ve developed a unified framework leveraging different machine learning techniques to solve both tasks and accurately classify and map bird species globally. Beyond birds, our models are general enough to be easily extended to any species of interest.”
The framework relies on a cutting-edge deep learning architecture to enable fine-grained visual classification (FGVC) and to retrieve accurate information across data types (e.g. between ground-level bird images and numeric metadata). Sastry and Jacobs found that BirdSAT achieved state-of-the-art performance on FGVC for a challenging birds dataset and proved to be remarkably versatile and generalizable when transferred to other datasets.
“By seamlessly blending ground-level imagery with metadata and satellite data, BirdSAT allows us to create species distribution maps for any geographic region and for any species,” Jacobs said. “This not only enhances our understanding of bird species but also paves the way for the development of exciting new tools for conservationists, ecologists and policymakers in managing and preserving biodiversity.”
Sastry S, Khanal S, Dhakal A, Huang D, and Jacobs N. BirdSAT: Cross-view contrastive masked autoencoders for bird species classification and mapping. Winter Conference on Applications of Computer Vision (WACV), Jan. 4-8, 2024. https://arxiv.org/pdf/2310.19168.pdf