Research Publication

Artificial intelligence for right whale photo identification: from data science competition to worldwide collaboration

By Christin Khan, Drew Blount, Jason Parham, Jason Holmberg, Philip Hamilton, Claire Charlton, Fredrik Christiansen, David Johnston, Will Rayment, Steve Dawson, Els Vermeulen, Victoria Rowntree, Karina Groch, J. Jacob Levenson & Robert Bogucki

Originally published in Mammalian Biology in June 2022



Photo identification is an important tool in the conservation management of endangered species, and recent developments in artificial intelligence are revolutionizing existing workflows to identify individual animals. In 2015, the National Oceanic and Atmospheric Administration hosted a Kaggle data science competition to automate the identification of endangered North Atlantic right whales (Eubalaena glacialis). The winning algorithms developed by were able to identify individuals with 87% accuracy using a series of convolutional neural networks to identify the region of interest, create standardized photographs of uniform size and orientation, and then identify the correct individual. Since that time, we have brought in many more collaborators as we moved from prototype to production. Leveraging the existing infrastructure by Wild Me, the developers of Flukebook, we have created a web-based platform that allows biologists with no machine learning expertise to utilize semi-automated photo identification of right whales. New models were generated on an updated dataset using the winning algorithms. Given the morphological similarity between the North Atlantic right whale and closely related southern right whale (Eubalaena australis), we expanded the system to incorporate the largest long-term photo identification catalogs around the world including the United States, Canada, Australia, South Africa, Argentina, Brazil, and New Zealand. The system is now fully operational with multi-feature matching for both North Atlantic right whales and southern right whales from aerial photos of their heads (Deepsense), lateral photos of their heads (Pose Invariant Embeddings), flukes (CurvRank v2), and peduncle scarring (HotSpotter). We hope to encourage researchers to embrace both broad data collaborations and artificial intelligence to increase our understanding of wild populations and aid conservation efforts.

Full Text

Affiliated Authors
  • Philip Hamilton

    Philip Hamilton, Senior Scientist, Kraus Marine Mammal Conservation Program, Anderson Cabot Center for Ocean Life

    Read bio

Research That Drives Action

Through pioneering conservation research and strategic partnerships, our team of 40 scientists at the Anderson Cabot Center for Ocean Life works to combat the unprecedented impacts on the ocean from climate change and other human activities.