Machine learning for fishes

Guest Author: Sarah Hoffmann

Editor: Patrick Cooney

Developments in technology have been a game changer for wildlife and fisheries research, conservation, and management. Specifically, advances in the utility and accessibility of unmanned aerial systems (drones) have made great strides in data collection by:

  1. Increasing the efficiency of data collection – a single 20-minute drone flight can generate tens of thousands of high-resolution images. Multispectral imaging adds another layer of complexity by including “bands” (or sensed data) in addition to the standard Red-Green-Blue bands captured by most cameras.
  2. Offering a lower-cost data collection alternative – drones range in price but entry points can be less than $1,000USD. The geographical range and detail that can be covered by drone would easily consume a budget in piloted LiDAR or days to weeks of on-the-ground sampling labor.
  3. Generating a standardized, permanent data source – high-resolution imagery is a standardized, translatable data source than can be mined for years to come. The collection and curation of imagery provides a platform for cooperative research and long-term status and trend monitoring.
  4. Providing safe access to remote, complex environments –technological advances such as telemetry and remote sensing offer a window into the areas previously inaccessible to research. Piloted surveys have been an attractive method for covering large spatial scale, but are also noted to account for 66% of job-related mortality for wildlife workers (Sasse, 2003). Drones offer a safer way to access remote areas while producing a high-quality data source.
Left: Drone generated red-green-blue (RBG) composite image- these are the standard bands captured by most cameras. Right: Near-infrared (NIR) imagery from the multispectral camera is capture for additional diagnostic data such as separating water from land and determining vegetation indices.

Despite their proven utility, safety, and cost effectiveness, drones have yet to “take-off” broadly in the field of wildlife population monitoring. Data handling, storage, and processing are commonly cited as barriers to entry for the use of remotely sensed data in wildlife science.  

Our goal was to develop tools that make the implementation of drones (and the associated imagery processing) easy, effective, and eventually, automated. The majority of our work occurs in the Columbia River Basin, focusing on estimating carrying capacity for salmonids. We relied heavily on historic habitat data from the Columbia Habitat Monitoring Program (CHaMP), to establish quantitative relationships between habitat characteristics and fish abundance, leveraged to estimate life-stage specific carrying capacity. A set of globally available attributes was identified to extrapolate continuous carrying capacity estimates throughout a reach and to a large spatial scale.

Since the initial development of this model, we have determined strategies to collect the necessary habitat data from drone imagery. We trained a random forest pixel classifier to automatically categorize landscape features such as bare earth, water, woody debris, and vegetation.

Sample reach that has been digitized using a random forest pixel classifier.

Similarly, a random forest pixel classifier was developed to estimate the amount of woody debris at the channel unit-reach scale.

Sample site with woody debris classified (left-red) using a random forest pixel classifier. The automation of imagery processing allows for near real-evaluations of habitat quality and quantity.

We are now exploring new computer vision tools, such as object-based classifiers, to improve the accuracy and efficacy of automated imagery processing. In a parallel development project (in a much saltier locale), we have partnered with the Marine Megafauna Foundation: Florida Manta Project to have train a convolutional neural network model for the automated detection of Manta rays in coastal Florida. Mantas are easily spotted thanks to their unique body shape and dark coloration, making drone surveys an ideal option.

A blacktip shark (blue box) and a juvenile manta ray (yellow box) are tracked by an object-based classifier developed in a convolutional neural network model.

We continue to add training layers to this model to pull out different taxa commonly sighted along the Florida coast such as sharks, sea turtles, manatees, and dolphins. Many of these species are declining in population and commonly interact with humans, putting them at further risk. A robust model that surveys heavily used recreation areas in near-real time can reduce negative anthropogenic interactions and provide important habitat use data for managers.

With these projects, we hope to make drones an easier and more effective tool for wildlife and fisheries management by continuing the advancement of wildlife technology.

For more information on our ongoing work, check out and

About the Guest Author

Sarah Hoffmann is a scientist in the Applied Biological Services division of Biomark, Inc. She specialized in the physiology and biomechanics of marine fishes at Florida Atlantic University, with a focus on the functional ecomorphology of sharks. At Biomark, she continues to pursue her interests in applied anatomy by working to develop data collection and analysis tools for wildlife researchers. The advancement of wildlife technology, particularly telemetry and remote sensing, gives researchers an opportunity to explore previously inaccessible ecosystems. This push to explore the unknown fuels her drive to find creative solutions to the many unforeseeable pitfalls of field work and wildlife research.

Please leave a thought provoking reply. We reserve the right to remove comments deemed inappropriate.

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.