Automatic visual detection of fish in Recirculated Aquaculture Systems using the Segment Anything Model

Abstract

Detecting individual animals is an important step for building monitoring tools for animal based welfare indicators. Latest advances in foundation models (FM) have shown promising performance in zero-shot segmentation over various domains. Here, we show that an FM Segment Anything Model (SAM) can be used to find rainbow trout (Oncorhynchus mykiss) and Atlantic salmon (Salmo salar) in a real-life recirculated aquaculture system (RAS) commercial production environment with a stocking density of 32–40kg/m3, without the need for costly training data. We evaluate two prompting methods over a benchmark containing 88 manually verified images and thousands of masks, comparing these hand-annotated prompts with fully automatic segmentation. We also propose a fast annotation workflow utilizing a lightweight classifier to refine the set of output masks, improving the proportion of good-quality whole fish or head masks for automatic detection from 60.5% to 77.7%, and for hand-annotated prompts from 75.6% to 85.2%.