Fish Detection AI, Optic and Sonar-trained Object Detection Models

Description

The Fish Detection AI project aims to improve the efficiency of fish monitoring around marine energy facilities to comply with regulatory requirements. Despite advancements in computer vision, there is limited focus on sonar images, identifying small fish with unlabeled data, and methods for underwater fish monitoring for marine energy. A YOLO (You Only Look Once) computer vision model was developed using the Eyesea dataset (optical) and sonar images from Alaska Fish and Games to identify fish in underwater environments. Supervised methods were used within YOLO to detect fish based on training using labeled data of fish. These trained models were then applied to different unseen datasets, aiming to reduce the need for labeling datasets and training new models for various locations. Additionally, hyper-image analysis and various image preprocessing methods were explored to enhance fish detection. In this research we achieved: 1. Enhanced YOLO Performance, as compared to a published article (Xu, Matzner 2018) using earlier yolo versions for fish object identification. Specifically, we achieved a best mean Average Precision (mAP) of 0.68 on the Eyesea optical dataset using YOLO v8 (medium-sized model), surpassing previous YOLO v3 benchmarks from that previous article publication. We further demonstrated up to 0.65 mAP on unseen sonar domains by leveraging a hyper-image approach (stacking consecutive frames), showing promising cross-domain adaptability. This submission of data includes: - The actual best-performing trained YOLO model neural network weights, which can be applied to do object detection (PyTorch files, .pt). These are found in the Yolo_models_downloaded zip file - Documentation file to explain the upload and the goals of each of the experiments 1-5, as detailed in the word document (named "Yolo_Object_Detection_How_To_Document.docx") - Coding files, namely 5 sub-folders of python, shell, and yaml files that were used to run the experiments 1-5, as well as a separate folder for yolo models. Each of these is found in their own zip file, named after each experiment - Sample data structures (sample1 and sample2, each with their own zip file) to show how the raw data should be structured after running our provided code on the raw downloaded data - link to the article that we were replicating (Xu, Matzner 2018) - link to the Yolo documentation site from the original creators of that model (ultralytics) - link to the downloadable EyeSea data set from PNNL (instructions on how to download and format the data in the right way to be able to replicate these experiments is found in the How To word document)

Resources

Name Format Description Link
0 The article that was used to compare the results from our experimentation, namely an article by Xu and Matzner from 2018, titled Underwater Fish Detection using Deep Learning for Water Power Applications. https://arxiv.org/pdf/1811.01494
21 The ultralytics website of documentation for Yolov8, which can be accessed by the public and has downloadable versions of several model version numbers, sizes, and datasets. https://docs.ultralytics.com/models/yolov8/
57 The files (shell, python scripts) that can be run to complete Experiment 5 of the project, namely the object detection done on sonar images https://mhkdr.openei.org/files/600/Exp5_client_deliverables.zip
57 The files (shell, python scripts) that can be run to complete Experiments 3 and 4 of the project, namely the pre-processing experimentation, and the single-source trained models https://mhkdr.openei.org/files/600/Exp3_and_Exp4_client_deliverable.zip
57 Example file directory structure for the PNNL EyeSea images, which should take effect after the user runs the data processing scripts after they have downloaded the raw data https://mhkdr.openei.org/files/600/data_div_subfolders_sample2.zip
57 Example file directory structure for the PNNL EyeSea images, which should take effect after the user runs the data processing scripts after they have downloaded the raw data https://mhkdr.openei.org/files/600/data_div_subfolders_sample1.zip
57 All the PyTorch (.pt) YOLO model weights, namely the trained best models from our experimentation, or downloaded models from the Ultralytics website which were the bases for our training https://mhkdr.openei.org/files/600/Yolo_models_downloaded.zip
21 Caltech Fish Counting Domain Adaptive Object Detection (CFC-DAOD) dataset which is available via their GitHub https://github.com/visipedia/caltech-fish-counting/tree/main/CFC-DAOD
57 The files (shell, python scripts) that can be run to complete Experiment 1 of the project, namely the object detection done using an out-of-the-box YOLO model not trained on any fish-specific items https://mhkdr.openei.org/files/600/Exp1_uploads_for_client.zip
10 The How To document that details what is fully in the upload, what each file does, how to understand the data, what to run to get certain results, and much more. https://mhkdr.openei.org/files/600/20250310_Yolo_object_detection_how_tos_for_client_upload.docx
21 PNNL website for the specific EyeSea dataset, with a button to click to download the dataset to a local computer. This data is large, 80GB, and contains both labels and images broken down into training and testing folders and subfolders based on source (i.e., ORPC Igiugig, Voith Hydro, Wells Dam). The date of original creation also varies based on image source, with dates from 6/25/2014, 7/5/2014, 7/19/2015, 7/22/2015, and 6/27/2017 https://data.pnnl.gov/group/nodes/dataset/12978
57 The files (shell, python scripts) that can be run to complete Experiment 2 of the project, namely the training and parameter experimentation for YOLO object detection algorithms done on optic images https://mhkdr.openei.org/files/600/Exp2_uploads_for_clients.zip

Tags

  • yolo-version-8
  • object-detection
  • small-fish-detection
  • sonar-trained-object-detection-models
  • cross-domain-adaptability
  • hydrokinetic
  • you-only-look-once-model
  • yolov8
  • mhk
  • shell-code
  • eyesea
  • hyper-image-approach
  • eyesea-optical-dataset
  • marine
  • fish-detection-ai
  • python
  • neural-networks
  • yaml-code
  • pytorch
  • yolo-model
  • power
  • code
  • ai
  • eyesea-dataset
  • yolo-performance
  • pytorch-code
  • energy

Topics

Categories