Line of action:

Status:

Execution date:

The general objective of the DeepFish 2 project, based on the artificial vision prototype for the identification and sizing of specimens in fish markets developed in DeepFish, is to advance the capabilities of the system, expanding both the number of species of target fish, as well as molluscs (octopus, cuttlefish and squid), covering a greater casuistry in types of fish markets, introducing traceability aspects through the incorporation of geo-referenced information and developing a module for the analysis and exploitation of the data provided by DeepFish. It is also intended to constitute a forum for collaboration between researchers interested in the recognition and sizing of species through artificial vision.

Specifically, the specific objectives defined for the project are:

1. Expansion and dissemination of the dataset.
2. Adaptation of the vision system to new fish markets.
3. Adaptation of the deep network architecture.
4. Development of the data exploitation module.
5. Implementation of the vision system in new fish markets of the Natura 2000 Network.
6. Collaborative forum on recognition and sizing of marine species using artificial vision.
7. Disseminate the results of the project.

The results obtained with the execution of the project are the following:

On the one hand, the dataset of target species, tagged with metadata on species typology and size, has been expanded, incorporating 14 new species to reach a total of 32, from the fish markets of “El Campello”, Altea, Torrevieja and Moraira. The dataset has been published in the open scientific repository ZENODO. An explanatory video has been produced on the dataset adaptation to new standard fish markets.

On the other hand, the prototype of the vision system developed in the first phase of the Deepfish project for “El Campello” has been started and has been improved for its implementation and autonomous operation in the Lonja del Campello. This video explains the system implemented in the fish market. Likewise, the recognition and image system has been adapted in a new wholesale market (Altea) , also capturing images in another new wholesale market (Torrevieja) and in a new retailer (Moraira). In the “El Campello” fish market, the vision system has been implemented autonomously and in the case of Altea, it has been improved and adapted for testing in the wholesale market.

In relation to the hardware and software infrastructure of the trained deep network model , the problem of species identification through the YOLACT++ network has been addressed and a new “Key fishes” strategy has been proposed. In relation to the problem of carving, the use of regression and calibration methods has been deepened. In addition, a GIS website has been implemented that allows fishing intensity data to be visualised on a map and a dashboard has been developed for the statistical analysis of fisheries data and a report on parameters of biological interest based on this tool.

An expert forum on the recognition and sizing of marine species using artificial vision has been created in which 19 researchers from 8 different institutions have participated (5 of them participating in the Pleamar Program) and, finally, dissemination actions have been developed, with participation in congresses (IJCNN, SOCO, SARTECO), forums (ForoPesca) and days for the presentation of results (monitoring and results days).