iCalPez: Artificial intelligence applied to the automatic capping of fish species

Pleamar program

Description

The preservation of fishing resources and the maintenance and improvement of economic activity around them is of fundamental importance for our society. Scientific and governmental studies on the state of fishing stocks coincide, together with statistical data in our country, on the downward trend in the evolution of catches and, although fish prices have increased, the real incomes of fishermen continue to decrease. All this predicts the future disappearance of both fishing resources and the activity in their surroundings if adequate action is not taken.

To ensure the continuity of these resources and improve the efficiency of the fishing industry and the economy of coastal societies, the European Union has been introducing, over the last few years, a series of measures, management plans, fishing limitations and environmental requirements that require, among others, detailed real data on catches, to be able to correctly analyze the situation.

The size is a relevant piece of information about fish products and helps to market products with or without a registered trademark. This is also essential for its management and preservation as a marine biological resource.

In the case of the marketing of fishery products in the fish markets, there are about 40 species with size and freshness categories, according to European regulations, and more than 200 species present in the Spanish fish markets without commercial categories and which, on the other hand, do not have common marketing standards in the European Union. The positive effects of having fish products calibrated are very important: they allow and improve commercial transactions between operators located in different markets, make it possible to sell online in more detail, help to implement fishing activity management plans and reduce the marketing of fish below the regulatory minimum size.

The distribution and separation of consignments of fishery products, with an established size, is normally done in the fish markets, relying on the experience of the people (fishermen, operators), and the control of the regulations is carried out through inspections by administration personnel, who travel to the fish markets and carry out random sampling to produce statistics.
This project would help determine these sizes in a more efficient and accurate way.

There are already cases of recent successful applications of artificial intelligence in fishery resources. We can mention the automatic recognition of species on boats, the counting and measurement of fish, the detection of specimens in fish boxes or the measurement of freshness. These and many other applications of deep learning open up a whole range of possibilities in the monitoring, conservation, management and marketing of fisheries.

In this project we propose to develop an artificial intelligence system, based on computer vision, using “Deep Learning” methodologies to help classify fish species in fish markets at their first sale. This system aims to determine the size (size/weight ratio, size histogram) and alert to the presence of specimens below the minimum reference size. To carry it out, a collaboration network has been established with the fishermen’s guilds participating in the project (OP80 Punta del Moral, Gandía Fishermen’s Guild, Llançà Fishermen’s Guild), in order to carry out an implementation, focusing on species at risk of overexploitation and of commercial interest at the same time (hake, red mullet, white prawns, langoustines, red or pink prawns, lobsters, rock octopus, etc.).

See the project

Project

Line of action:

Marine ecosystems

Status:

Finalizado

Execution date:

2021

Objectives

The information on the size and distribution by size of the fish, in the first sale in the fishermen’s markets, is a very important piece of information both at a commercial level and in the management of marine biological resources. Currently, most fish markets in Spain estimate the size based on subjective observations.

We propose a project consisting of an artificial intelligence system to automatically calibrate fish specimens and extract data and size of the most overexploited species on the Mediterranean coast. To this end, this project will be implemented in the fishermen’s guilds of Llançà (Catalonia, GALP Costa Brava), Gandía (Valencian Community, GALP La Safor) and in the OP80 Punta del Moral de Ayamonte (Andalusia, GALP CostaLuz). The result will be of general utility for the guilds and the sustainability of marine biological resources.

Actions

A1. Analysis of species of interest and adjustment of the pilot plan

Together with the fishermen’s guilds and collaborating companies, the functions of the system will be determined, what results will be obtained and which species are of greatest interest.
The possibilities in terms of measurements obtained with computer vision will be explained in detail and a strategy for obtaining data and subsequent implementation of the prototype system in the chosen brotherhoods will be established.

We will work with 3 species and, if there is enough time, it will be extended to the species that the duration of the project allows.

A2. Creation of a species of interest database

Imaging of species of interest at first sale, along with weight, catch date, sale price, and other additional data as available
Definition and creation of the database.

A3. Development of computer vision algorithms

We will work with supervised vision systems, based on “Deep Learning” that need a database (images), with the objects of interest (fish) identified in order to train the neural network.

Manually segment and tag the image database
Neural network training and parameter refinement.
Post-processing to obtain the caliber (fish/kg, size).

The models we will use allow for evolutionary learning and for us to add images throughout the project to improve their performance.

A4. System Implementation

TEST implementation to validate its practical operation on computers and GPUs at the UPC.
OPERATIONAL implementation, in host-GPU in the cloud, connected to the cameras of each fish market and monitored.

A5. Analysis and presentation of the results

Preparation of informative material on the project and sources of funding.
Study of the results and their influence on sustainable marketing and management. Presentations, seminars and surveys of guilds, federations, companies and entities responsible for environmental management.

In short, a system is proposed that responds to the needs of the fishing sector and guarantees and makes its activities compatible with the good environmental status of the sea.

Results

In this project, it has been possible to verify that progress must be made in the automatic and objective determination of the size of fishery products and that this is possible thanks to artificial intelligence. The species with which we have worked show that it is an achievable objective that can result in benefits for the fishing and commercial sector, the management of marine biological resources and the environment.

Among the results obtained by iCALPEZ we can highlight:

Generation of a database of species of interest from images of the collaborating fish markets.
Development of computer vision algorithms.
Development of a software implementation on local GPU and its twin in the cloud.
Statistics on sales prices with and without automatic caliber, for the selected species, and also on the influence of size on the selling price.

Collaborators

Punta del Moral Fishermen's Producers Organization OP80 (Ayamonte), Gandía Fishermen's Guild and Llaní Fishermen's Guild