Description

The FOTOPEIX and FOTOPEIX 2.0 projects, beneficiaries of previous calls of the Pleamar Programme, have made it possible to develop the artificial intelligence tools necessary to automatically generate data on the size of up to four species from images of fish boxes. Both projects have not only led to a methodological change, but have also made it possible to strengthen a close collaboration with the company that markets the catches of the Mallorca fleet, OPMALLORCAMAR, establishing a solid channel of collaboration that allows the two-way transmission of knowledge between the fishing and scientific sectors.

The RETORNO project aims to close the conceptual and practical framework that began with the granting of the FOTOPEIX projects and to provide the institutions and companies involved with useful information both for the monitoring and evaluation of the populations and for possible internal improvements in the first sale process.

The RETORNO project focuses on the case of size data only as a proof of concept, i.e. to verify that the underlying concept or theory is likely to be exploited in a useful way. In essence, RETORNO aims to develop the basic core of a comprehensive system that can grow and evolve in the future. The fundamental objective of the integral RETORNO system, modular and dynamic, is to structure and automate the flow of information between the observations and the information demanded by the end user, necessary to solve each specific problem. To channel the flow of information, we have identified four phases:

1. Data collection: a quality control protocol for the taking of images will be articulated, together with periodic sampling at the fish market to validate the size estimates
2. Processing: application of the deep learning tools developed in the FOTOPEIX projects for the extraction of sizes.
3. Generation of new information: a long-term time series will be maintained, with a weekly cadence, of the average size of each of the species considered. Likewise, a short-term predictive model (one week ahead) will be implemented. In this second case, it is intended to demonstrate that one of the most relevant potentialities of RETORNO is the iterative and automated updating of the models as time goes by. Finally, a Bayesian Belief Network will be implemented for one of the species considered (Coryphaena hippurus). This type of model allows the relationship between time series of landings, sizes, prices, production costs and other economic variables, together with environmental variables.
4. Storage on three levels: images (all images of the boxes of fish sold each day), size data (size of each fish measured, with box identifier, day, species…) and output generated in the third phase (models and the weekly time series). The repositories in which the information will be stored will be those of the research centre (IMEDEA) and access will be restricted to project participants. This phase is of vital importance, as it will allow the information to be reviewed retrospectively.

A size bulletin will be published, which will be distributed among the fishermen, agreeing on the final format and exact content with OPEMALLORCAMAR. As a complementary activity, meetings will also be organised with other fish markets and fishermen’s organisations from outside Mallorca, in which the advantages of a comprehensive system such as the one implemented in RETORNO will be shown. The integral system will be structured in R language, emulating a recent proposal (White et. to the , 2019). Likewise, the advantages and disadvantages of alternative languages will be explored.

An example of information flow in the case of fish sizes would start with automatic imaging (Phase 1) whenever there is a sale at the fish market. Each image will be evaluated and, if it contains any specimen of any of the four species considered, the size of those individuals will be extracted (Phase 2). Each day the sizes obtained will be stored together with a box, species and day identifier (Phase 4) to maintain a long-term time series. These data will be automatically processed once a week to update the average size time series for each species and the expected size distribution prediction for the following week (Phase 3). Likewise, an automatic quality control will be carried out, comparing the estimates and observations of each week. The predictive model that will be implemented as a proof of concept will be a first-order autoregressive process, but as many alternative models as appropriate can be integrated into this module. Likewise, the quality control module will allow you to compare the history of the predictions of the different models. In parallel, for models involving predictive variables, an automatic process will be articulated to incorporate new values of these variables.

See the project