Line of action:

Status:

Execution date:

The specific objectives of the project, of a technical nature, are the following:
O1. Testing of new vision technologies and selection of the one that allows better results to be obtained in the identification
of O2. Redesign of the iObserver system for compaction/miniaturization.
O3. Adaptation and retraining of species recognition and quantification algorithms based on deep learning and neural networks
.
O4. Development of the sensors for a correct synchronization of the new system with the conveyor belt in the fishing parks.
O5. Calibration and tuning of the new system designed during tests carried out on board oceanographic vessels.
O6. Proof of the efficiency of the new iObserver 2.0 system on board fishing vessels.

In SICAPTOR 2.0, the iObserver system will be redesigned in order to minimize its size and increase its performance as a reference device for remote electronic monitoring of captures. To do this:

1) New commercially available vision devices that could replace the iObserver’s dot matrix camera will be tested: linear cameras and smart cameras.

2) A new, more compact system will be designed, both at the level of vision system and lighting, as adaptable as possible to the different configurations
of fishing parks and/or types of fishing in order to maximize its implementation.

3) The sensors will be readapted for automatic image capture to avoid image overlapping and unnecessary processing as well as for the autonomous operation of the iObserver 2.0.

4) The algorithms developed in SICAPTOR will be retrained to adapt them to the new image formats generated by the different devices.

The results of the SICAPTOR 2.0 project conclude that, despite the progress made since its inception, there are still significant challenges to ensure full compliance with the objectives of the CFP. Among them, the availability and quality of fisheries data should be improved, and protocols and standards should be necessary to ensure that these data are systematically shared among all relevant entities, including fisheries scientists. Digitalization and advanced tools applied to fisheries, together with information from different sensors, have enormous potential to optimize fishing operations and improve our ability to collect and analyze data and ultimately support sustainable management of marine resources. Within this framework, the objective of SICAPTOR 2.0 has been to carry out a redesign exercise of the EM iObserver system, which will bring it closer to the market and generalize its use by the Spanish fleet in the short-medium term:

Two devices have been developed from the iObserver system, with a very significant reduction in their dimensions and weight, making their installation and use easier:

2. a) iObserver 2.0 Linear, based on the new linear camera.
3. b) iObserver 2.0 Matrix, based on the matrix cameras reconfigured for a hybrid work between linear and matrix capture.

In both cases, the location of the processing and communication hardware is located on the bridge (they can be used for other tasks, since they do not require specialized hardware, which implies lower costs), leaving the camera and the lighting system on the fishing park belt, whose number of spotlights has been reduced from four to two, but multiplying the luminosity by eight.

Improvements have been made to the sensors for image capture, avoiding overestimations/duplications in the quantification of the capture through two solutions:

A system of physical sensors (encoders) that allow perfect synchronization between the movement of the tape and the taking of images of the iObserver2.0.

A software system that, based on a flow analysis algorithm created ad hoc, determines the progress of the tape and reconstructs the images by consecutively capturing strips transverse to the tape, without the need to use physical sensors (encoders) on the tape, being very useful for cases with complex installations.

New algorithms for identifying and quantifying the capture, based on object tracking, known as MOTS (Multiple Object Tracking and Segmentation), were developed, which are much more powerful, because they allow differentiating more than twice as many species as in SICAPTOR (31 instead of 15); they work interchangeably with the iObserver and iObservers 2.0 images, and the generated test set has a greater number of complex images with multiple fish and overlap. All this without losing capacity in terms of accuracy and total sensitivity, which reach 96% and 92%, respectively.

Products obtained in the project:

• Reports:
  • New design of the iObserver2.0, including detailed comparisons of the different types of cameras used, their advantages and disadvantages, the lighting requirements and redesign work of this system.
  • Adaptation and re-training of the software developed for the matrix camera in SICAPTOR, including detailed comparisons with the initial version in terms of percentage of success in the identification and quantification of species, number of individuals used in the training of each species, automation of the equipment, etc.
  • Results of the use of iObserver2.0 on board.
  • Feedback obtained from the skippers of the different segments of the trawl fleet analysed on the potential for use, in a fishing environment, of this new device developed based on its technical characteristics and with a view to its real improvement and implementation.
• Interview with the project coordinator, Luis Taboada Antelo, available on the Pleamar website.
• Video on the implementation and use of iObserver2.0 on board oceanographic vessels, available on the Youtube channel.
• Forum on Artificial Intelligence Applied to the Fishing Sector held in October 2021.
• Two articles in high-impact scientific journals (Q1):
  • On the use of deep learning for fish species recognition and quantification on board fishing vessels.In the journal Marine Policy (Elsevier Publishing)
  • Deep Learning techniques applied to the photo identification of fish individuals of undulate skate (Raja undulata) within a Few-Shot Learning context, in the scientific journal Methods in Ecology and Evolution (Wiley Publishing)
• Four reports in industry magazines:
  • Smart Fishing (Mar Magazine – October 2020)
  • New technologies and interdisciplinarity set the course for fisheries science (Fisheries Industries Journal – April 2021)
  • Data to better manage (Mar Magazine – February 2022)
  • Artificial intelligence arrives in fishing (Fishing Industries Magazine – February 2022).
• Celebration of two days of presentation of results. One at the headquarters of the General Secretariat of Fisheries in Madrid and the other in Vigo.
• Dissemination material:
  • Posters for the conference
  • 250 units of the initial project presentation brochure
  • 250 units of the final earnings presentation prospectus
  • 250 notebooks and 250 pens