Deep learning identifcation for citizen science surveillance of tiger mosquitoes

Global monitoring of disease vectors is undoubtedly becoming an urgent need as the human population rises and becomes increasingly mobile, international commercial exchanges increase, and climate change expands the habitats of many vector species.

Traditional surveillance of mosquitoes, vectors of many diseases, relies on catches, which requires regular manual inspection and reporting, and dedicated personnel, making large-scale monitoring difficult and expensive. New approaches are solving the problem of scalability by relying on smartphones and the Internet to enable novel community-based and digital observatories, where people can upload pictures of mosquitoes whenever they encounter them. An example is the Mosquito Alert citizen science system, which includes a dedicated mobile phone app through which geotagged images are collected.

This system provides a viable option for monitoring the spread of various mosquito species across the globe, although it is partly limited by the quality of the citizen scientists’ photos. To make the system useful for public health agencies, and to give feedback to the volunteering citizens, the submitted images are inspected and labeled by entomology experts. Although citizen-based data collection can greatly broaden disease-vector monitoring scales, manual inspection of each image is not an easily scalable option in the long run, and the system could be improved through automation.

Based on Mosquito Alert’s curated database of expert-validated mosquito photos, we trained a deep learning model to find tiger mosquitoes (Aedes albopictus), a species that is responsible for spreading chikungunya, dengue, and Zika among other diseases.

The highly accurate 0.96 area under the receiver operating characteristic curve score promises not only a helpful pre-selector for the expert validation process but also an automated classifier giving quick feedback to the app participants, which may help to keep them motivated.

In the paper, we also explored the possibilities of using the model to improve future data collection quality as a feedback loop.

Artificial intelligence identifies the tiger mosquito from the photos received in the Mosquito Alert app

Posted on by ssn_admin

Researchers from Mosquito Alert (who belong to CEAB-CSIC, CREAF and UPF) together with researchers from the University of Budapest have shown that an artificial intelligence algorithm is capable of recognizing the tiger mosquito (Aedes albopictus) in the photos sent by Mosquito Alert users. The results of the study published in Scientific Reports have been obtained by applying deep learning technology or deep learning, an aspect of artificial intelligence that seeks to emulate the way of learning of humans and that has previously been used in the health field to interpret medical images (X-rays of patients with COVID19 to detect pneumonia, or facial features to detect heart disease, among others). Deep learning needs a lot of training data for the machine to learn. In the case of the Mosquito Alert app, these images have been sent by the public and labeled by the project experts as “tiger mosquito” or “no tiger mosquito” for years. Specifically, the study used 7,168 classified photographs of mosquitoes that the project participants had sent between 2015 and 2019. After training, the algorithm has been able to correctly classify 96% of the photographs of this insect.

“The initial idea is to get the machine to classify the simplest photos, and leave the task of identifying the most problematic images that require consensus to the experts. As the artificial system learns from the classifications of the experts, we will be able to expand the range of automatically cataloged species”

John Palmer, UPF researcher and co-director of Mosquito Alert

More predictability

This milestone can mark a before and after in the surveillance and monitoring of the tiger mosquito and other mosquitoes capable of transmitting diseases. “We are training a social immune system against these mosquitoes. The faster the threat is detected, the faster it can be acted upon”, comments Frederic Bartumeus, co-director of Mosquito Alert and ICREA researcher at CEAB-CSIC and CREAF. On the one hand, the citizen science of Mosquito Alert allows anyone to be part of this new social immune system and contribute a massive number of photos of mosquitoes, on the other, artificial intelligence allows, to accelerate the classification process of the received photos and thus help public health experts make better and faster decisions about mosquito management.

Figure 1. Top row: correctly classified images, where the model was confident, thus assigned close to 1 probability to the correct category. Middle row: examples from the images which were mistakenly predicted by the CNN model. Bottom row: selected examples where the entomology expert could not tell if the presented mosquito is a tiger mosquito or not. The images were resized to have the same aspect ratio for visualization purposes.

“In times of greatest need, such as in the months of greatest mosquito activity or in a context of epidemiological crisis, artificial intelligence can help us so that the system can absorb a greater amount of information, controlling its quality at all times, which it is key if the data is to be used for decision-making in public health”

Frederic Bartumeus

Automating saves lives

The presence of the tiger mosquito in Spain poses a threat to public health. Millions of people are affected by its presence and are exposed to the risk of transmitting diseases such as dengue or chikungunya. In Europe, the tiger mosquito has been implicated every year since 2007 in small locally transmitted outbreaks of these viral diseases for which no vaccines are available. The only preventive measure is to control the mosquitoes that transmit them. Assessing the risk and the necessary action measures to mitigate it requires having accurate information on tiger mosquito populations, a costly and laborious task that requires manual placement and inspection of traps and their subsequent analysis in the laboratory where the insects are identified. A methodology that is not feasible to cover large geographic areas.

Mosquito Alert’s citizen science methods, which allow anyone to report the presence of a mosquito through a mobile application available on Android and iOS, is an alternative that makes it easy to cover large geographic areas throughout the mosquito season. Since 2015, the initiative receives thousands of photographs every year that help estimate the abundance of mosquitoes. However, this large volume of photographs continues to be classified by visual examination by expert entomologists, a task that requires time and years of experience. Integrating artificial intelligence into this process can speed up classification and thus develop near-real-time hazard maps that improve tiger mosquito management.

The study is part of the Versatile Emerging infectious disease Observatory (VEO) project coordinated by Marion Koopmans from the Erasmus Universitair Medisch Centrum (EMC) Rotterdam, The Netherlands,, and funded by the European Commission’s Horizon2020.