Aarhus University Seal / Aarhus Universitets segl

New monitoring to decode and recognise images

A picture says more than 1,000 words, and PhD student Oskar Liset Pryds Hansen is trying to pull words out of pictures. Monitoring images will help a computer learn to recognise images such as insects, spiders and beetles, and in the long term, this can be used to monitor nature.

Photo: Morten DD Hansen

A pitfall trap dug into the ground helps monitor arthropods in the Mols Laboratory areas in Mols Bjerge National Park as part of the Rewilding Mols project. Although the pit is a trap for the animals, it also collects data for a team of researchers from the Museum of Natural History Aarhus and Aarhus University. The two parties are interested in the insects that fall into the trap, because this tells them something about the wildlife and development of nature in the area.

The method of using a pitfall trap has been used for many years and it is commonly applied when monitoring insects and other arthropods. The pit is regularly emptied of the animals that have been trapped. After this, the contents are separated, counted and identified in the laboratory. The new technology may make it possible to reduce this resource-intensive, manual work.

- It’s important to develop methods that can streamline and collect data at a level of information we have not been able to achieve previously, explains industrial PhD student Oskar Liset Pryds Hansen, who will be using camera surveillance and machine learning to decode and recognise wildlife from pictures.

More monitoring than before

A camera will not only be able to streamline nature monitoring. It will also be able to collect more data than is possible using pitfall traps, and this may be of great value to research," says Oskar Liset Pryds Hansen.

- In contrast to a trap that is emptied once a week, using a camera you can suddenly see things in a higher temporal resolution. You get more information about your insect or arthropod community, and about the activity taking place. The temporal resolution can be very high, because we can decide for ourselves how often the camera is to take a picture, he explains.

There’s nothing new in using photos for monitoring, and camera traps are already common. What is new is using a computer to identify and count arthropods directly on the photos from a camera trap. It's simply not been done before," explains one of the industrial PhD’s supervisors, Kent Olsen from the Museum of Natural History Aarhus.

- The industrial PhD project is innovative, and it has far-reaching potential. We currently spend enormous resources on sorting the contents of pitfall traps, so a successful PhD project will take us a long way and we can start asking new biological questions, he says.

Photo: Oskar Liset Pryds Hansen

Captures what other cameras can't see

Camera monitoring of mammals in nature is possible using sensors that register movement. When something moves in front of the camera, the camera takes a photo. In the world of insects, it is not possible to register movement in this way, so we have to use other methods.

- You can set up a sensor, but nothing will happen. Arthropods are too small, and they are not warmer than their surroundings. Our options for monitoring insects really are very limited, says the industrial PhD.

However, fragments of existing monitoring methods may pave the way for development. The industrial PhD is using known methods to recognise what is in the photographs, but instead of depending on a trigger function or a group of people with time to review the images, a computer is trained to extract the data. The machines simply learn to decode specific patterns for each individual animal.

- It's an important research project, and we think that we hit the bull’s eye when we started the project with the university and Oskar, says Kent Olsen, smiling.

Tests with 65,000 beetles

An image consists of pixels which, via specific values, form patterns in the image, and a computer can be coded and trained to read the patterns of various species of arthropods and thus recognise species, based on examples of images of species that it has previously been shown. The computer gives a value for how certain it is that a specific individual is a specific species. A high value means that the computer's network is very confident that an image is a specific species.

However, the training process requires that insects and other arthropods are pixel-coded, and this is a time-consuming process. In order to explore the challenges in recognising species this way, the industrial PhD tested the method on a previously known dataset at species level in a simplified version of the camera trap.

- This required that we select well-known animals from a museum collection, take pictures of the animals directly from above, and then train a computer to classify the pictures at species level. I photographed a museum collection of 65,000 beetles and scanned them into the computer. Then we cut out the pictures to mark all individuals and name the species. We could use the database to train a model that could start classifying ground beetles at species level. However, we also found that the method has limitations, for example the size of the species influences how well the computer recognises it, explains Oskar Liset Pryds Hansen.

Photo: Oskar Liset Pryds Hansen

Very topical research

- In the long term, the method could do much more than just monitor arthropods in Danish nature, and this is why the project is so important, explains Professor Jens-Christian Svenning, who is the head of the BIOCHANGE centre at the Department of Bioscience at Aarhus University and is the principal supervisor for Oskar Liset Pryds Hansen. A more extensive reference library with images of known species would enable the computer to decode and recognise more species on the monitoring photographs.

- Image recognition is a much more general method and it can be used in many other contexts. We’re trying to make this approach more practicable so that we can use it to get ecologically relevant data, and if we can get this to work, it could be used across a wider platform than has been possible so far. We’re helping to move a milestone, he explains.

And the possibility to push development forward through research is the part of the industrial PhD programme the museum appreciates, says Kent Olsen.

- The industrial PhD programme provides a very good opportunity to move forward with research, and it provides a fantastic opportunity to help shape a researcher. It's nice to be able to meet this obligation and help form the new researchers of tomorrow, he explains.

Oskar Liset Pryds Hansen’s industrial PhD runs from May 2017 to April 2020 in Mols Bjerge National Park, the site of the Museum of Natural History Aarhus' research area, Mols Laboratory. The industrial PhD is part of the Rewilding Mols project, which started in 2016.

Contact Jens-Christian Svenning for more knowledge about the project