Merging statistics and geospatial information, 2017 projects - the Netherlands

Monitoring spatial sustainable development: semi-automated analysis of satellite and aerial images for energy transition and sustainability indicators; 2017 project; final report 22 October 2019

This article forms part of Eurostat’s statistical report on the Integration of statistical and geospatial information.

Full article

Problem

The project was devised to answer several questions.

What is the usability of different types of aerial and satellite images for detecting solar panels and what is the minimum required resolution?
Which method is best suited for detecting solar panels?
Is it possible to develop a harmonised method across European Union (EU) Member States?

Objectives

The aim of the project was to evaluate whether deep learning models could be developed that worked across different EU Member States to answer the questions listed above.

Method

Two remote sensing data sources were considered: aerial images and satellite images. It was soon found that the resolution of the satellite images (available from the Pleiades and Superview satellites) was insufficient. The project therefore focused on aerial images available for the province of Limburg in the Netherlands and the state of North Rhine-Westphalia in Germany.

Software was developed to pre-process aerial images, cut them down to an appropriate size and label them. Concerning size and resolution, the first approach used DOP20 images (with a scale of 1:2 000) and with 75 x 75 pixels. Due to high shares of false positives and false negatives, this was changed to DOP10 images (with a scale of 1:1 000) and either 200 x 200 pixels or 330 x 330 pixels (depending on the region).

During the project, several deep learning algorithms were evaluated to detect solar panels in remote sensing data. Two types of deep learning models were evaluated: classification models and object detection models.

Classification models give a coarse overview whether a certain area contains solar panels or not.
Object detection models can distinguish the location of individual solar panels and can help with detailed statistics about the number of solar panels, their area, and their expected yield.

For image classification, datasets are labelled as to whether they have positive images (in this case, containing solar panels) or negative images (in this case, without solar panels). For object annotation, each image is labelled with either bounding boxes or bounding polygons for each of the solar panels.

A tool for image classification was developed within this project to help users quickly distinguish images with solar panels, those without and those for which this was not known. This was used to correct an initial pre-annotation done using a neural network. Multiple users checked the annotation of images used for training, in order to measure the uncertainty of annotations.
The VIA annotation tool was used for the annotation for object detection. This was used to produce information on the central coordinates of bounding boxes as well as their height and width, with multiple bounding boxes possible within individual images. Object detection was annotated by single users.

Both models were trained by a process called supervised learning in which a dataset contains the images to train the algorithm and labels for each image containing the so-called ground truth.

For the classification models, this entailed labelling each image as either containing or not containing solar panels. 80 000 images were labelled for this model.
For the object-detection algorithm, this entailed drawing the boundaries of each individual solar panel and was therefore more labour intensive. This model was only evaluated for North Rhine-Westphalia for a relatively small dataset (5 000 images).

A cross-site evaluation approach was used for the evaluation of the models. The models were trained in one geographical area and then evaluated on a different geographical area, previously unseen by the algorithm. The cross-site evaluation for the classification models was carried out twice.

One evaluation was on the same aerial image in a different geographical area in the same country. This provided information on whether or not the model was resilient to differences in geography, building style, and urbanisation, without issues related to different image colour distributions caused by normalisation and camera equipment.
A second evaluation was on a different aerial image in a different country. This provided information on whether or not the model was resilient to differences in different image colour distributions as well as differences in building style, architecture and urban planning.

Results

The validation performance and generalisation of the convolutional neural networks was higher using DOP10 datasets than with lower resolution datasets. By providing more detail, the annotation of images was easier. However, better performance is expected with lower resolution datasets. Furthermore, information gained from training on high resolution datasets might be transferrable to lower resolution datasets.

While the deep learning models were able to detect solar panels successfully, false detection remained a problem. For example, cars, greenhouses, and rooftop windows were detected as solar panels. Performance differed between urban and more rural areas. Performance also decreased dramatically when evaluated in different cities within a country and in different countries; training a model that performs reliably across different EU Member States would therefore appear challenging.

To improve network performance, more data can be added or data added in a better way. Nevertheless, work needs to be done to see if the false positive rate can be improved sufficiently to provide a fully automatic solution. In particular, the small number of images with solar panels (in comparison with the number of images without solar panels), requires a model with very high classification accuracy.

Nevertheless, the method applied was suitable to automate a part of the task of solar panel detection.

Large datasets were pre-processed and it was possible to identify a) large numbers of possible solar panel candidates for checking by human observers and b) a large number of images as containing no solar panels. As such, the need for human intervention could be reduced to checking aerial images for false positives.
The models detected quite a high share of solar panels that were not present in current solar panel registers and therefore can already be used to help reduced manual efforts in checking these registers.