PROBA-V SR Challenge
10 October 2018 The latest on the ACT’s Kelvins competition website is a challenge for “citizen scientists” to apply the latest artificial intelligence and image processing techniques to upscale images acquired by the Earth-watching PROBA-V minisatellite.
Upscaling is a familiar concept from modern HD or 4K TVs, which perform image processing to enhance the appearance of low-resolution, standard definition footage when played on higher-resolution screens. That is a purely cosmetic processing however; what this new “super-resolution” competition is interested in is enhancing the resolution of Earth observation images by combining multiple lower-resolution images together, in order to extract extra information on detailed features of the target site.
“PROBA-V is a very interesting mission because it provides two types of imagery to researchers”, explains Dario Izzo of ESA’s Advanced Concepts Team (ACT), overseeing the competition. “The minisatellite’s wide-swath main vegetation instrument images 90% of the vegetated surface of Earth every day, at 300m spatial resolution. It also acquires a limited amount of imagery at 100m resolution. So the challenge we’re setting is to take nine or more of these 300m resolution images of a given view then combine them together using super-resolution techniques to enhance the quality beyond the capability of the original camera, to try and come as close as possible to the equivalent 100m resolution view.”
New large-scale dataset becomes publicly available
During a recent research project, the ACT prepared a dataset of more than 1460 PROBA-V images covering 74 selected locations around the globe. “We don’t include 100m resolution images for 20% of the dataset,” explains ACT research fellow Marcus Märtens. “Instead the idea is for competitors to learn from the data how to reconstruct them from multiple 300m resolution images, plus the 80% of 100m resolution images that can be employed as a benchmark. We’re interested in seeing how traditional processing algorithms compare in performance to modern machine learning, where neural networks can be taught how to interpret the combined images.”
Super-resolution has found various terrestrial applications, such as enhancing the resolution of security video footage at airports to facilitate facial recognition software or medical imaging at hospitals.
“For our competition we’ve selected areas that are mostly cloud free, but some pixels will still be lost to cloud cover,” adds Marcus. “The teams will also have to contend with different illumination and atmospheric conditions, as well as shifts in vegetation or the built environment on the ground across a timescale of up to a month between images.”
The competition was planned together with ESA’s PROBA-V mission manager Jorg Versluys: “PROBA-V’s Vegetation instrument is actually composed of three telescopes side by side, providing a 100m resolution in its central field of view, declining to 300m resolution at the sides – this a unique situation where different resolution images are available of a given scene on the ground using the same instrument. These images are also geo-referenced to a sub-pixel level accuracy using startrackers and GPS to precisely fix PROBA-V’s orbital position, along with additional fine-tuning performed with known control points on the ground.”
This is the latest competition hosted at the ACT’s Kelvins platform, named after the temperature unit of measurement – with the idea that competitors should aim to reach the lowest possible error, as close as possible to absolute zero.
- Advanced Concepts Team: Marcus Märtens
- Advanced Concepts Team: Dario Izzo