Astro Drone - A crowdsourcing game to improve vision algorithms

The Astro Drone app is available for free download in the Apple App store though is no longer maintained by the team. People who possess a Parrot AR.Drone can play the game, in which they are challenged to perform different space missions in an augmented reality environment. The first release contains the training level, in which players learn to dock as well as navigate to the International Space Station. The second level involves the Rosetta mission of ESA. In this level players have to navigate the Rosetta spacecraft to comet 67P/Churyumov–Gerasimenko, while avoiding space debris. When that part of the mission has been accomplished, the Philae lander must be released at the right moment and in the right direction, in order to land successfully on the surface of the comet. New levels will be added incrementally with new releases. Below are screenshots of the first and second level of the game.

The Experiment

Players can choose to contribute to a scientific crowd sourcing experiment that aims to improve autonomous capabilities of space probes, such as landing, obstacle avoidance and docking. The app processes the images made by the AR drone's camera, extracting abstract mathematical image features. These features cannot be interpreted by humans, nor can the original image be reconstructed. However the features can be used by robots to learn how to navigate in their environment. Players can join the experiment by going to the high score table. If they agree, the feature data is sent over the internet.

Known issues:

• The marker is not always recognized, even if it is within camera view and very clearly detectable for a human. The marker detection happens onboard of the drone and works reasonably well, given the computational constraints on the drone. However, recognizing the marker in different backgrounds is not an easy computer vision problem. One may need to try variety of places - typically daylight helps.
• When a player is connected to the drone while visiting the high score table, he/she will see a transmission symbol but no message regarding internet activity. A warning has been added, but it takes a while to show up if there is no internet connection. This typically means that the player is connected with the WiFi network of the drone, and not to the internet. A solution that always works (but is slightly cumbersome) is to leave the app, reconnect to your internet WiFi and go back to the app.
• Ensure the correct color of the marker you use (blue or green) is selected in the in-game settings menu. Once the marker is detected by the drone, the flashing 'looking for marker' symbol will disappear.