Jumping Spider Vision

Jumping spiders are considered to have the best vision among invertebrates used for hunting, courtship and navigation. During hunting, spiders attack by jumping on their prey accurately, inserting their fangs and injecting a rapidly acting venom. This sequence happens in less than a second. To make this method successful, the spider needs to determine the distance to its prey very accurately.

Recently it has been found that accurate distance measurement is possible through the anterior median (AM) eyes by using image defocus cues for very accurate depth perception. The principal eyes of a jumping spider use one lens system with four photo-receptor layers. Of the four layers, light can only be focused on one of the two deepest layers (furthest away from the lens). Due to the spatial separation of the layers, a projection that is focused on one layer must be blurry on the other. Using chromatic aberration of the lens, Nagata et al. showed that the images on the two deepest layers are indeed used for depth perception. Inspired by the jumping spider visual apparatus, we will investigate a simple monocular vision based distance sensor for autonomous spacecraft decent.

Advantages of a spider-based vision sensor:

• Low cost equipment is highly suitable for simple spacecrafts like cubesats, and an optical distance sensor would be advantageous for formation flying and swarm behavior.
• Can be used in debris removal, during docking and landing and as a general fall-back or assisting option for other distance sensing equipment such as LADAR, SONAR and stereo vision systems.
• Spiders achieve a very accurate depth estimation with very simple components: A single lens and very few photoreceptors. Such a system consisting essentially of only lenses and sensors and no mechanical parts would be less fault-prone and lower in power consumption than above mentioned methods.

Intern Project

Jumping spiders are capable of estimating the distance to their prey relying only on the information from one of their main eyes. Recently it has been suggested that jumping spiders perform this estimation based on image defocus cues. In order to gain insight into the mechanisms involved in this blur-to-distance mapping as performed by the spider and to judge whether inspirations can be drawn from spider vision for Depth-from-Defocus (DFD) computer vision algorithms, we modeled the relevant spider eye with the 3D software Blender and render scenes how the spider would see them. We present images created by a model of the AM eye of spider species M. aeneolus and analyze why a simple and well known DFD algorithm, namely Subbarao's DFD algorithm fails to estimate distance accurately on these images.

We tested Subbarao's DFD algorithm, on this data set and show why it is not capable of estimating distances correctly in this set-up. We conclude that our model as based on considerations and measurements by Land is too abstract to allow inferences to be drawn about the spider's Depth-from-Defocus mechanism. Further we propose another straightforward way to quantify defocus and suggest additional aspects to be included in the model.

Goals of the project:

• Create a model of the anterior median (AM) eye of spider species Metaphiddipus aeneolus to see what kind of images are created on the spider retina.
• Test a well known but basic DFD-algorithm on these images to see if the spider's depth assessment performance can be explained by the principles of this algorithm. We analyze if both CV-DFD algorithms and the spider DFD mechanism are based on the same high level principles and if not, if CV-DFD algorithms can be improved by learning from the spider's DFD mechanism.

This project had a duration of three months and was taken on by a trainee, Aleke Nolte. For details read the project report and download the Blender model and all necessary files to set-up the simulation.

References

1. Subbarao, M. (1988, December). Parallel Depth Recovery by Changing Camera Parameters. In ICCV (pp. 149-155).
2. Nagata, T., Koyanagi, M., Tsukamoto, H., Saeki, S., Isono, K., Shichida, Y., ... & Terakita, A. (2012). Depth perception from image defocus in a jumping spider. Science, 335(6067), 469-471.
3. Land, M. F. (1969). Structure of the retinae of the principal eyes of jumping spiders (Salticidae: Dendryphantinae) in relation to visual optics. Journal of experimental biology, 51(2), 443-470.
4. Blender Online Community. Blender - a 3D modelling and rendering package. Blender Foundation, Blender Institute, Amsterdam, 2014.

Outcome