Transfer Learning for Hyperspectral Images

State-of-the-art deep learning architectures have been dominating computer vision benchmarks for the last few years. However, training these models requires large supervised datasets. The conventional approach to solve specific computer vision problems with little supervised data available is transfer learning: a network is first trained on a large generic supervised dataset (e.g. ImageNet), then this pre-trained model is used for initializing the training procedure on the small specific dataset. However, this approach relies on large supervised datasets, which are not necessarily available for non-RGB imagery (i.e. HSI, single channel medical, radar, etc.). Therefore practitioners either need to constrain their imagery to three selected channels (possibly throwing away useful information contained in the extra channels), or they need to use smaller networks, that can be trained from scratch with the limited available dataset.

In this project the transfer of RGB pre-trained models to hyperspectral imagery is investigated. The network architecture is augmented with a trainable input pre-processing module, which is optimized to provide an transform the input image to a representation that is compatible with ImageNet pre-trained models (using domain adaptation and dimensionality reduction to 3 channels), while the task relevant information is best preserved. This module allows practitioners to make use of the state-of-the-art deep architectures on smaller supervised non-RGB imagery datasets.