On the Optimality, Adabtibility and Robustness of Neuro-controllers

Can we design a neuro-controller that can adapt to varying conditions? What is the tradeoff between the optimality of such neuro-controllers and their adaptibility and robustness?

Back to 'Ongoing Projects'

Study Description

Space systems, unlike most robotics systems, usually require extreme precision (in the task execution) and have rather limited resources, therefore the controllers are designed towards optimality. Optimal control theory can provide the best control algorithm given perfect knowledge of the environment an agent (rover, spacecraft etc.) will operate in. However, such conventional design tools fail to design good control strategies when the environment is dynamic or in general not fully predictable. This raises the following questions:

• What is the best strategy an agent could use when prediction is impossible?
• What methodological tools can we employ to design a controller using this strategy?

Intelligent machines, or agents, are systems that try to maximize their chance of success through adaptation and learning to take actions based on their perception. The key to designing successful artificial intelligent entities, not just for space, is to include also here adaptability and autonomy and answer the questions posed.

In particular, we use artificial neural networks (ANNs), which are evolved using reinforcement learning based on a fitness function, as controllers for robots or spacecrafts that have to navigate in dynamic environments. We are using this methodology, used in evolutionary robotics, to generate spacecraft controllers.

Evolving Automatic Docking and Rendezvous (AR&D)

We here approach the problem of designing a controller for automatic rendezvous and docking (AR&D). As a first step towards a fully reactive neuro-controller (based on state feedback), an artificial neural network is trained offlne by a fitness based genetic algorithm to fulfill a docking task.
Using the well understood theory of optimal control analytical solutions to optimal control problems can only be obtained for some simple problems, e.g. linear systems, in most cases though, it only allows to find necessary conditions for optimality. Using numerical algorithms this returns the optimal control history as a function of time. In short, numerical techniques derived from the theory of optimal control allow to find u(t), an open loop control strategy that is bound to fail in an uncertain and disturbed environment. The vision, and basic motivation for this research, is to find a state-feedback, that is u(x), providing more adaptiveness and robustness to the controller.

We investigate neuro-controllers as a possible solution to approximate u(x). It is also of great interest to compare the 'price' one has to pay in terms of optimality when using this (or other reactive control) techniques. This problem is split into the following parts:

• (a) Reproducing the optimal control with a neuro-controller, i.e. using u(x) instead of u(t), for a selected docking case, given a fixed initial position and boundary conditions
• (b) Training of the controller for multiple initial conditions, leading, most likely, to a lower optimality but allowing for generalization, i.e. a 'price' is paid in exchange for robustness and adaptability

Currently we are using PaGMO, an open-source global multi-objective optimizer, developed at the ACT, to evolve the controllers. Trajectories generated by the output of the neurocontrollers, at various levels of evolution, can be seen at http://Juxi.net/projects/EvolvingDocking/.

Evolution of adaptive behavior in a gravity-varying environment

The aim of the study is to design controllers for autonomous agents that are general enough to produce adaptive behaviour in unknown environments. The focus on the design is shifted from optimal to adaptive design. In particular, we use ANNs designed by artificial evolution as controllers for robots or spacecrafts that have to navigate in environments with varying  and a priori unknown gravity fields. It is well known that ANNs have very good generalisation capabilities and traning them via evolutionary techniques, (a technique called Evolutionary Robotics), can produce good solutions that take advantage of the interaction between the agent and the environment.

First simulations were performed during a 3 month Stage by Nicolas Weiss; his final report with preliminary results can be found here.

Bibliography, References

Leitner, J., Ampatzis, C. and Izzo, D., Evolving ANNs for Spacecraft Rendezvous and Docking, Proceedings of the 10th International Symposium on Artificial Intelligence, Robotics and Automation in Space (i-SAIRAS), 2010. (link)