On the Optimality, Adabtibility and Robustness of Neuro-controllers
Can we design a neuro-controller that can adapt to varying conditions? What is the tradeoff between the optimality of such neuro-controllers and their adaptibility and robustness?
Study Description
Space systems, unlike most robotic systems, usually require extreme precision (in the task execution) and have rather limited resources, therefore the controllers are designed towards optimality. Optimal control theory can provide the best control algorithm given perfect knowledge of the environment an agent (rover, spacecraft etc.) will operate in. However, such conventional design tools fail to design good control strategies when the environment is dynamic or in general not fully predictable. This raises the following questions:
What is the best strategy an agent could use when prediction is impossible? What methodological tools can we employ to design a controller using this strategy?
Intelligent machines, or agents, are systems that try to maximize their chance of success through adaptation and learning to take actions based on their perception. The key to designing successful artificial intelligent entities, not just for space, is to include adaptability, autonomy and answer the questions posed.
In particular, we use artificial neural networks (ANNs), which are evolved using reinforcement learning based on a fitness function, as controllers for robots or spacecrafts that have to navigate in dynamic environments. We are using this evolutionary robotics methodology to generate spacecraft controllers.
Evolving Automatic Docking and Rendezvous (AR&D)
We here approach the problem of designing a controller for automatic rendezvous and docking (AR&D). As a first step towards a fully reactive neuro-controller (based on state feedback), an artificial neural network is trained offline by a fitness based genetic algorithm to fulfill a docking task.
Using the well understood theory of optimal control analytical solutions to optimal control problems can only be obtained for some simple problems, e.g. linear systems, in most cases though, it only allows to find necessary conditions for optimality. Using numerical algorithms this returns the optimal control history as a function of time. In short, numerical techniques derived from the theory of optimal control allow to find u(t) , an open loop control strategy that is bound to fail in an uncertain and disturbed environment. The vision, and basic motivation for this research, is to find a state-feedback, that is u(x) , providing more adaptiveness and robustness to the controller.
We investigate neuro-controllers as a possible solution to approximate u(x). It is also of great interest to compare the 'price' one has to pay in terms of optimality when using this (or other reactive control) techniques. This problem is split into the following parts:
Reproducing the optimal control with a neuro-controller, i.e. using u(x) instead of u(t), for a selected docking case, given a fixed initial position and boundary conditions Training of the controller for multiple initial conditions, leading, most likely, to a lower optimality but allowing for generalization, i.e. a 'price' is paid in exchange for robustness and adaptability
PaGMO, an open-source global multi-objective optimizer, developed at the ACT, was used to evolve the controllers. Trajectories generated by the output of the neurocontrollers, at various levels of evolution, can be seen at http://Juxi.net/projects/EvolvingDocking/.
Evolution of adaptive behavior in a gravity-varying environment
The aim of the study is to design controllers for autonomous agents that are general enough to produce adaptive behaviour in unknown environments. The focus on the design is shifted from optimal to adaptive design. In particular, we use ANNs designed by artificial evolution as controllers for robots or spacecrafts that have to navigate in environments with varying and a priori unknown gravity fields. It is well known that ANNs have very good generalisation capabilities and traning them via evolutionary techniques, (a technique called Evolutionary Robotics), can produce good solutions that take advantage of the interaction between the agent and the environment.
First simulations were performed during a 3 month Stage by Nicolas Weiss; learn more about the results in his final report.