Yet Another GA Book

I recently came across this book: "Practical Genetic Algorithms". I already studied 3 chapters, and found it really intresting. As the name implies it has a practical approach in which the authors try to explain the implementation details of GA.

In most of the GA books I've ever read the discussion of GA operators such as crossover or mutation is vague. Statements like "Mutation is to change non-junk part of the chromosome with some probability" are everywhere, and there is usually no distinction between real-valued and binary GAs. Conversely this books explains all these different variations with reasonable detail, and has good references to relevant papers. For example the book clearly describes different types of selection methods, different variations of crossover and mutation in both real-valued, and binary GAs, and for all of these it has references to papers where a more in-depth discussion could be found.

In short this book is a pragmatic tutorial on GA which has a good balance between theory an practice.

I guess anybody who has every worked with a GA can borrow a lot of ideas from this book in order to fine tune their implementation.

Step by Step

Re-evaluation plan for Gyoomard is to assess the results of different stages step by step. This would mean to start off with no control and run the biped simulation as a Passive Dynamic Walker, then add a very simple control with a simple neural network. Finally achieving the full problem configuration.

Another main area for investigation the exact mechanism for the GA. It seems currently that the solver falls into local maximum points.

One other feature which can be added in the step by step progression is a support to help Gyoomard in the walking process. The support will be used for learning and then taken out at some future stage.

Blocked

Gyoomard's evolution has come to a block so far. Hasn't been able to cover a longer distance. The whole Genetic Algorithm operations need to be revised, mutation of one per gene of a genome on an initial population of 50 doesn't seem to cover the search space in a suitable format.

8m

8 meters has been the record for Gyoomard walk so far before falling down. This would be something like taking two steps.
There are many parameters needing to be tweaked and the overall code needs to be enhanced regarding efficiency.

Gyoomard is trying

Gyoomard is undergoing training at the moment. Or better say Gyoomard's are being trained, evaluated, reproduced and evaluated again, until an elite genome is found that would be able to walk on a straight ground.

Generation 165 of the simulation is running currently and the best displacement so far has been 3.4 meters. 

This is the first try in making Gyoomard walk and no expectations yet, lots of things need tweaks.

Lets see how things work out but the experience is very exciting already.

Gyoomard's brain is currently a CTRNN with 10 nodes, fully connected. Its body is comprised of a lower leg, upper leg and hip. 1 DOF knees and 2DOF hip joints.

Evolving Integrated Controllers using CTRNN

The first attempt for evolving an integrated dynamical neural network as control system failed in 1994 in an experiment by Yamauchi, and Beer. As a result they recommended a modularized approach in which separate controllers were evolved for tasks such as reactive and sequencing behaviors and the learning task. They also used hard wired reinforcemennt signals as feedback into the learning module, which ruined the inital idea of an integrated (Not modularized) control system. Later on Harvey et al showed the possibility of evolving an integrated conroller system by evloving a CTRNN for controlling the behavior of a simulated khepera mini-robot. The details of their experiments could befound in the following paper:

Evolving integrate controllers for autonomous learning robots using dynamic neural networks

Overall points

The following paper outlines the overall ideas regarding evolutionary robotics.

Evolutionary Robotics in Behavior Engineering and Artificial Life, Dario Floreano

There are two main areas covered in the study, the first being the use of evolutionary techniques to model robots which operate in specified constraints and have well known engineering goals. The second part studies the application of evolutionary methods in artificial life studies in general. Co-evolution is researched which seems to be a huge topic which has not been discovered much yet.

Morphological Computation

The idea of Morphological Computation is how the morphology which is about the form of a robot is important in the overall behaviors and abilities and how the form can help the overall computation efficiency and hence called morphological computation.

The following paper provides a good discussion on the topic and examines a few cases where the sensor positions or type of material used for the robots helps in the development of robust systems which do not necessarily need to model everything internally and the form helps the overall information processing requirements.

Morphological computation: connecting body, brain and environment. Rolf Pfiefer and Fumiya Iida

GA library

An open source Genetic Algorithms library in c++ from MIT:
http://lancet.mit.edu/ga/

The architecture seems to be flexible enough to enable extending the features or modifying the behaviors.

Controllers

One of the main questions of this first milestone is how to model the controllers for the biped simulation.

The neural network will output the desired angle for every joint and we hope to apply a relevant torque to each joint in order to reach that desired angle. A PD controller can be assumed to control each joint, in this case a constant value for the proportional coefficient and one for the derivative coefficient needs to be assumed. Large values can make the physics simulation unstable and these values seem to need some tweaking. Other options are to use the built in features for PhysX motor enabled joints which can make the joint move at a specific speed or to a specific angle. The torque can be limited on these motor joints which should be a very good feature.

Some more testing on the model needs to be done in order to come up with this efficient control method for each joint.

This project from Ecole Polytechnique Federale De Lausanne mentions some problems associated with joint controllers, the solution selected in the project is to use a constant speed for the joint equal to the proportional coefficient. Teta_dot = k(delta_Teta). The justification being that larger difference in desired angle from existing angle needs larger values for velocity to move the joint.