Here is a link to the contribution page of Frédéric Boyer.  He has done some work with an underwater robot.

https://www.researchgate.net/profile/Frederic_Boyer

Hope this helps

I give a short overview of genetic algorithm concepts in my latest paper:

http://iopscience.iop.org/1748-3190/10/5/056005

I'd also recommend that you check out Hod Lipson's work on evolutionary robotics.

If what you want to do is learn a motion policy from sensors, then maybe you're better off using reinforcement learning.

Thank you for reply about my quesiton,Lehtihet , Spitz   . I have a review of the passive navigation paper of Frédéric Boyer, but it is not associated with evolution of robot, it is a new kind of awesome navigation method for underwater vehicle really.

As for the research of Hod Lipson, does the motion learn from the body of robot? What I really want to do is to create a robot can learn from sensors and actuators to generate a self-learning movement.

Hod Lipson works on evolutionary robotics. He co-develops robots and their controllers. For example, if you evolve a robot for swimming, the algorithm may find a snake-like robot + a controller that causes it to undulate (like a CPG).

If you already have a particular robot with a given set of sensors and actuators, then you can evolve the controller alone or use reinforcement learning to learn a controller. With the first option, you create many controllers that convert a certain sensory input into an actuation output, then you test every controller to see how it performs. The best controllers are selected to create the next generation. With reinforcement learning, you create a policy that decides which actions to take based on the system's state (e.g. sensory input). You can reward the controller for good behavior and/or penalize it for bad behavior. Then the policy is updated based on the reward/penalty.

I hope that helps you decide which approach to use.

Just some thoughts. Movement is often periodically,

so you could use a context free grammar for

describing a movement module. The grammar could

be evolved with genetic algorithms. Every grammar rule

should have its own genome so that you have to deal

with a population of different species.

As for the sensors, you could have lamps in

a basin at different places, and the robot

should try to approach the one and only lamp

which is switched on as fast as possible,

or as energy saving as possible.

Instead of lamps you can also work with

electric fields. Two electrodes in a distance

of about 10cm should be able to produce

field lines that can be sensed with

a voltage detector and similar electrodes

at the robot.

Regards,

Joachim

@Jonathan Spitz

Thank you for listing two options for evlolving robot, I have read some literatures about q-learning, and it shows the low efficiency of evolution in an example of bicycle control. The robot I created before is attached, which is an ROV with four thrusters(two vertical along z axis and two horizontal thrusters along x axis). Besides, I get knowledge of NEAT algorithm, but I donot understand how to express controllers in genetic algorithm, could you tell me more clearly or recommand some papers about how to use genetic algorithm to evolve controllers or neural networks? 

@Joachim Pimiskern 

Thank you for your suggesions about how to express controllers in genetic algorithm, I have got the thoughts you transfer but I still have no idea about how to implement the expression of grammar rules.

In my case, I used a genetic algorithm to optimize the parameters of a controller that I designed, instead of tuning it by hand. My controller uses a single phase oscillator to coordinate the activation of 3 torque pulses (2 applied at the hip and 1 at the ankle). My genome encoded the frequency of the oscillator and the strength, timing and duration of each pulse.

I've also used a similar genetic algorithm to tune PID parameters.

As long as you think of a good way to encode your controller parameters and a good way to measure the fitness of your controller, a genetic algorithm is the easiest way to go, IMHO.

Thank you for your suggestions, Jonathan Spitz. The evolution of robotic control problem is converted into the tuning of controller parameters, which mainly includes two critical parts, the first is how to encode the parameters of controller selceted for robot; Would you recommand some more books on encoding problem for me? ; The second is to set fitness function according to the performance of robot, for example, if the robot is trained to evolve a controller to track a line trajectory, the fitness function is is set on the basis of controller error or distance the robot has travelled?

There are actuators in your robot. At a certain

point in time, a tick, you can switch an actuator

to on or to off. Let's say such actions are

denoted with lowercase letters, called terminals.

Uppercase letters are called nonterminals and

have to be replaced by a grammar rule that maps

a nonterminal to a non-empty sequence of letters.

Such rules could be S->AbC, A->aa, A->bA, C->c.

Starting with the string S, a possible derivation

sequence could be S, AbC, bAbC, bbAbC, bbaabC,

bbaabc.

Such a terminal sequence bbaabc could steer your

robot. Restriction to sequences that are brought

forth only by the grammar makes movement modular.

How grammar rules could be bred by genetic

algorithms: the above grammar could be written

as {SAbC, Aaa, AbA, Cc}. Maybe you are familiar

with GA. The most important operators are

mutation and crossover, perhaps also reversal.

Examples:

Mutation: SAbC goes to SCbC.

Crossover: Aaa and AbA goes to AaA and Aba

Reversal: SAbC goes to CbAS

Note that the first letter must be a capital

since a rule replaces only one nonterminal.

For reaction to sensor input you can encode

a binary decision tree whose inner nodes

are if then else conditionals and whose

leafs are actions.

How the fitness of a set of rules should

be determined is a different matter. Maybe

there should be a good/bad signal that is

operated manually by a trainer.

Regards,

Joachim

Hi

Any periodic type movement can be easily generated using Central Pattern Generators "CPGs". This use nonlinear oscillators. The simplest among these is Mastouka Oscillator.

Good luck.

Maki

@Joachim Pimiskern,

Hi, Thanks very much for describing the thoughts in details. It seems that it is a GP based method, using which the actions are defined using lowercase letters, such as acututor 1 (on -->a; off -->b), acututor 2 (on -->c; off -->d).....etc. I once used genectic programming technique to model dynamic of robot, where symbols represents function model like + - * / sin cos.... It seems that the technique used are same. However, I can not get your meaning for Uppercase letters clearly. Why no terminals are required in generating a sequence?  Because the sequence is always composed with lowercase letters in the end. Should the Non Terminal  be used or it serves the role of function model? Besides, fitness function used seems able to optimize the evolution of sequence.  

@Maki Habib

Hello.Thank you for your suggestions for my question. In my case, the body of vehicle is considered to be rigid body with four thrusters. CPG methods is more suitable for robots like snake, fish, or spiders.  Thank you.

A nonterminal can be conceived as kind of zipping

a sequence. The idea is that every longer

sequence of actions should have its own grammar,

and that the actual sequence of actions is

derived from the start symbol S.

After thinking about the concept, there's

maybe a simpler approach. Use a complete

neural net, one, where every neuron gets

its input from every other neuron. Use

a sigmoid activation function. Treat all

thresholds and weights as one big float

number vector. This way you can also apply

genetic programming to evolve the net

parameters. In order to assign input

and output, simply choose a free neuron.

The output neurons are for the actuators,

and the input neurons are from the sensors,

pattern generators, and steering commands.

How one thinking step works: if n is the

number of neurons, choose for example

5*n neurons randomly and compute their

transfer function (map inputs to output).

Regards,

Joachim

Let's say for example that you have a simple line follower robot with a light sensor in the front and a motor on each side. The robot must follow a line on the floor from point A to point B. Instead of using a single black line on white floor let's use a gradient line, i.e. a line composed of several gray-scale lines (from white to black).

This system has a single analog input (0 - black to 1 - white) and two outputs (motor left, motor right). Let's use a PWM output that goes from 0 (no speed) to 1 (full motor speed). A simple controller could have 4 parameters:

Out_right = k1*(input - i0_1)

Out_left = k2*(input - i0_2)

where k1, k2 are linear gains and i0_1, i0_2 are desired input values. Then you can encode a simple genome g = [k1, i0_1, k2, i0_2]. You can make a more complex controller if you look at the difference between the current reading and the one before (like PD controller), then you get 6 genes: g=[k1, i0_1, k2, i0_2, k3, k4].

The encoding in this case is quite easy. Each gene is a floating point number with a given range, e.g. -100 to 100 or 0 to 1.

The choice of fitness function depends on what you want to achieve with the controller. You could integrate the distance of the robot to the line over a certain period of time (or until it reaches point B). You could use the time it took to reach B as another fitness, or how much control effort it used.

When you have several fitness functions you can use a weighted sum of all the functions to sort your population or you can use a pareto front approach, like I used in my latest publication. This method gives you a range of optimized solutions rather than a single one, .e.g some will be very fast, some very efficient.

Good luck!

Best,

Jonathan