Artificial Intelligence Lab Department of Informatics University of Zurich Andreasstrasse 15 8050 Zurich Switzerland

ShanghAI Lectures
Fall Term 2011

Exercise 3: Co-evolution of body and brain

Distribution: 3 November 2011 Due date: 14 November 2011 (23:00 CET) Correction date: 21 November For questions, please contact your group supervisor or: Qian Zhao (Task 1), zhao@ifi.uzh.ch or Matej Hoffmann (Task 2), hoffmann@ifi.uzh.ch Total amount of points: 30

Introduction
Remember the Puppy robot from the lecture (Lecture 4, for instance)? In this exercise, you will have the chance to improve its brain as well as body. Fig. 1 shows a picture of one of the Puppy family members that we currently have at the lab: the Minidog.

Fig. 1. Minidog

Unlike in the case of the Passive Dynamic Walker that you have witnessed in the previous exercise, the Minidog is powered by motors. The controller is, however, very simple. We applied a simple oscillatory position control of the motors. Moreover, we have used the same parameters for the front leg pair and another set for the hind pair. The target joint angle of each motor Pf(t) and Ph(t) for the fore legs and hind legs respectively was determined as:

where the oscillation can be varied by changing the amplitude A, offset B, angular frequency ( = 2** f), and phase parameters. Each pair of legs thus oscillates in synchrony, however, there can be a phase difference of between the leg pairs. If they are close to out of phase ( = 3.14), the gait is called a bounding gait. Please watch a video of the Minidog running at: or

https://files.ifi.uzh.ch/ailab/people/hoffmann/puppy/videos/minidog.WMV https://files.ifi.uzh.ch/ailab/people/hoffmann/puppy/videos/minidog.mp4.

You can see that the behavior is far from optimal. How would you improve it? Maybe change the amplitude at which the hind legs move? Or the frequency at which the motors move is not right? Or maybe the body is not the best? Is the robot too wide? Or the legs too short? There are infinitely many possibilities. To get a better grip at this complex problem, we have created a model of this robot in the Webots simulator [1] (see Fig. 2), where it is more easy to experiment with different parameter settings.

b a tte r y

body

m o to r h ip s e rv o jo in t fe m u r k n e e jo in t tib ia

Fig. 2. Simulated Minidog.

ShanghAI Lectures Fall Term 2011 Exercise 3

Task 1 (10 points in total)- Manual manipulation of brain (controller) and body
In this task, first, you are asked to play with the controller to observe how the robot behavior changes when one parameter changes. Then you will choose a case where the robot fails and try to achieve stability by tuning the spring in the legs.

1.1 (4 points) Manipulation of brain (controller) The parameters are the following: Controller C_Amp_F C_Amp_H C_Frequency C_Offset_F C_Offset_H C_Phase_H 0.7 0.7 2.5 0 0 from 0.1 to 1 by 0.1
Table 1. First, set up the simulator see Appendix Setup for Task 1. Now you will be tuning the phase lag between front and hind legs (C_Phase_H) from 0.1 to 1 in steps of 0.1. You can proceed like this: 1. Go to individual_controller.txt under the directory controllers/helper_classes/paramOptimization and set the appropriate C_Phase_H parameter (you can start with 0.1) and save. 2. Now press the Revert button in the simulator and let the robot run (you may need to press Run). 3. Stop simulation at 104 s as shown in the console and calculate the speed by averaging the last five speeds (if you are impatient, switch to the Fast simulation mode). If the robot falls before 100s, then the speed is 0. Ignore what happens after 104 s. After you have collected all the values (10 speeds), do the following: 1. Plot the average speed over phase, i.e. make a diagram with phase on the x-axis and speed on the y-axis. 2. Describe how you interpret the chart. Take a video of the best performance (like you did in Exercise 2 on Passive Dynamic Walkers in Webots; File->Make Movie). Deliverables: Document containing the chart, a brief interpretation of the chart, a proper value for C_Phase_H which makes Minidog walk with a good performance, and a video when phase equals 0.7.

Morphology FrontKneeSpringConstant HindKneeSpringConstant 0.5 0.5

1.2 (6 points) Manipulation of body morphology

The Minidog has four actuated hip joints driven by a simple controller. The knee joint is passively connected by a spring as shown in the figure on the right. Although the robot doesn't have sensors, stability can be achieved if the body dynamics is appropriately tuned (the self-stabilization phenomenon). In this context, we will play with the morphological parameter of spring constant in the hind legs and observe the behavior of the Minidog. ShanghAI Lectures Fall Term 2011 Exercise 3 3

First, Change C_Phase_H to 0.7 in individual_controller.txt and save. Now you will change the spring constants in the passive knees. To do that, you need to use the Scene tree, similarly to what you did with the Passive Dynamic Walker. To change the spring constant of front legs, you will access: DEF PUPPY ROBOTDEF HIP0 ServochildrenDEF KNEE0_SERVOSpring Constant. DEF PUPPY ROBOTDEF HIP1 ServochildrenDEF KNEE1_SERVOSpring Constant. For the hind legs, it is: DEF PUPPY ROBOTDEF HIP2 ServochildrenDEF KNEE2_SERVOSpring Constant. DEF PUPPY ROBOTDEF HIP3 ServochildrenDEF KNEE3_SERVOSpring Constant. Note: Be cautious to change the Knee servo and not the Hip servo spring constant. (a) Tuning the hind legs spring constant with front legs spring constant fixed to 0.5 Keep the front legs spring constant 0.5, and tune the spring constant in the hind legs from 0.1 to 1 in steps of 0.1. The procedure is similar to task 1.1, but this time you will be changing the world (.wbt) file through the Scene tree. You will proceed like this: 1. 2. 3. Stop the simulation and Revert. Change the appropriate spring constants and save the world. Revert and Run.

In each case, you proceed like in task 1.1 you will record the speed. (b) Tuning the hind legs spring constant with front legs spring constant fixed to 0.2 Same like (a), but fix the front legs spring constants (in the knees not hips!) to 0.2. Deliverables: Document containing the charts, a brief interpretation of the charta, a proper value of spring constant which makes the puppy walk with a good performance, and the associated video.

Task 2 (20 points in total) Artificial Evolution

Evolutionary computation has not been covered in the class yet. Therefore, we will ask you to do some reading on your own. Artificial evolution is covered in Chapter 6 of How the Body Shapes the Way we Think. Please read at least from the beginning of the chapter to Section 6.2 and then Sections 6.5 and 6.6. In this exercise you will have the opportunity to test one type of evolutionary algorithm yourself. The genome of each Minidog is composed of real numbers (see Table 2). An initial population of 80 genotypes is generated randomly. These are then directly turned into the phenotype - the Minidog model in webots and its controller parameters. The phenotypes are evaluated based on average speed in 10 seconds. Then, the best ones are selected using a roulette wheel selection. In addition, elitism is turned on to preserve the best individual of each generation. New genotypes are generated using mutation only. Every new genotype has only one parent. The fitness

ShanghAI Lectures Fall Term 2011 Exercise 3

function is the speed of the robot1. The parameters and their ranges are listed in Table 2. Parameter Controller Amplitude front Amplitude hind Offset front Offset hind Phase lag Frequency Body Length Width Front Femur Length Front Knee Spring Constant Front Tibia Length Hind Femur Length Hind Knee Spring Constant Hind Tibia Length Angle at front knee joint Angle at hind knee joint PuppyLength PuppyWidth FrontFemurLength 0.133 0.1 0.052467 0.08 0.05 0.01 0.1 0.01 0.01 0.1 0.01 0.9 0.9 0.18 0.15 0.07 1 0.07 0.07 1 0.07 1.5 1.5 C_Amp_F C_Amp_H C_Offset_F C_Offset_H C_Phase_H C_Frequency 1 1 0 0 0.5 3 0.5236 0.5236 -0.698 -0.698 0 2.5 1.57 1.57 0.698 0.698 1 3.5 Param short name Default value Min Max

FrontKneeSpringCon 0.5 stant FrontTibiaLength HindFemurLength 0.056994 0.052467

HindKneeSpringCon 0.5 stant HindTibiaLength 0.056994 KNEEFServoRotatio 1.25 n KNEEHServoRotatio 1.25 n

Table 2. Amplitude and offset values are in radians, phase lag is normalized to <-1,1>. Frequency is in Hz. From the body paramaters, the lengths are in meters, angles at passive knee joints are in radians.

Task 2.1 (2 points)

From the information you have about the algorithm, would you classify it as a genetic algorithm or an evolutionary strategy? Why?

Task 2.2 Automatic optimization of the controller (brain)

Now, you will need to set up your simulator again. Please follow the steps in Setup for task 2 of the appendix. Please just follow the steps there and do not run the simulation yet. First, we will see how our evolutionary algorithm can deal with optimizing the controller. 1 In fact, it is 4* the speed in m/s. This is for implementation reasons for another optimization algorithm, we wanted to avoid too small fitness values. ShanghAI Lectures Fall Term 2011 Exercise 3 5

If you run your simulator now, the optimization starts. 2 You will see a new Minidog controller being tested every 10s. The evolutionary algorithm is set for 30 generations, with 80 individuals each. This will take some time (e.g. 1 hour). Therefore, make sure you press the Fast button in the simulator like this you do not see the rendering, but the simulation runs much faster. If you want to see intermediate results, just press Run (play) button again, watch for a while, and then press Fast again. Depending on your system configuration, the simulation can take about an hour. You can watch the progress in the console. If there are no more outputs from the [supervisor_controller], just the speed from [mini_dog_controller], the evolution has finished. Then you can press Stop and go the question.

Question 2.2 (3 points)

Now collect your results (see Appendix Task 2 Collecting Results). What is the fitness of the best individual? How have the control parameters changed compared to the initial situation (Table 2)? What is your interpretation of the changes?

Task 2.3 Automatic optimization of the body

We will now try to see what effect the body optimization will have like in the Passive Dynamic Walker case. For that, you need to initialize your simulator. Please see the Appendix Setup for Task 2 - Re-initialization for new experiments. Now we need to set the parameters to optimize. This time we want to try with the morphology. Do the following: 1. 2. Under controllers/helper_classes/paramOptimization delete paramsRange_body.txt and

paramsRange_controller.txt Go to the controllers/helper_classes/paramOptimization/parameter_ranges/ directory, and copy the files paramsRange_body.txt paramOptimization dir.3 3. 4. Go to the paramOptimization dir and rename paramsRange_controller_noController.txt to paramsRange_controller.txt Start Webots. and paramsRange_controller_noController.txt one directory up to the

You will be seing Minidogs with different morphologies being tested (not all the MiniDog look so different in the beginning they started from one common ancestor). Press Fast and let the morphology evolve.

2 Please note that we are expecting you to follow the exercise sequentially. Hence the first parameter sets that are prepared in the controllers/helper_classes/paramOptimization directory paramsRange_controller.txt and paramsRange_body.txt are prepared for the controlleronly optimization. That is they are the files paramsRange_body.txt and paramsRange_controller_noController.txt from controllers/helper_classes/ paramOptimization/parameter_ranges/, where the latter file has been renamed to paramsRange_controller.txt.

3 Note that the original paramsRange_body.txt that you deleted before was actually from
paramsRange_body_noBody.txt, because in the previous task, the body was constant. ShanghAI Lectures Fall Term 2011 Exercise 3 6

Question 2.3 (3 points)

Now collect your results (see Appendix Task 2 Collecting Results). What is the fitness of the best individual? How has the body morphology changed when compared to the initial situation ((Table 2))? What is your interpretation of the changes?

Task 2.4 Automatic co-optimization of the body and controller (brain)

Finally, we would like to see what happens when both morphology and controller co-evolve (see Section 6.6 of How the Body Shapes the Way we Think or Sims' original paper [2]).

Please re-initialize your simulator. Please see the Appendix Setup for Task 2 - Re-inialization for new experiments. Then do the following: 1. Under controllers/helper_classes/paramOptimization delete paramsRange_body.txt and

paramsRange_controller.txt 2. Go to the controllers/helper_classes/paramOptimization/parameter_ranges/ directory, and copy the files paramsRange_body.txt and paramsRange_controller.txt one directory up to the paramOptimization dir.4 3. Start Webots. You will be seeing Minidogs with different morphologies and controllers being tested (not all the MiniDog look so different in the beginning they started from one common ancestor). Press Fast and let the parameters evolve.

Question 2.4 (6 points)

Now collect your results (see Appendix Task 2 Collecting Results). What is the fitness of the best individual? How have the body morphology and controller changed? What is your interpretation of the changes? How does the best fitness compare to the situations where only morphology or control was optimized? Which optimization do you consider the most promising? Controller, body or both? What are the implications in a practical situation.

Task 2.5. Putting the results back to the real world.

The Minidog robot (Fig. 1) was designed in a modular way5 and its body is composed of parts that can be printed from ABS plastic on a 3D printer. The model in Webots was built with the aim of creating a physically realistic model of the real robot (the morphology, including mass distribution, the motor strength etc. was modeled to correspond to the real counterpart). The results of our simulation could now be used to optimize the behavior of the real Minidog not only the controller, but also the body, since we have the opportunity to modify the body easily. A

4 Note that the original paramsRange_body.txt that you deleted before was actually from
paramsRange_body_noBody.txt, because in the previous task, the body was constant. 5 Thanks to Juan Pablo Carbajal and Konstantinos Dermiztakis. ShanghAI Lectures Fall Term 2011 Exercise 3 7

similar approach was used by Lipson and Pollack [3].

Question 2.5.
(a) (3 points) Do you think the behavior of the robot in the simulator will exactly match the behavior in reality? If not, what could be the reasons for a potential mismatch (a so-called) reality gap? (b) (3 points) The alternative is to optimize the behavior directly on the real robot. Can you list some pros and cons of this approach as opposed to the simulation and then transferring the results to the real robot?

References
[1] Webots, http://www.cyberbotics.com/ [2] Sims, K. (1994a). Evolving virtual creatures. Computer Graphics, 28:1534.
[3] Lipson, H., and Pollack, J. B. (2000). Automatic design and manufacture of artificial lifeforms. Nature, 406:974978.

Acknowledgment
We would like to thank Stefan Hutter this exercise was based on the code that he developed as part of his Master thesis at the AI Lab. We would also like to gratefully acknowledge the support from Olivier Michel and Yvan Bourquin in the preparation of this exercise.

ShanghAI Lectures Fall Term 2011 Exercise 3

Appendix Preparation:
for Mac user: install Xcode 4 for Apple to compile files in webots

Setup for task 1

1. Unzip the folder ex3_task1.zip to a convenient location on your harddisk. 2. Start Webots, choose Open world. Then open world file miniDog.wbt from the directory /worlds/ of the exercise. A window will pop up about the missing controllers, just close it, we will fix it in the next step. 3. In the Editor window, open mini_dog_controller.cpp under the directory of /controllers/mini_dog_controller/ and build the controller (cogwheel icon).

Setup for task 2

Setting up:
1. Unzip the folder ex3_task2.zip to a convenient location on your harddisk. Then go inside the mini_dog_webots folder this will be your working directory from now on. 2. Open the world file worlds/miniDog.wbt in Webots. Some windows pop up about the missing controllers, just close it, we will fix it in the next step. 3.Build controllers In the editor window: Open the following files in the editor window and build them (the cogwheel icon). (The simulator will ask you if you want to revert after each succesful build say no.) mini_dog_controller.cpp from directory of /controllers/mini_dog_controller/, supervisor_controller.cpp from directory of /controllers/supervisor_controller/ . Note: If the supervisor_controller does not compile and you get messages starting with:

supervisor_controller.cpp:27:35: error: ./GA/geneticAlgorithm.h: No such file or directory. then a likely source of the problem is that your operating system or file manager has changed the directory name controllers/supervisor_controller/GA to ga - lower-case. You need to make sure it is in capitals.

Re-inialization for new experiments

Please initialize every time when you redo the experiment. This will restore the initial morphology of the robot and ShanghAI Lectures Fall Term 2011 Exercise 3 9

delete the data from the previous experiment. Make sure you copy the results from the previous experiment (mainly the controllers/supervisor_controller/GA/eaStats.csv file) to another directory before initialization. First, quit Webots. Then: For Linux /Mac users: go to controllers/supervisor_controller/ and run setupGA.sh in a terminal.6

For Windows users: ( please manually execute the commands listed in setupGA.sh.) For all the copy commands use overwrite. Copy the protos directory from controllers/helper_classes/paramOptimization/optimization_seed/ to the current directory (mini_dog_webots) Copy worlds/ Copy individual.txt and individual_controller.txt from controllers/helper_classes/paramOptimization/ optimization_seed/ to controllers/helper_classes/paramOptimization/ Delete all the files under controllers/supervisor_controller/GA/todo/ Delete all the files under controllers/supervisor_controller/GA/done/ Delete individual.txt under controllers/supervisor_controller/GA/current/ Delete supervisor_memory.txt under/mini_dog_webots/controllers/supervisor_controller Delete (actually better store somewhere else) eaStats.csvunder controllers/supervisor_controller/GA/ miniDog.wbt from controllers/helper_classes/paramOptimization/optimization_seed/ to

Collecting results
The results are summarized in the eastats.csv file in the controllers/supervisor_controller/GA directory. You can open it in MS Excel, Open Office or similar software. It is semicolon (;) separated. In MS Excel you may need to do Data From text Import and choose of the columns (S AJ). semicolon ; as the delimiter. Now the first columns (A-R) correspond to a parent of each individual, the current individual is only the second half

The results are now in chronological order starting with the individuals from the 0th generation. To simplify the 6 Note, it can happen that some of the directories that you are trying to clean are already empty. Hence a message like rm: cannot remove `./GA/todo/*': No such file or directory does not have to worry you. ShanghAI Lectures Fall Term 2011 Exercise 3 10

evaluation, you can sort them by Fitness (Column AJ). To do this, mark all columns and sort by the last column fitness descending. The best individual will then be at the top. (If for some reason there are some individuals for which the fitness was not available there is nan, just ignore those individuals.) If you want, you can save this as eaStatsSortedByFitness.csv.

ShanghAI Lectures Fall Term 2011 Exercise 3 11