An Extension of Genetic Network Programming with Reinforcement Learning Using Actor-Critic.pdf

2006 IEEE Congress on Evolutionary Computation Sheraton Vancouver Wall Centre Hotel, Vancouver, BC, Canada July 16-21, 2006 An Extension of Genetic Network Programming with Reinforcement Learning Using Actor-Critic Hiroyuki Hatakeyama, Shingo Mabu, Student Member; IEEE, Kotaro Hirasawa, Member; IEEE, and Jinglu Hu Abstract- A new graph-based evolutionary algorithm named learning methods is applied to GNP (GNP with Actor- Genetic Network Programming, GNP has been already Critic, GNP-AC). The proposed method is applied to the proposed. GNP represents its solutions as graph structures, controller of the Khepera simulator and its performance is which can improve the expression ability and performance. In evaluated. The evolution of the proposed method determines addition, GNP with Reinforcement Learning (GNP-RL) was evaluctureofoGNP,oi.ofthe opos bethodndes, andproposed a few years ago. Since GNP-RL can do reinforcement the structure of GNP, i.e., the connections between nodes, and learning during task execution in addition to evolution after Actor-Critic efficiently determines node functions, i.e., the task execution, it can search for solutions efficiently. In this speed of the wheels and the parameters of judgment nodes. paper, GNP with Actor-Critic (GNP-AC) which is a new This paper is organized as follows. In the next section, the type of GNP-RL is proposed. Originally, GNP deals with . . . . discrete information, but GNP-AC aims to deal with continuous algorithm of the proposed method is described. Section III information. The proposed method is applied to the controller shows the results of the simulations. Section IV is devoted of the Khepera simulator and its performance is evaluated. to conclusions. I. INTRODUCTION II. GENETIC NETWORK PROGRAMMING WITH A new graph-based evolutionary algorithm named Ge- ACTOR-CRITIC netic Network Programming (GNP) has been proposed[l]. A. Basic structure of GNP-AC GNP represents its solutions as graph structures which have some