深度学习论文POLICY DISTILLATION_20180118194149.pdfVIP

Under review as a conference paper at ICLR 2016 POLICY DISTILLATION ´ ˘ ¨ Andrei A. Rusu, Sergio Gomez Colmenarejo, C¸ aglar Gulc¸ehre, Guillaume Desjardins, James Kirkpatrick, Razvan Pascanu, Volodymyr Mnih, Koray Kavukcuoglu Raia Hadsell Google DeepMind London, UK andreirusu, sergomez, gdesjardins, kirkpatrick, razp, vmnih, korayk, raia@, gulcehrc@iro.umontreal.ca ABSTRACT 6 1 0 Policies for complex visual tasks have been successfully learned with deep rein- 2 forcement learning, using an approach called deep Q-networks (DQN), but rela- tively large (task-specific) networks and extensive training are needed to achieve n good performance. In this work, we present a novel method called policy dis- a J tillation that can be used to extract the policy of a reinforcement learning agent and train a new network that performs at the expert level while being dramati- 7 cally smaller and more efficient. Furthermore, the same method can be used to ] consolidate multiple task-specific policies into a single policy. We demonstrate G these claims using the Atari domain and show that the multi-task distilled agent L outperforms the single-task teachers as well as a jointly-trained DQN agent. . s c 1 INTRODUCTION [ 2