democratic population decisions result in robust policy-gradient learning a parametric study with gpu simulations民主人口决策导致健壮policy-gradient学习参数与gpu的模拟研究.pdfVIP

Democratic Population Decisions Result in Robust Policy- Gradient Learning: A Parametric Study with GPU Simulations 1 2 3 1 Paul Richmond , Lars Buesing , Michele Giugliano , Eleni Vasilaki * 1 Department of Computer Science, University of Sheffield, Sheffield, United Kingdom, 2 Gatsby Computational Neuroscience Unit, University College London, London, United Kingdom, 3 Department of Biomedical Science, University of Antwerp, Wilrijk, Belgium Abstract High performance computing on the Graphics Processing Unit (GPU) is an emerging field driven by the promise of high computational power at a low cost. However, GPU programming is a non-trivial task and moreover architectural limitations raise the question of whether investing effort in this direction may be worthwhile. In this work, we use GPU programming to simulate a two-layer network of Integrate-and-Fire neurons with varying degrees of recurrent connectivity and investigate its ability to learn a simplified navigation task using a policy-gradient learning rule stemming from Reinforcement Learning. The purpose of this paper is twofold. First, we want to support the use of GPUs in the field of Computational Neuroscience. Second, using GPU computing power, we investigate the conditions under which the said architecture and learning rule demonstrate best performance. Our work indicates that networks featuring strong Mexican-Hat-shaped recurrent connections in the top layer, where decision making is governed by the formation of a stable activity bump in the neural population (a ‘‘non-democratic’’ mechanism), achieve mediocre learning results at best. In absence of recurrent connections, where all neurons ‘‘vote’’ independently (‘‘democratic’’) for a decision via popula