Genetically-regulated Neuromodulation Facilitates Multi-Task Reinforcement Learning

In this paper, we use a gene regulatory network (GRN) to regulate a reinforcement learning controller, the State-Action-Reward-State-Action (SARSA) algorithm. The GRN serves as a neuromodulator of SARSA’s learning parameters: learning rate, discount factor, and memory depth. We have optimized GRNs with an evolutionary algorithm to regulate these parameters on specific problems but with no knowledge of problem structure. We show that genetically-regulated neuromodulation (GRNM) performs comparably or better than SARSA with fixed parameters. We then extend the GRNM SARSA algorithm to multi-task problem generalization, and show that GRNs optimized on multiple problem domains can generalize to previously unknown problems with no further optimization.

In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2015)