Adaptive temporal difference learning of spatial memory in the water maze task

Abstract

The goal of this work was to evaluate the performance of a biologically inspired Temporal Difference (TD) approach to the learning of spatial memory for a robot in a dry version of the Morris water maze task. The Morris water maze task is a spatial memory task in which an association between cues from environment and position must be learned in order to locate a hidden platform. To that end, earlier work was extended from a simulated dry water maze environment into a physical environment on a real robot. This effort resulted in a system that was capable of learning the necessary action preferences to successfully, and efficiently, navigate to a hidden platform. Additionally, the TD learning approach was extended to improve its performance in non-stationary environments where the hidden platform location was not fixed. The original TD learning approach was not adaptable in these non-stationary environments, as previously learned action preferences hindered the learning of new action preferences necessary to successfully navigate to a new hidden platform location. Specifically, the TD learning approach was extended by adding the ability to explicitly forget current action preferences based on previous rewards received. The adaptability of this extended version in non-stationary environments was then evaluated in simulation.