investigating the role of fast vs. slow components in learning
I've been working during the last days but haven't been able to report on any of it. Mainly because I have not made as much straightforward progress as I can easily explain.
I've been working on evolving learning on a continuum in the abstract scenario. I have managed to evolve circuits that can 'memorise' features on a continuum without the need to introduce any parameter-changing rules.
How do they memorise the features? The key to answering this question is in the interaction at very different time-scales between the 'neurons' of the circuit.
Currently I am performing experiments in this one-shot remembering features on a continuum where the delay can vary greatly (between 10 and 100 units of time).
From 20 evolutionary runs using 10 node CTRNNs I observed that there were, roughly, 2 possible types of evolved behaviors: discrete categorisers or generalisers. The discrete categorising circuits are unable to differentiate most of the features within the continuum and rather they make up 2 or 3 categories for them. Either small and large, or small, medium and large type of features. The ones that make 3 categories score better than the ones that only make 2 categories. The generalisers do much better and can distinguish between many more signals, without making any obvious categories a priori.
In terms of the mechanisms available, there's 10 nodes and they can act in very different time-scales. The fastest ones have a time-parameter of 1 while the slowest can be as slow as 140 units of time. This parameter describes the way in which state leaks out of this component. One thing is common among the categorisers, they do not make use of slow acting neurons, but only fast-acting ones. Whereas the generalisers make use of both.
For the purpose of determining the role of fast versus slow acting components in the circuit in this memorisation task, the best evolved agent's time parameters were studied and from it we defined what slow acting neuron ranges and what fast acting neuron ranges where needed to solve the task. Slow acting then corresponded to neurons with time parameters between (50 and 100) while fast acting nodes where defined to be between (1, 7).
To observe the role of this different components I am running evolutionary runs while the circuit is constrained to have different numbers of fast/slow neurons. I am using a 10 node CTRNN so there are 11 cases. In one extreme is the case when all neurons are restricted to the fast range, then there is the case when one is restricted to be slow and the rest restricted to be fast, and so on until all neurons are restricted to be slow. I am running 10 evolutionary runs using different seeds for each case. The intuition is that neither of the extremes do well, and that the optimum for the learning task lays somewhere in-between, possibly with more fast neurons than slow neurons. The resulting curve should be interesting.
On the other hand, I have not managed to evolve circuits that can continue to remember the feature after they have had to make a first decision. I am working on changes to the current evolutionary methodology that will lead to success here. One of the changes that I will try might be to add a feedback input (I have to explain why I think this is important but will do so later on). Also I must simply wait longer (more generations) as well as try larger circuits (currently the biggest has been a 10 node one).
On a completely different note, I have started to design the experiments in the embodied version of this memorisation task loosely inspired on behavioral plasticity observed in C. elegans. A similar form of learning to imprinting has been observed in these organisms. In particular, it has been observed that the animals that were cultivated normally with food at temperatures ranging from 15C to 25C migrate to the cultivation temperature on a temperature gradient and move isothermally at that temperature. By contrast, the animals migrate away from the temperature at which they were previously starved. They don't call it imprinting, but it is a very appropriate behaviorual paradigm to continue to study learning and memory in the embodied version. I will say some more on this soon.