Wednesday, March 15, 2006

results from noise-less experiments: success

This post will describe results from 20 evolutionary runs using 5 and 10 node CTRNNs (10 runs each). Each evolutionary run uses a different seeds and stops after 5000 generations [roughly around 11 hours in the cluster]. The experimental set-up is as follows: [1] parents and test individuals are picked at random from a uniform distribution (no incremental beta distribution), [2] the random initialisation has been taken off - CTRNNs now begin from the same state, [3] there is a fixed delay at the start and between presentations - no longer is there a random component, [4] Gaussian-weighted evaluation is used, [5] traditional gen-phen mapping is used and [6] number of trials per fitnes went up from 100 to 200. The main idea of these experiments will be to see whether I can evolve agents for the continuum irreversible learning task without noise. There is still a lot of inter-trial variability.

Most (7/10) runs with 5 node CTRNNs end up categorizing ‘just’ two signals – indifferent to the differences between the ones in-between. Some make very strict categories from the inputs, others less. See this figure for an example of this kind of 'discrete' learning performance. As in this blog you will encounter this particular type of figure all over the place I will provide once the explanation of it. The vertical axis represents the different ‘parent’ signals that the agent can receive. The horizontal axis represents the different ‘test’ signals. Notice that both of these axes provide information about the continuum of that feature. The line along the diagonal, then, represents the trials where the parent and test individual are the same. The colour shading represents the agent's decision to accept as their parent or not, the test individual. The colormap goes from blue (i.e. 'this is not my parent') to red (i.e. 'this is my parent').

Out of the 5node runs, number 10 did much better than the rest. See its performance here in this figure. The activation of the nodes and the input for several trials can be seen in this figure for 10 random trials. You will probably see this style of figure often as well around here so I will explain it once first. The figure shows all ‘neuronal’ activations over time. The signals at the bottom represent the input presented to the circuit at that particular time. The vertical dashed lines represent the end and beginning of a different trial. At the beginning of each trial the circuit’s state is re-initialised. The activation in the shaded area is the ‘designated’ output node and the black boxes around the end of each trial represent the evaluation period as well as what the ‘correct’ of the output node should be for the previously presented individuals. This five node CTRNN is using all of its nodes to generate this behaviour, but the ‘strategy’ we cannot tell from this view alone. We can see that the agent makes mistakes, in for example, the 5th trial, where two different individuals are presented but the agent classifies them as the same. We can also appreciate from that figure, nevertheless, the relative proximity in ‘appearance’ of these two individuals.

Out of the 10-node runs, 2 did particularly well [runs 3 and 9]. I will show here their performance (3, 9) and the circuit’s activation for 10 random trials (3 and 9), but I will not analyse them much further yet.

It is likely that the agents use the transients and the precise timing of the presentations and delays in this particular scenario. Experiments introducing delays in the in-between periods for these best evolved agents show that, delays do affect their performance (see for example how the performance shifts as delays are introduced in one of the 10 node scenarios in this figure). One question of interest is whether we can re-evolve the successful agents to cope with random delays? The same applies for the random initialisation of the state of the circuit? If we can, then an incremental approach to noise would be rather useful.


Post a Comment

<< Home