Neural Network Analysis

Introduction

The main aim of my network analysis research is to develop better ways of making predictions about consciousness as part of work on neuro- and synthetic phenomenology. Network analysis techniques can also help us to debug complex robots that learn from their experiences. My network analysis research focuses on representational states, the relationships or integration between representational states and predictions about consciousness according to different theories.

Representational States

In my work, a representational state is defined as a part of the system whose states co-vary with the data entering or leaving the system, or a part of the system whose states co-vary with another internal part of the system. In a neural network the identification of representational states is useful because it can tell us what firing patterns actually mean: the ultimate goal would be to analyse a network in a particular state and know exactly what information is being actively processed by each neuron. With better scanning technology we would be able to make precise predictions about what is going on inside another person's brain.

A good example of representational states are the neurons in the cat visual cortex that were identified by Hubel and Wiesel (1959). In these experiments electrodes were inserted into the brains of cats and the neural activity was measured as different stimuli were presented in different parts of the visual field. Neurons whose activity changed when the external stimulus was presented were judged to be representing the information in the stimulus. More recently a similar approach was pursued by Quian Quiroga et al. (2005), who implanted electrodes into human subjects and measured the response of neurons to pictures of individuals, landmarks or objects. They discovered that some neurons had higher level representational properties: responding to both pictures of an individual and to the written name of that individual - for example, the "Bill Clinton" and "Jennifer Aniston" neurons.

During my PhD I identified representational states in an artificial neural network by injecting noise into layers with known response characteristics and measuring the mutual information between neurons in the injection layers and neurons in the rest of the network. This enabled me to identify the extent to which neurons were representing combinations of sensory and motor information.

The relationships between a system's representations also have a significant effect on its perception. For example, split brain patients have independent sets of representations in each half of their brain with little integration between them, leading to two separate minds within the same brain. The relationships between mental states can be identified using methods for measuring functional and effective connectivity, such as Granger causality (Seth et al., 2007), neural complexity (Tononi, Sporns and Edelman, 1994) or information integration (Balduzzi and Tononi, 2008). In collaboration with Igor Aleksander I have developed a new approach based on liveliness (Aleksander, 1973), which will be described in detail in a forthcoming paper.

Information Integration and Liveliness

Information integration is a property of systems of connected elements that expresses the extent to which they are capable of entering a large number of states that result from causal interactions between their elements. For example, a digital camera sensor with a million photodiodes has low information integration because none of its large number of states result from causal interactions among the photodiodes. A system consisting of a million lights controlled by a single switch has a high level of integration between its elements, but a low level of information integration because it can only enter a small number of states (all lights on; all lights off). The mammalian brain is an example of a system with high information integration because it can enter a large number of different states, and each of these states results from causal interactions between the neurons (see Figure 1).

Figure 1. Pseudo graph comparing the balance between differentiation and integration in different systems

A number of people have suggested that there is a link between information integration and consciousness or that information integration actually is consciousness (Tononi, 2008), and a number of algorithms for calculating information integration have been put forward. One of the most recent of these algorithms is the state-based measure of Φ, which was developed by Balduzzi and Tononi (2008). This algorithm can identify the areas of maximum information integration for each firing state of the system, but its factorial dependencies severely limit the number of neurons that can be analyzed. To address this problem I worked with Igor Aleksander on an alternative measure of information integration known as liveliness, which was based on earlier work by Aleksander (1973). When the causal model of the system is known, both liveliness and state-based Φ can be used to plot the changes in information integration over time, and they could be used to make predictions about the moment to moment consciousness of a system. A paper comparing the accuracy and performance of state-based Φ and liveliness is currently in preparation. The SpikeStream plugin that we developed to analyze systems for liveliness is illustrated in Figure 2.

Figure 2. SpikeStream liveliness plugin

References

• Aleksander, I. (1973). Random Logic Nets: Stability and Adaptation. International Journal of Man-Machine Studies 5: 115-31.
• Balduzzi, D. and Tononi, G. (2008). Integrated information in discrete dynamical systems: motivation and theoretical framework. PLoS Comput. Biol. 4(6).
• Hubel, D.H. and Wiesel, T.N. (1959). Receptive fields of single neurones in the cat's striate cortex. Journal of Physiology 148(3):574-91.
• Quian Quiroga, R., Reddy, L., Kreiman, G., Koch, C. and Fried, I. (2005). Invariant visual representation by single neurons in the human brain. Nature 435: 1102-7.
• Seth, A.K. and Edelman, G.M. (2007). Distinguishing causal interactions in neural populations. Neural Computation 19(4):910-33.
• Tononi, G. (2008). Consciousness as Integrated Information: a Provisional Manifesto. Biological Bulletin 215:216-242.
• Tononi, G., Sporns, O. and Edelman, G.M. (1994). A measure for brain complexity: Relating functional segregation and integration in the nervous system. Proc. Natl. Acad. Sci. USA 91:5033-7.