2.2 Software agents
2.2.4 Defining an agent
As stated in the previous sections we are not able to get a clear picture of what defining properties an agent should have. It is however clear that it ultimately should be a tool for humans. The agent or the multi-agent system, must there-fore be able to handle both the human context paradigm and the often vast amount of high dimensional data in the virtual environments.
When speculating about an agent’s most advanced form, it should be able to adopt to any given task. This calls for adaptiveness, like we see in living organ-isms. Also, agents should be able to form communities in order to solve larger tasks and parallel tasks. This is in principle what the artificial intelligence and artificial life community are trying to solve from their own perspective[29, 95].
As the software agent community evolves together with the virtual environ-ments, we suspect that agents will evolve into virtual living and intelligent en-tities. In order to serve humans they need to learn reasoning and meaning in a human context. We will not speculate on the possibility that software agents can evolve further than humans1, or if they will become conscience at some point, since this is long past the topic of this thesis.
Thus, as presented in the introduction, we aim to investigate the use of the inde-pendency criteria for the use in VE tools, given that it should reflect separation properties natural in a human sense. For this we use independent component analysis. We therefore turn our attention to the ICA algorithm in the next chap-ter, regarding its properties and framework.
1It is not clear in what regards the human paradigm is optimal.
CH A P T E R
3
Independent component analysis
Achieving blind source separation (BSS) with independent component analysis (ICA) is a fairly new and fast growing field. In BSS the word blind refers to the fact that we do not know how the the signals were mixed or how they were generated. As such, the separation is in principal impossible. Allowing some relatively indirect and general constrains, we however still hold the term BSS valid, and separate under these conditions.
A classic problem in BSS is the cocktail party problem, as shown in figure 3.1.
The objective is to sample a mixture of spoken voices, with a given number of microphones - the observations, and then separate each voice into a separate speaker channel - the sources. The BSS is unsupervised and thought of as a black box method. In this we encounter many problems, e.g. time delay between microphones, echo, amplitude difference, voice order in speaker and underdetermined mixture signal.
At seminar work in 1986, Herault and Jutten pursued the idea that the sepa-ration could be done by reducing redundancy between signals, in a artificial neural network like architecture. This approach initially lead to what is known as independent component analysis today. The fundamental research involved
BSS
Voice 1 Voice 2 Voice 3
Voice 7
Figure 3.1 The figure illustrates the cocktail party problem. Seven people are talking and the BSS task is to separate each of the speakers voices, without knowledge of the mixing or generation of the voices.
only a handful of researchers up until 1995. It was not until then, when Bell and Sejnowski [7] published a relatively simple approach to the problem named infomax, that many became aware of the potential of ICA. Since then a whole community has evolved around ICA, centralized around some large research groups1and its own ongoing conference, International Conference on indepen-dent component analysis and blind signal separation[80]. ICA is used today in many different applications, e.g. medical signal analysis, sound separation, image processing, dimension reduction, coding and text analysis.
In ICA the general idea is to separate the signals, assuming that the original underlying source signals are mutually independently distributed. Due to the field’s relatively young age, the distinction between BSS and ICA is not fully clear. When regarding ICA, the basic framework for most researchers has been to assume that the mixing is instantaneous and linear, as in infomax. ICA is often described as an extension to PCA, that uncorrelates the signals for higher order moments and produces a non-orthogonal basis. More complex models assume for example, noisy mixtures[72, 34], nontrivial source distributions[52, 97], convolutive mixtures[4, 63], time dependency, underdetermind sources[68, 41], mixture and classification of independent component[64, 57]. A general introduction and overview can be found in [62].
In the following we will look at the properties of ICA, and present the ICA
1e.g. at Computational Neuroscience Lab lead by Terry Sejnowskis[22], Laboratory of Com-puter and Information Science at Helsinki University of Technology lead by professor E. Oja[38]
and TSI Department Signal-Images lead by J. Cardoso[94]
3.1 Model 15
algorithms used in this text. Finally we address the topics of preprocessing with PCA, and model selection using the Bayesian information criterion.
3.1 Model
The general model for ICA is that the sources are generated through a linear ba-sis transformation, where additive noise can be present. In this text we consider the model to be,
X=AS+ ; X
whereXis the matrix holding theNmmixed or observed signals in each row with N samples, A is the NmNk basis transformation or mixing matrix, and S is the matrix holding theNk independent source signals in rows of N samples. The noise is added by theNm
N matrix that is generally defined to be Gaussian or fully neglected.