Argonauts:Independent Component Analysis
From Wasteland
Contents |
Sat 18 November 2005
Spatio-temporal coordinates
WMVL, 5.00pm to 7.00pm (exactly) + 15 min for conclusions.
Attendees (in alphabetical order)
- Pablo Barrera González.
- Ramón Casero Cañas.
- Mike Kadour.
- Rohan Loveland.
- Rama Aravind Vorray.
- Catherine White.
Minutes
Instead of Level Sets (II) we decided to do Independent Component Analysis (ICA), so that Olivier and Mike K. have time to catch up and new people can join the crew. Initially I would suggest this paper for Saturday:
A. Hyvärinen and E. Oja. Independent Component Analysis: Algorithms and Applications. Neural Networks, 13(4-5):411-430, 2000.
I would say that this is the hardest topic we have ever chosen. ICA is the most common way of solving the Blind Source Separation problem, and it goes against a mighty notion in this field: Gaussianity is good. So one has n signals that are linearly mixed: X = A·S. Here we know X, but we don't know either A or S.
If the signals in S are Gaussian random variables, ICA does not work. Otherwise, we can go ahead and most interestingly, work with each signal X_j separatedly. The main idea here is that mixing non-Gaussian signals makes them more Gaussian. So if you want to recover the original non-Gaussian signals, just look for the linear combination A that makes S most non-Gaussian.
To solve the problem there is an algorithm called FastICA, which basic outline is provided in p. 423 of the paper. Let w be a row in A, i.e. X_j = w_j·S.
- For each X_j pick a random w_j.
- Compute the gradient of a function that gives you an estimate of how non-Gaussian each S_j is, given that w_j.
- Upgrade w_j in the direction of maximum non-Gaussianity.
- Iterate until convergence.
How to compute lack of Gaussianity is a big issue, so we wouldn't go into detail even if we knew how.
The meeting didn't go without anecdotes. First the Argonauts welcomed Pablo Barrera, a Spaniard who is reading for his PhD as a visiting student in London. Given that we had a guest, we went fully clothed to the meeting (this time).
Saturday is a bad day for meetings, because the lab is full of hard-working people and we don't want to interrupt their web-surfing. So we set up a table, the whiteboard and the rack by the photocopier, in the IEB's atrium. This is truly X-Science (Extreme Science).
The most amusing thing ever happend. We had a member who came late, took a copy of the paper, sat down and fell asleep straight away. After a while he woke up and left. Mysterious, innit? Our guess is that said member is a vigilante and had spent the night fighting crime. He then went to the batcave, changed into civilian clothes and came to the meeting to protect his secret identity. But although crime never sleeps, vigilantes need to take a nap now and then.
It is also worth mentioning that we have boldly improved the wiping system of the whiteboard. Somebody noticed that there is no need to use the same paper napkin we had since last Summer, and that was soaked in ink. So emotional attachment notwithstanding, we fetched a clean one and (sigh) threw the old one away. Such is life.
--
