Reading brain activity could help in forensics and communicating with those who can't

A mind trapped inside a helpless body?

Eight years ago Leonard Rodrigues suffered a massive heart attack in his sleep. His wife, Winifred, woke and immediately realized something was wrong. She called 911 and Leonard was rushed to hospital where emergency surgery repaired his heart. Unfortunately, there was no surgery to repair the other damage that had been done. Lack of oxygen to his brain had left Leonard Rodrigues in a persistent vegetative state. In this condition, patients are apparently awake, but appear to be completely unaware — or at least if they are aware, they can't communicate it. Their consciousness may persist, trapped in their unresponsive body, but they have no way of signalling that to the outside world. Winifred Rodrigues is convinced her husband's mind is active and aware — if only we could find a way to reach it.

Techniques to do this are on the horizon, as new developments in extracting information directly from the brain could transform the lives of patients like Mr. Rodrigues. Scientists are exploring brain-scanning technologies that can look past the body, and directly at activity of the brain. This could provide a channel of communication for people with no other options.

Reading images from a mind using brain scanning

Researchers are coming at this problem from two directions. Dr. Adrian Nestor, from the psychology department at the University of Toronto Scarborough, has been working on using technologies to "read" visual images from the brains of healthy volunteers. "I've always been passionate about trying to understand what people truly see when they look at something, and what they remember when they think they remember something, and I think we're finally in a position to answer some of those questions," he says.

He has previously successfully used functional Magnetic Resonance Imaging (fMRI) to analyze brain activity and translate it into an accurate representation of what his volunteers are looking at — faces in his experiments. In new work he's accomplished the same task using EEG, or electro-encephalography. EEG is a much less complex, expensive and cumbersome way to get signals from the brain, but before Dr. Nestor's work, it wasn't clear that it could extract enough information from the brain's electrical activity to do tasks like reconstruct images.

• Paper on EEG and image reconstruction in the journal eNeuro

Dr. Nestor's latest work shows that EEG, in fact, has much more potential for this than previously thought. Part of the key to this work was a machine learning system that, through many trials, can develop a mapping algorithm to translate the EEG signals into a recognizable representation of the face that the volunteer subject is looking at. He says once he's improved this technology, he can imagine using it in lieu of police sketch artists with eye-witnesses.

"Being able to reconstruct faces has a lot of immediate applications. For instance, for forensic purposes," he says.

From a theoretical perspective, Dr. Nestor hopes that in time this will give us new insights into how the brain represents images. On the more practical side, this might also be a path for helping people who can't communicate. He's working on a system in which people might be trained to think of the shape of a letter, which could be read from the brain by the EEG and computer algorithm, and spell out words. He's also currently testing it to see if they can reconstruct entire words. "That kind of neural-based communication could be very important," he says.

Making contact with brain scanning

Dr. Adrian Owen has been approaching this same goal more directly by working with patients who are in a vegetative state. More than a decade ago, he demonstrated that contrary to most medical opinion, some patients in this condition are, in fact, aware of their surroundings. He used fMRI to monitor areas in the motor cortex of the brain, and asked these apparently unresponsive patients to think about particular kinds of movements. By detecting brain activity associated with imagining these movements, Dr. Owen was able to communicate with people who hadn't had any interaction in years.

That communication was severely limited. The technology could only interpret simple "yes/no" answers, and it took several minutes for each answer to be read from the scan. Dr. Owen is currently working on developing new strategies that are more flexible, and less cumbersome and expensive than fMRI. One is EEG, but another is a new technology called functional Near Infrared Spectroscopy (fNIRS), which uses head-mounted infrared lasers and sensors to read brain activity.

Each technology has its strengths and weaknesses, says Dr. Owen. "Ultimately it's probably going to be a combination of all of these techniques that gives us the answers we want," he explains.

Leonard Rodrigues has been a research subject in Dr. Owen's work. Unfortunately Dr. Owen hasn't been able to use his fMRI techniques to reach and communicate with him. So far, in fact, the fMRI technique is only reaching about 20 per cent of the patients it's been tried on. There's no way to tell if this is because of a failure in the technique, or because the unresponsive patients aren't actually conscious and aware.

Winifred Rodrigues thinks, however, that it's just a matter of time before her husband starts communicating. She says she knows he can hear her — and that she sees subtle signs of him responding to her. "Having the research done by Dr. Owen, and the ability that he will find ways to connect, will be a huge validation for our family and others," she says.

She looks forward to the day when contact with her husband is made. "It will help others communicate with Leonard," and open up his world, she says.