Controlling robots with your mind
Nov 8, 2008, 12:15p
It's the stuff of which sci-fi dreams are made. Mind control, well, not of another person, but of a robot. Want your laundry done? Tell your robot to do it. But not just tell it, think it.
That's the idea, anyway.
Before I applied to grad school, this stuff really intrigued me. While working at Google, so often I remember thinking, "If only I could just think my emails, instead of having to type them, I would be so much more efficient." I could send email while I drove, or while walking from my car to my apartment, or while sitting on the shitter. Anything to be more productive...
While productivity as a goal has its own issues (described a bit in a previous post Why I believe in God), let's put those aside for now. The idea that thinking alone, without any muscle activity, could cause something to happen is pretty cool. A couple years ago, I described a device called BrainGate that let a paraplegic control a cursor on a computer screen with his brain. The company that produces these devices is called Cyberkinetics. How does it work? Basically, a grid of electrodes is implanted on the surface of the brain, specifically above a cortical area known to be involved in muscle control. Through trial and error on the person's part and learning algorithms on the software's part, the person is able to learn how to change the electrical activity detected by the electrodes to move the mouse cursor where he wants it to be. To be clear, this type of technology doesn't tell us much about how the person thinks, or how the brain works. Rather, it's an example of how you can take an arbitrary signal from the brain and interpret it to achieve an outcome (moving the mouse). As an analogy, think about typing. We learn to move our fingers so that words come out, but watching our fingers move doesn't tell us much about how our brain produces language. Instead of moving his fingers, the paraplegic is changing the electrical activity in his head. That in its own right is interesting, but this is more of a medical device engineering problem than a neuroscientific one. Now that I'm in grad school, I've lost interest in working on this problem, though I still find the results to be really cool.
One thing to note is that the electrode grid tends to move over time. So the long-term medical value of this device is still dependent on further engineering innovation.
Anyhow, I ran across a recent paper that takes BrainGate off the computer screen and brings it into the real world. Here, instead of controlling a cursor on a computer screen, monkeys were trained to control a robotic arm to grab food. How does this work? The robotic arm has 5 degrees of freedom (3 at the shoulder, 1 at the elbow, and 1 at the hand, i.e. how much to open/close). However, the monkey's brain is not being used to control these 5 variables directly. Rather, the monkey's brain is being used to identify a point in 3-D space that the monkey wants to reach to, and software then figures out how to get the robotic arm to move to that point in space (this is a slight over-simplification, but it's the overall idea).
The algorithm the researchers used to determine the point in 3-D space is surprisingly simple. The electrode grid records from roughly 100 neurons, and measures the firing rates for each neuron when food is presented to the monkey at a specific location in space. Based on the location associated with the highest firing rate for each neuron, a position in space can be defined by averaging over the actual firing rates of a collection of neurons, each with a different preferred direction.
Let's look at a simple example. If food is presented in the left corner of the field of view, neuron #1 may fire 5 times per second (5 Hz), while neuron #2 may fire at 10 Hz. If food is then presented at the right corner, neuron #1 may fire at 10 Hz and neuron #2 may fire at 5 Hz. Based on this "training" information, the algorithm then determines that neuron #1 encodes a vector pointing to the right corner (neuron #1 fires more when food is there), and neuron #2 encodes a vector pointing to the left corner (again, neuron #2 fires more when food is there). After the software has learned this relationship, it can make general predictions about the location of the food based solely the neural firing rates it has access to. Now, let's say that the food is presented in an unknown location, and neuron #1 fires at 7.5 Hz and neuron #2 fires at 7.5 Hz. The algorithm will interpret this to mean that the monkey wants to move the arm to the center, because this is in the middle of the range of firing for both neurons. Hopefully that makes sense - sorry if it's confusing!
Anyhow, I think that's enough technical details. On to the movies!
Sachin
- Nov 9, 2008, 9:50p
Movies hosted on Posterous.com
http://nikhil.posterous.com/
Garry
- Nov 14, 2008, 11:36a
Wow, this is remarkable work.
« How I came to believe in God
-
You have 2 time travels: Choose wisely »
|