Brain to Computer: The aim of the project Brain2Robot is to develop an arm prosthesis, which is controlled through the power of thoughts to give back a part of independence to the patients.
People paralyzed as the result of an accident or a serious illness are reliant on the help of others in many situations. The aim of the EU funded Brain2Robot project is to develop a prosthetic control system based on intended movements. The idea is to help patients regain a certain degree of independence. To this end, the Brain-Computer Interface (BCI) developed at Fraunhofer FIRST is combined with an eyetracker, which determines from the direction of the patient’s gaze where the prosthesis (e.g. a robot arm) should move to. The movement is initiated by a signal from the BCI.
To start the system, an eyetracker first determines the direction in which the robot arm should move. The direction of the patient’s gaze is monitored by two cameras mounted on a specially designed pair of glasses. First, the exact position of the pupils is captured in stereo (stereo eyetracking). In addition, a headtracker determines the position of the head. A software component analyzes the two systems’ data and derives from this the intended direction of the movement. The test person sees an object, looks at it, imagines moving his/her arm, and the robot grasps the desired object.
Triggered by Thought
The actual movement of the arm is triggered by a signal from the Brain-Computer Interface. Researchers at Fraunhofer FIRST and Berlin’s Charité have been working for some seven years now on the development of the Brain-Computer Interface using a conventional electroencephalogram (EEG) of the sort employed in routine clinical procedures. Electrodes attached to the patient’s scalp measure the brain’s electrical signals. These are then amplified and transmitted to the computer. High-efficiency algorithms analyze these signals using machine-learning methods. They are capable of detecting changes in brain activity triggered by the purely mental conception of a particular behaviour. They can, for instance, unequivocally identify patterns reflecting the idea of moving the left or right hand and extract them from the many millions of neural impulses. They are then converted into control commands for the computer, enabling one to choose, for example, between two alternatives. Here – and this is the Fraunhofer researchers’ particular achievement – the main learning task is performed by the computer. While in other BCI groups the test persons learn to adapt their brain signals so that the computer can recognize them, this task is performed at Fraunhofer FIRST by the algorithms. This enables the test persons to operate a computer using the BCI system after training for only a short period (approx. 30 minutes).
In the Brain2Robot project, the pattern reflecting the idea of moving the right hand is used to set the robot arm in motion. The signal for the left hand triggers a certain action of the arm, e.g. grasping or lifting a coffee cup.
Application Areas
In developing the Brain2Robot, the focus has been on medical applications, in particular the control of prostheses, supportive robots or wheelchairs. Another development to emerge from BCI research is the mental typewriter, a communication device enabling very severely paralyzed patients to select letters and write texts. But other application areas are also ultimately conceivable, e.g. the use of BCI technology to control computer games or in automotive safety systems (driver monitoring, driver assistance).
Marie Curie Excellence Grant
The Brain2Robot project is being funded within the EU 6th Research Framework Programme by means of a Marie Curie Excellence Grant (EXT). Florin Popescu Ph.D. was awarded with the EXT in 2004 and is currently Team Leader of Brain2Robot at Fraunhofer FIRST. EXT grants are designed to help identify and promote scientific excellence. They make it possible to establish excellence teams and conduct ambitious top-class research in all scientific disciplines of relevance to the EU. 108 Marie Curie Excellence Grants, worth a total of EUR 159.7 million, have been awarded so far.
Other Partners and Sponsors
- Charité - University Medicine Berlin, Neurophysics Group
- Technical University of Berlin, Institute for Software Engineering and Theoretical Computer Science
- The development of BCI is being funded by the Federal Ministry of Education and Research (BMBF)
Contact
Fraunhofer Institute for Computer Architecture and Software Technology FIRST
print
Brain2Robot
