Category:Medicine and Neuroscience
From EuroVR Knowledge Base
Contents |
Brief overview of area
The scope of WG 2.11 “Medicine and Neurosciences” research roadmap has, as the name already implies, two parallel streams of interests: neuroscientific oriented VR and medically oriented VR. In the “Terms of Reference” of WG 2.11 both research fields and the scope of the WG are described in detail. Mainly based on the research interest of the WG but also encouraged from the feedback of invited external research labs and companies, the research topics, “Neuroscience and Neuroscientific oriented Brain Computer Interfaces”, “VR for Diagnostic and Rehabilitation”, “VR for Medical Training/ Interaction and Interfaces” and “Distant and remote collaboration tools” have been identified as focus points.
Vision and potential scenarios of use
Neuroscience & Neuroscientific oriented Brain Computer
The following example is a visionary scenario of what we could face in a VR lab in maybe 10 to 15 years if we are able to combining VR with brain research:
Prof. Dr. Bunsen Honeydew is the managing director of a DNA laboratory. He and his assistant and volunteer Beaker are working together, discussing and evaluating a virtual model of a DNA structure in a highly immersive virtual environment. They are surrounded by very complex visual information. All they wear is a lightweight breathable hat with integrated dryskin electrodes for EEG, EOG, etc. The sensors are based on carbonanotubes as currently developed in the IP “SENSATION”.
They interact with the DNA structure with gestures and speech. The interaction devices they use, the devices for the gesture recognition and interaction techniques don’t hinder the two researchers with data gloves, tracking, mechanics and electronics because just grasping with the hand or the simple imaginary of grasping is introduced with Rhythms of the brain which are associated with those cortical areas most directly connected to the brain’s motor output (measured via EEG). The interaction devices they are using for additional specific interaction are only designed to be most comfortable for the user – not to carry heavy batteries, cables or electronics. Obviously it is not just the navigation as common interaction technique which benefits form the brain signals detection, but also many other interaction techniques. For example there are numerous situations in which it is important to know exactly where Prof. Dr. Bunsen Honeydew is looking at in the VE. During the design of a new DNA for example it is of great advantage to know which part of the DNA Prof. Dr. Bunsen Honeydew is talking about i.e. where he is looking at. As soon as he is looking at a certain part of a model it is highlighted so that it is clear to Beaker, which part exactly is in the centre of interest. At the same time the highlight gives a positive feedback to the Professor. This behaviour is realised by measuring steady-state visual evoked potentials (SSVEP) of the brain. Combining the visual stimulus i.e. the flickering of a virtual object at a certain frequency (which is just not perceivable) and the SSVEP measured with unobtrusive sensors (integrated in the hat for example) this technique not only recognizes where Prof. Dr. Bunsen Honeydew is looking at, but it is also used to define where the point of interest of him is at that moment - and in this way, focus adaptive optics maximize the level of detail at the point of interest!
Even as the Professor is going to do a mistake during his interaction with the environment, it does NOT affect the virtual DNA structure or his assistant Beaker! By measuring Ne (Error related negativity) and N2 (conflict related negativity) with the electrodes integrated in his hat and by evaluating the signals in real time, the mistake he is going to do is prevented because his brain signals which are clearly indicating that he will do this mistake are measured just some milliseconds before he is actually doing the mistake.
The very interesting point here is, that both researchers do not suffer from motion sickness, headache or fatigue even after a 4 hour session because the simple hat they are wearing objectively detects their stress, fatigue, workload and information overload of their senses. The system has for example detected at an early stage that Beakers’ information management capacity – especially the visual sense – is overloaded. Therefore it automatically switched to more audio information instead of the visual information and at the same time calming him down with a relaxing voice.
By feeding the physiological parameters to the VE, the VE reacts to these parameters and provides adaptive interfaces and environments to Prof. Dr. Bunsen Honeydew and Beaker. The system is supported by Knowledge Based Systems which enhance the intelligent, adaptive and personalized behaviour of the VE. During the sampling of the behavioural data, a recognition sequence starts which compares anticipated behaviour with the actual data set. With motion and gesture recognition a set of data points are generated as a set of behavioural data.
Unlike in earlier times where Beaker suffered from motion sickness, information overload or being blown up, he can now work in a user friendly environment.
Virtual DNA modeling and analysis with BCI
VR for Diagnostic and Rehabilitation (in Aeronautics)
Psychotherapy has become a big research and application field for VR in the last decade. One newly defined field for “Cyberpsychology” is aerospace and aeronautics. Of course, flying simulators have been around since a long time. Now, in preparation for the ISS mission or the Mission to Mars vivid virtual scenarios for exposition, training, and monitoring could be of great use. For example the preparation for the experience of solitude, the training of difficult technical scenarios or even interactive communication tasks could be implemented on a space ship/ station. Of course all scenarios could be of use at any exposition and training application “on earth”.
Situated immersive environments for psychological extreme situations
VR for medical training/ Interaction and Interfaces
The challenge in virtual human modeling is to achieve the representation of the main human characteristics with as much realism as possible. Such achievements would enrich the understanding of the human body and behavior. Simulation is especially useful to derive information from the models so as to predict and/or reproduce real situations. Computer methods in visualization and simulation have thus great potential for advances in medicine.
Virtual Reality technology is especially suited for the simulation of endoscopic operations, as the surgeon has neither visually nor manipulative direct contact with the operation site.
The idea of using computer-based surgical simulators for training of prospective surgeons has been a topic of research for more than a decade. However, surgical simulation (for example Bone Drilling medical training system, Munich Knee Simulator) is still far from being included into the medical curriculum.
Virtual human model for medical research and training
Integration of endoscopy in medical training
Distant and remote collaboration tools
The following examples is a visionary scenario of what we could face in an operating theatre in maybe 10 to 15 years.
Dr. Crusher enters the medical “holodeck” and meets his colleagues. Today they have a joint activity among two teams; one located on the Queen Mary III in the south pacific the other one in the brain-cancer research institute in Zürich. They have to analyse the complex medical image derived from multi-slice computed tomography of a passenger. Each member of the two teams has a high resolution, correct spatial perspective view (without wearing any special glasses) onto the virtual brain. They are able to discuss while at the same time interact with a model of the patients brain. They are planning a session of the surgical intervention to be done on the ship. Since they can share opinions and expertise among different specialists this is not a very risky situation. The surgeon describes the procedure he intends to do followed by simulating the various steps. Each of the assistants and experts on the ship give their opinion and some of them interact by proposing alternative approaches. A 3D reconstruction of a computed tomography displayed on a large auto stereoscopic 3D display gives better information on the spatial relations. The surgeon, instead of reconstructing the overall image by visualizing in sequence a series of CT slices or cross-sections, can see the images - all at once - combined into an entire series in 3D space and thus benefits from easier interpretation, more accurate identification and more precise location. The interaction is immediate and natural. Since the 3D visualization system provides a spatial, interactive view for simultaneous collaboration by multiple viewers on spatially complex work, resulting in faster and more accurate comprehension and outcomes, the radiologist can understand the exact location of the tumour on the CT scan before the surgery treatment.
Telesurgery planning of a brain tumor
Useful links
- Position available - Diploma Thesis: "Responsive Virtual Elevator". Requirements: Student of Medical Engineering or Electrical Engineering or similar studies. On the job training: LabVIEW; Basics in Virtual Reality; Physiological Measurement (EKG, GSR); Realisation of a Closed Loop System: A Virtual Elevator controled by physiological reactions of a user. Contact: Oliver Stefani ols@coat-basel.com
Pages in category "Medicine and Neuroscience"
The following 4 pages are in this category, out of 4 total.
