A new use for touchless technology in the operating theatre
Microsoft Research, created by Microsoft Corp. in 1991, has more than 1,000 scientists and engineers working across multiple research areas in research labs worldwide, making significant product contributions and collaborating on external projects that help solve global challenges.
The Challenge
Surgeons operate in a challenging environment where they are required to maintain sterility at all times. Re-scrubbing is time-consuming, therefore surgeons are frequently compelled to instruct others to manipulate visual-aid equipment for them, an often impractical and imprecise method. The organisation wanted to explore the use of touchless interaction within surgical settings, enabling surgeons to view, control and manipulate medical images without contact.
Expertise Sought
- Software development
- Experience with touchless interaction technology
The Solution
The School of Computing and Communications at Lancaster University has research expertise in communications and networking, computer systems, intelligent systems, software engineering, and human-computer interaction.
A collaborative team, including Dr Mark Rouncefield from the School of Computing and Communications, Guy's and St Thomas' Hospital, Microsoft Research, and King’s College London, piloted the touchless interaction component using Kinect for Windows hardware and the Kinect for Windows Software Development Kit (SDK). This gesture-based system allowed the vascular surgery team to maintain a sterile environment, whilst being able to view and manipulate medical images through a combination of gesture and voice control. It has also been extended to applications within neurosurgery.
The computer program visualises on-screen the patient’s 3D anatomy, which is acquired from a group of 2D images (which look like x-rays) taken at different view directions. The Kinect technology allows the surgeon to manipulate (e.g. rotate, pan and zoom) the medical imaging system by themselves, rather than instructing an assistant to do so.
Cost
The research was funded by the Research Council.
Impact
Guy’s and St Thomas’ NHS Foundation Trust in London successfully piloted the technology in vascular surgery procedures in an operating theatre. The team from Lancaster University and Microsoft Research also partnered with neurosurgeons at Addenbrookes NHS Foundation Trust and Cambridge University to apply the technology to the manipulation of 3D volumetric models of the brain for neurosurgery. The newer system was also piloted in operating theatres. The project was a finalist for the Innovation & Technology Award at the 2014 Biomedical Awards, hosted by Bionow.
Benefits to the company
- Development of the touchless interaction component using Kinect for Windows hardware and the Kinect for Windows Software Development Kit
- Potential to create a marketable product
Benefits to the university
- Increased the university’s knowledge of contactless interaction software and its uses in surgery
Benefits to society
- Potential increased sterility in vascular and neurosurgery, which could save time and lives
Company Feedback
"Adapting the technology for neurosurgery has allowed us to understand how the system works across different surgical domains. As well as refining the gesture set, the new system incorporates enhanced voice control that enables the surgeon to control the system using only voice, leaving both hands free to work with surgical instruments." Professor Kenton O'Hara, Microsoft Research.
Researcher Feedback
"This project on 'Touchless Interaction' has successfully combined the skills and knowledge of social and computer scientists with the professional experience of surgeons, to design and develop an application that is already proving of real benefit in the operating theatre.
"This is a lovely example of a successful interdisciplinary research project, combining the technical skills of computer scientists with a social scientific and medical expertise that ensures the new technology resonates with the way in which surgeons actually do their work." Dr Rouncefield, School of Computing and Communications, Lancaster University.
Mr Tom Carrell, a vascular surgeon at Guy's and St Thomas', said, "This technology is very exciting as it allows me to easily and precisely control the imaging I need during operations. Touchless interaction means there is no compromise in the sterility of the operating field or in patient safety."
Awards
The team made the finals of the 2014 Biomedical Awards, Innovation and Technology Award category for their work on touchless technology for surgery. The Biomedical Awards celebrate the achievements of biomedical organisations in the North of England, and the strength and depth of the cluster. Sixteen organisations were shortlisted within five categories for the 2014 Biomedical Awards Ceremony.
Geoff Davison, CEO of Bionow, said: "The award entries demonstrated innovative, forward-thinking approaches to research, product development and business strategy, with all of the companies impressing the judges with both their dynamism and their exceptional contributions to the sector. The quality, number and variety of award entries was great to see, and the final group of 16 companies are a testament to the vast potential of this growing industry."
www.bionow.co.uk/news/2014biomedicalawardsshortlistannounced.asp
Future Plans
The ultimate aim is to develop a touchless interaction in surgery toolkit that can be used in any hospital or system interested in applying touchless interaction to their imaging system. This work is part of a broader initiative within Microsoft Research to promote this use of Kinect in the medical domain. The 3D imaging system is currently being commercialised by a new start-up organisation. There have been initial discussions with this organisation regarding the licensing of the gesture and voice control interface component as part of this system.