Artec Space Spider breathes life into medical school dissections
3D and VR technologies alike are growing ever more popular throughout industries all over the world and are becoming especially common in the healthcare field due to their precision, ease of use and versatility. One of Artec’s long-standing clients, IMA solutions, has been involved in a very exciting project that features a combination of the latest in 3D scanning and VR developments, and looks to significantly alter the way medicine is practiced and taught.
Montpellier Medical University
It all began with two surgeons from the Montpellier Medical University city anatomy laboratory, M.D, Ph.D Guillaume Captier and M.D Mohamed Akkari, who were seeking to resolve certain difficulties in teaching medical students at the university. The lessons required the use of real cadavers, and while this was an effective way of learning about the human anatomy, issues with availability and costs of the cadavers would often arise. Performing dissections on bodies of the deceased is quite costly and inexperienced medical students do not always perform a proper dissection on the first try, often damaging or even destroying multiple cadavers at a time.
After careful consideration of the options available, a decision was made to design a highly accurate, real-time 3D dissection application, which will allow students to practice procedures and reach an acceptable standard prior to dissecting real cadavers and will also enable teachers to adapt a completely new and effective way of teaching. This versatile and unique application will also be able to run in an examination mode in order to check the level of students’ knowledge, allowing them to hone their medical skills, without having to use real cadavers.
To ensure that the application data was as realistic and accurate as possible, all of the dissection steps on actual corpses had to be 3D scanned and transferred to the training application. The experts at the IMA Solutions undertook this vital task. Using the highly-precise Artec Space Spider 3D scanner, they captured all of the dissection stages to provide 3D photorealistic models to feed the application.
3D result of a neck dissection step using the Artec Space Spider. Geometry rendering
The surgeons proposed the areas of the body for dissection. One area was the neck, where it was necessary to see the facial nerve, the vascular anatomy of the neck, and the trachea, as well as some other parts. To make sure that the data of all these areas were in the training application, eight separate 3D scans, or anatomical layers, had to be made, going from the skin down to the deep layers. Because the colour of the cadaver changes over time, all the scanning had to be performed as quickly as possible, and not exceed a maximum of one day, to ensure that the colours were the same in all stages.
After all the data had been captured by the Space Spider 3D scanner, it had to be processed and layered correctly on the 3D model that will be used for virtual dissection. To achieve this, IMA Solutions used the practical and versatile Geomagic Freeform 3D software, widely popular in the industry for its extensive modelling capabilities for organic shapes. An experienced user of this software in previous projects, IMA Solutions values how Freeform allows you to quickly manipulate 3D objects in a 3D scene, as well as the sense of touch you get from using the accompanying haptic device.
“Freeform is a software we have use since the beginning for its ability to quickly manipulate 3D objects in a 3D scene and for being able to use the sense of touch by a haptic device. From our side we use the Phamtom Omni Haptic device in combination with the Freeform. As the software is based on a voxel clay system, you can use some operators like Booleans very quickly, or just keep the biggest object in the scene. Also, you can sculpt on 3D objects with the sense of touch mimicking sculpting on a clay model, adding details, modifying shapes…In a way, you can, of course, do a lot of these things in other 3D software, but not at the speed of using Freeform.”
For this VR dissection project, IMA Solutions experts used the Freeform software to correct and enhance both the geometry and the textures of the captured 3D models. These corrections include removing islands, 3D sculpting tiny geometries and 3D texturing. Capturing the different steps in the dissection of real cadavers can be rather challenging, especially when it comes to areas of fat tissue and blood vessels. These tissues are partially translucent and absorb and reflect light. This can affect the quality of scanning and the final result. Therefore, for these complex areas, where small 3D scanning artefacts can sometimes be present, Geomagic Freeform is a fantastic solution for correcting the geometry and texture. All of these tasks can easily be carried out using the effective combination of the Freeform software and the haptic device, which allows you manipulate and sculpt 3D objects as you would do it in real life.
According to Benjamin Moreno, IMA’s solutions founder and CEO, combining the Artec 3D scanner with the Freeform software is a perfect solution because they both have a “hands-on” approach and can be used intuitively, allowing the user to focus on their creative work, rather than be distracted by technical issues or complex technical operations.
“Combining Artec scanners with Freeform could be a perfect solution for digital artists because both are easy to take in hand and are very intuitive to use. You can focus your attention on your creative work without being frustrated by technical issues or complex technical operations”
3D result of a neck dissection step using Freeform for digital sculpting
During the dissection process, Benjamin and his team had to interact with different tissues, all with their own textures and material properties. The first step was to make feasibility tests on three dissections: two dissections of the neck and one dissection of the pelvic area. The IMA Solutions scanning experts wanted to see the quality of scans they can expect and be able to check if any unforeseen problems appeared. The initial results were very exciting, and the team was able to validate the 3D scanning procedure.
“We were clearly impressed by the Artec scanners ability to grab very nice geometries and textures on some hard-to-scan tissues that can have translucency or light diffusion and absorption. The possibility to manage the sensitivity of the scanner was helping us a lot to get 100% of the geometries and colors, so that no special preparations of the cadavers were needed!”
Benjamin Moreno, IMA’s solutions founder and CEO
For the 3D scanning and data processing, the latest version of Artec Studio software was used, where the small object filter was applied to remove any floating islands, as well as the mesh simplification tool to generate low-polygon 3D models for 3D real-time engines.
3D result of a neck dissection step into Artec Studio 12
The test run of the final version of the application is scheduled for the fourth quarter of 2018, at the Montpellier Medical University. Meanwhile, IMA Solutions continue working on the pilot project, gathering enough data to create at least five datasets on selected anatomy areas to validate the complete workflow, as well as improving the design of the user interface and adding the tools needed by both medical students and surgeons to get the best out of this interactive application.
Demonstration of the first version of the application at the Montpellier Medical University by Prof. Guillaume Captier, Chief of the Anatomy Lab of the University
Prof. Guillaume Captier,Interacting with the training application in the early stages of its development
These datasets will run on Inside Explorer technology from Swedish high-tech company Interspectral. This remarkable technology offers an interactive and intuitive 3D learning experience, as it is capable of displaying the multimodal real time acquisitions in 3D, where it is possible to combine CT, MRI and 3D scans in one 3D scene.
The combination of the latest 3D and VR technologies is making it possible for healthcare training opportunities to take a giant leap forward. The successful integration of the interactive training application in healthcare education processes will have great benefits for students, professors and doctors alike. Young and upcoming medical professionals will have the ability to practice on virtual cadavers, unrestricted by factors such as their availability and cost, while lecturers, doctors and other experienced medical staff will have access to a revolutionary new way to teach medicine. Inventions such as this seemed impossible only a few years ago, but with the implementation of new, cutting-edge technologies, this is just the beginning.