Handheld medical surgery simulation with Virtual Reality aspects
This is part of the Touch Surgery case study.
Project Overview
When the TS Android/iOS app already held a decent amount of of pre-rendered surgery simulations, the time was ripe for a natural next move and prototype a real-time 3D module.
The Solution
Smartphone's gyroscope and camera allowed for a low-effort Extended Reality implementation that would enrich the experience and challenge of the surgery simulation.
- While we explored off-the-shelf engines like Unity and Unreal, the decision was made to create a mobile-friendly, lightweight engine custom-made for surgical training simulations.
- With real-time 3D, the user would not only benefit from looking at the anatomical cavity from any angle. The angle of their approach could be a soft-rule subject of the test, when warranted by a particular surgical simulated procedure, enriching the experience and the learning process.
- An anchor in the 3D space would act as a centre of mass from which the user could navigate the anatomical representations.
Design Strategy
While the 3D environment expanded upon the existing 2D game mechanics, it remained the same at fundamental level. This meant that any graphic interface updates could be retrofitted onto the legacy modules.
The clearest example of this is the Procedure timeline (pictured below, still used to this day), which devides the surgical procedure into Preoperative, Inter-operative, and Postoperative, each with their steps, points of evaluation, postponing, complications, etc.
The interactive timeline would indicate progress, collapsing and expanding sections, success and fail, etc.; and allow for draggable behaviour, progress bar, or scrubber bar, much like Youtube's.
Target audience and use
Our research revealed that medical students frequently accessed our content on handheld devices while studying or on the go. To reach a broader audience, the Minimum Viable Feature needed to be both functional and enjoyable in these contexts.