Date of Award
Open Access Thesis
Medical Doctor (MD)
Janet P. Hafler
Tracii A. Wolbrink
PBLCLOUD VIRTUAL PATIENT SIMULATOR: ENHANCING IMMERSION THROUGH NATURAL LANGUAGE PROCESSING
Pierre Martin M.Ed.(1), Lisa DelSignore M.D.(2), and Traci A. Wolbrink M.D. M.P.H.(2)
(1)Yale School of Medicine (2)From the Division of Critical Care Medicine, Department of Anesthesia, Perioperative and Pain Management, Boston Children’s Hospital and the Department of Anesthesia, Harvard Medical School, Boston, MA, USA. (Sponsored by JH, Department of Pediatrics, Yale School of Medicine)
Virtual patient simulation has been utilized to teach interviewing skills, often employing selection-based methods (e.g., multiple-choice lists and menu-based prompts) to simulate doctor-patient conversations. Users have evaluated these systems as inauthentic, which can diminish user immersion (influenced by control, realism, distraction and sensory factors) and, in turn, negatively affect skill acquisition, mastery and transfer.
Our objectives were to design and develop PBLCloud, a scenario-based and highly interactive platform that uses natural language processing to support a more realistic doctor-patient conversation and create an immersive clinical learning environment.
PBLCloud was developed utilizing an iterative design thinking process and its initial evaluation involved a mixed methods approach. We recruited a convenience sample of 11 participants: three (27%) fourth-year medical students from Harvard Medical School as well as two (18%) residents, four (36%) fellows and two (18%) attendings from Boston Children’s Hospital. There were two rounds of formative evaluation testing with eight participants in Round 1 and three participants in Round 2. Each participant completed a semi-structured think–aloud protocol exploring our pilot case, 10-item system usability scale (SUS) and 10-item open-ended questionnaire.
The chat-based functionality provides users with computer-generated context-specific responses during the historical encounter. Users have the opportunity to perform physical examinations, review incorporated multimedia, order and interpret diagnostic investigations, order therapeutic interventions that have appropriate effects on patient vitals and laboratory data, formulate and refine a differential diagnosis, receive just-in-time feedback regarding user-initiated actions and complete embedding learning exercises. 73% of participants strongly agreed that PBLCloud was useful (i.e., it is clinically-oriented, realistic, provides helpful feedback and is widely applicable) and 64% of participants strongly agreed that their experience with the system was enjoyable (i.e., it is relevant with an engaging interface). It was deemed to be more interactive and engaging than other simulators and 82% of participants were very interested in utilizing the system in the future. The average SUS score for Round 1 and 2 were 79.7 ± 12.0 and 82.5 ± 19.8 respectively. Areas of improvement were identified, in particular, the unsatisfactory response accuracy of the chat-based functionality.
Future work will include the investigation of various strategies to optimize the platform’s natural language processing algorithm as well as the formal evaluation of the system’s validity, reliability, level of induced user immersion and educational impact. We anticipate that PBLCloud will serve as a cost-effective and scalable approach for the instruction and assessment of clinical reasoning.
Martin, Pierre Rodricus, "Pblcloud Virtual Patient Simulator: Enhancing Immersion Through Natural Language Processing" (2017). Yale Medicine Thesis Digital Library. 2151.