Augmented Learning Environment for Physiotherapy Education

Project overview

Physiotherapy is currently taught via multiple modalities including lectures, tutorials, demonstrations and hands on training in practical classes. In some cases, students can have difficulty in learning clinical reasoning in physiotherapy. Clinical reasoning can be particularly difficult to simulate and to augment with pedagogical technology as it requires students to critically review clinical decisions which is often brief and implicit. The best practice for learning such skills typically involves a class of students observing an expert therapist conducting a consultation with a real patient (or a surrogate patient). However, such settings do not make the therapist's clinical reasoning explicitly available to students who rely on written notes and sketches.

This project aims to enhance student learning in physiotherapy via an augmented learning environment titled the ‘Augmented Studio’ developed using projectors and Microsoft Kinect sensors to enable projections of an anatomy model onto a moving body for practical physiotherapy classes. Anatomical knowledge about human movements comes to life as the instructor asks the patient to perform simple to complex movements to demonstrate the kinesiology understanding of muscles, joints, and bones in the body. The patient body becomes a live canvas for the instructor to illustrate clinical reasoning. Using a tablet interface with pen input, the instructor can draw annotations directly on the virtual body, which are rendered in real time on the patient’s body. With the Augmented Studio, the researchers explore the capability of creating virtual annotations, to identify certain anatomical structures or to illustrate the joints or muscle connections to teach students. Currently the instructor uses still photos, figures, and diagrams which do not capture the real time and complex dynamics of human movements.

Augmented reality based learning and teaching presents an innovation of novel pedagogical practice delivering benefits to both the staff and the students. The student learning experience is enhanced in multiple ways with the usage of the augmented reality learning environment, with augmented kinaesthetic information and annotated information provided by the instructor to support interactive observation with augmented visualisation for better clinical understanding of human anatomy and musculoskeletal structures. This will enhance appreciation by students of dynamic change in anatomical configuration of the body through 3 dimensions and the ability to track a full motion and change throughout range. The project aims to deliver positive learning outcomes for the students by enabling experiential learning and increased connection with the learning materials, lessons and the instructor, catering for both tactile and visual learning, and encouraging cognitive and clinical reasoning.


This project is a collaboration between the Microsoft Research Centre for Social Natural User Interfaces (SocialNUI) in the Department of Computing and Information Systems and the Department of Physiotherapy at the University of Melbourne.

Project team

  • Thuong Hoang
    Thuong Hoang, Research Fellow, Microsoft Research Centre for SocialNUI, University of Melbourne
  • Frank Vetere
    Frank Vetere, Professor & Director, Microsoft Research Centre for SocialNUI, University of Melbourne
  • Zaher Joukhadar, Lead Software Engineer, Microsoft Research Centre for SocialNUI, University of Melbourne
  • Martin Reinoso, PhD Candidate, Microsoft Research Centre for SocialNUI, University of Melbourne
  • Louisa Remedios, Associate Professor, Department of Physiotherapy, University of Melbourne
  • David Kelly, Lecturer, Department of Physiotherapy, University of Melbourne
  • Egemen Tanin, Associate Professor, Department of Computing and Information Systems, University of Melbourne

Contact details