Social Play in Immersive Gaming Environments

Project overview

Gaming often drives the early adoption of Natural User Interfaces (NUIs), and serves as a testing ground for their development. Technologies created for gaming, such as the Microsoft Xbox Kinect and the Tobii Eye Tracker 4C, enable new types of playful and immersive experiences – and often go on to impact wider society. However, while there is substantial research into the user experience of NUIs in work-like computer tasks, very little research has examined NUIs in the context of games and immersive environments. We believe that studying NUI in the game context is relevant to the use of these technologies, because the game experience can be quite different when immersed or embodied.

Research under this project generates knowledge about designing with NUIs in social games, through case studies that interrogate social play in games. The research agenda is multi-modal, and aims to contribute a rich understanding regarding the relationship between game embodiment, immersion and interface design.

In addition to academic articles, we communicate results in game industry and non-specialist publications, host talks at game conventions, demonstrate and exemplify our findings through videos, and develop game prototypes that demonstrate the foundations of our design guidelines.

Social Voice

Voice is widely used in social and competitive gameplay as a communication modality between players, where it has been shown to vastly improve player experience. However, as an input device for play, voice has a mixed status. It is often marginal and maligned due to the limitations of the technology, but sometimes fêted as a socially engaging way to play. In this project, we seek to understand the parameters of designing for voice interaction with game-like characters, through a study of existing practices and through the evaluation of for-purpose experimental games.

This project is led by PhD student Fraser Allison, under the supervision of Martin Gibbs, Wally Smith and Marcus Carter.

Social Gaze

EyePlay research seeks to understand how gaze detection technologies can provide new and novel player and spectator experiences. This project examines the effects of gaze and how it can create social experiences in multiplayer games, through the exploration of its detection and display.

This project is led by PhD student Joshua Newn, under the supervision of Frank Vetere and Eduardo Velloso.

Social Tabletop

Tabletop gaming is a highly social situation due to the configuration of players and the social interactions afforded by tabletop game pieces. This project examines non-digital tabletop games and their digitisations to identify opportunities for NUI technologies to afford social play.

This project is led by PhD student Melissa Rogerson, under the supervision of Wally Smith and Martin Gibbs.

Social Minecraft

The sandbox-world builder game Minecraft is exceedingly popular, near ubiquitous in children between 8–12 and was recently purchased for $2billion dollars by Microsoft. This popularity, and its emphasis on creative world building, social interactions and cooperation have seen it adopted in schools and released as a standalone Minecraft for Education project. This project examines the social play of Minecraft and how it can support opportunities for social learning.

This project is led by PhD student Jane Movoa, under the supervision of Martin Gibbs and Marcus Carter.

Project team

  • Martin Gibbs, Associate Professor, School of Computing and Information Systems, University of Melbourne
  • Eduardo Velloso, Research Fellow, Microsoft Research Centre for SocialNUI, University of Melbourne
  • Frank Vetere
    Frank Vetere, Professor & Director, Microsoft Research Centre for SocialNUI, University of Melbourne
  • Wally Smith, Senior Lecturer, School of Computing and Information Systems, University of Melbourne
  • Fraser Allison, PhD Candidate, Microsoft Research Centre for SocialNUI, University of Melbourne
  • Joshua Newn, PhD Candidate, Microsoft Research Centre for SocialNUI, University of Melbourne
  • Melissa Rogerson, PhD Candidate, Microsoft Research Centre for SocialNUI, University of Melbourne
  • Jane Mavoa, PhD Candidate, Microsoft Research Centre for SocialNUI, University of Melbourne
  • Brandon Syiem, Masters Student, Microsoft Research Centre for SocialNUI, University of Melbourne
  • Robbie Fordyce, SocialNUI Researcher, Microsoft Research Centre for SocialNUI, University of Melbourne
  • Marcus Carter, Department of Media and Communications, University of Sydney

Contact details

Publications

Mavoa, J., Gibbs, M. & Carter, M. (2017) Constructing the young child media user in Australia: a discourse analysis of Facebook comments. Journal of Children and Media, 11(3): 330-346. [DOI:10.1080/17482798.2017.1308400]

Carter, M. & Allison, F. (in press) Guilt in DayZ. In Transgressions in Games and Play, Eds. Jørgensen, K. & Karlsen, F., MIT Press.

Mavoa, J. (2017) The games children play: an analysis by title, gender and age. Extended Abstract presented at Digital Games Research Association Conference (DiGRA 2017), Melbourne, Australia.

Allison, F., Luger, E. & Hofmann, K. (2017) Player expectations of a learning AI companion in Minecraft. Extended Abstract presented at Digital Games Research Association Conference (DiGRA 2017), Melbourne, Australia.

Rogerson, R. & Cocks, J. (2017) F**king with psychology: psychological tests as boardgame design inspiration. Extended Abstract presented at Digital Games Research Association Conference (DiGRA 2017), Melbourne, Australia.

Rogerson, R., Gibbs, M. & Smith, W. (2017) Me and My Shelfie: hobbyists’ domestication of boardgames. Extended Abstract presented at Digital Games Research Association Conference (DiGRA 2017), Melbourne, Australia.

Carter, M. & Allison, F. (in press) Fear, Loss and Meaningful Play: Permadeath in DayZ. Journal of Gaming and Virtual Worlds

Velloso, E., Carter, M., Newn, J., Esteves, A., Clarke, C. & Gellersen, H. (2017) Motion Correlation: Selecting Objects by Matching their Movement ACM Transactions on Computer-Human Interaction (ToCHI) 24(3):article22. [DOI]

Allison, F., Luger, E. & Hofmann, K. (2017) Spontaneous Interactions with a Virtually Embodied Intelligent Assistant in Minecraft. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, Denver, USA. [DOI]

Allison, F. (2016) Are You Out of Your Mind? Focalization in Digital Games. Transactions of the Digital Games Research Association 2(3):31–60.

Carter, M., Fordyce, R., Gibbs, M., Robertson, N., & Witkowski, E., (2016) eSports Spectatorship in Australia. In Proceedings of the 2016 DiGRA Australia Conference, Melbourne [PDF]

Velloso, E. (in press) Game Controller Research from the Lab into the Wild: The Case of Eye Tracking. In Proceedings of the 2016 DiGRA Australia Conference, Melbourne [PDF]

Velloso, E. & Carter, M. (2016) The Emergence of EyePlay: A Survey of Eye Interaction in Games. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play (CHI Play 2016). Austin, TX, USA [PDF]

Newn, J., Velloso, E., Carter, M. & Vetere, F. (2016). Exploring the Effects of Gaze Awareness on Multiplayer Gameplay. In Proceedings of the 2016 Annual Symposium Extended Abstracts on Computer-Human Interaction in Play (CHI PLAY 2016). Austin, TX, USA [DOI] [PDF]

Newn, J., Velloso, E., Carter, M. & Vetere, F. (2016). Dynamically Exposing Gaze to Foster Playful Experiences in Multiplayer Gameplay. In CHI PLAY 2016 Workshop on Designing for Emotional Complexity in Games: The Interplay of Positive and Negative Affect. Austin, TX, USA [PDF]

Newn, J., Velloso, E., Carter, M. & Vetere, F. (2016). Multimodal Segmentation of a Large Interactive Tabletop: Extending Interaction on Horizontal Surfaces with Gaze. In Proceedings of the 2016 ACM International Conference on Interactive Surfaces and Spaces (ISS 2016). Niagara Falls, ON, Canada [PDF]

Allison, F., Carter, M., & Gibbs, M. (2016). A History of Voice Interaction in Games. In Proceedings of the First Joint International Conference of DiGRA and FDG [PDF]

Rogerson, M. J., Gibbs, M., & Smith, W. (2016) "I Love All the Bits": The Materiality of Boardgames. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI 2016), New York: ACM Press, pp. 3956–3969 [DOI]

Carter, M., Newn, J., Velloso, E., & Vetere, F. (2015) Remote Gaze and Gesture Tracking on the Microsoft Kinect: Investigating the Role of Feedback. In Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction (OzCHI 2015) New York: ACM Press, pp. 167–176 [DOI]

Allison, F., Carter, M., & Gibbs, M. (2015). Good Frustrations: The Paradoxical Pleasure of Fearing Death in DayZ. In Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction (OzCHI 2015) New York: ACM Press, pp. 119–123. [DOI]

Carter, M., Allison, F., Downs, J., & Gibbs, M. (2015). Player Identity Dissonance and Voice Interaction in Games. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play (CHI Play 2015) New York: ACM Press, pp. 265–269 [DOI]

Rogerson, M. J., Gibbs, M., & Smith, W. (2015) Digitising Boardgames: Issues and Tensions, In Proceedings of the 2015 DiGRA International Conference. Lüneburg, Germany [PDF]

Carter, M., Nansen B., Gibbs, M. (2014) Screen Ecologies, Multi-Gaming and Designing for Different Registers of Engagement. In Proceedings of CHI Play, ACM Press, pp. 37–46. [PDF, DOI]

Media