What is a Social Natural User Interface (Social NUI)?

Social NUIs are NUI technologies that have a focus on social interactions. Social NUIs explore human capabilities of movement, speech, touch and gaze, and facilitate human relationships through new and emerging technologies beyond the keyboard and mouse.

Social NUIs are based on Natural User Interfaces (NUIs), which are inspired by the way people interact with, manipulate and communicate in the physical world. The history of NUIs span many decades, but it is only in recent years that they have become available as mature, robust and commercially available technologies. For example, although the multi-touch tablet was originally described in 1985, it is the recent hardware advances in the Apple iPhone and the Microsoft Surface (both released in 2007), that have made multi touch interaction accessible for everyday computing applications. Similarly progress in gesture (e.g. PlayStation Move, Microsoft Kinect, Nintendo Wii), and speech recognition (e.g. Siri, Cortana, Google Now), have now opened up a rich repertoire of interaction possibilities.

Social NUIs are concerned with how we relate to each other, how we nurture human relationships and how we engage with the social dynamics of everyday life. Examples of Social NUI include:

  • Sign Language Translator: detects sign language and translate to spoken language to enable a conversation between two people.
  • Encounters: a large outdoor installation that recognises social groups and interplay between people and public spaces.

In the Microsoft Research Centre for SocialNUI we are exploring:

  • The social implications of these technologies. For example, how does it change human interactions with a technology when it is inherently performative? Speech technologies require people to say things out loud that other people can hear. Body tracking via the Microsoft Kinect requires people to move in ways that other people can see. These are both different to interactions that a person might make with a traditional form of interaction – NUIs turn interactions from private to public.
  • The social applications of these technologies. For example, how can we use NUI sensors beyond a person-to-machine interaction, and use them to understand human relationships and social activity?
  • How to design technologies that both incorporate natural user interaction and support social relationships.