Artificial Actors: Directable digital humans based on psychological models of relational reasoning

Objective
Digital humans and AI are becoming an integrated part of society. Modern chatbots such as GPT3 (Generative Pre-trained Transformer 3) have shown a remarkable capability of generating human-like responses and may even be prompted to act sarcastic, depressed or shy. To bring such systems into an embodied agent, such as a digital human, the agent’s body motion should reflect the same psychological inner state. This is, however, lacking in modern synthesis systems of non-verbal behaviours, which only generate generic motion based on a neutral speaker.

In this project, we propose to enhance virtual agents’ state of the art by giving them a psychological inner state that colours their nonverbal behaviour. We term these agents artificial actors – virtual digital humans that can take directions and produce expressive and convincing acting behaviour, much like a real actor takes instructions from a director. This means that it should be possible to instruct a character not only what to do or talk about but also how these actions should be performed (e.g. as a shy person with social phobia).

The project includes

• A) recording a large database of acted behaviours containing a range of psychological states
• B) developing probabilistic generative methods to synthesize gestures from such high-level traits and,
• C) developing a psychological, cognitive model guiding the synthesis.

We will specifically create a virtual agent simulating a therapy patient and evaluate its performance with therapists or therapists in training.

Background
TBC

Crossdisciplinary collaboration
The researchers in the team represent the School of Electrical Engineering and Computer Science, KTH and the Psychology Department at Stockholm University.