KTH School of Electrical Engineering and Computer Science seeks two PhD students in generative AI for 3D gesture motion.

Project description

In face-to-face communication, humans use co-speech gestures – such as hand, arm, head, and body movements – to enhance clarity, naturalness, and efficiency. Our lab is a pioneer and global leader in leveraging 3D motion-capture data and generative AI to synthesise these gestures for virtual characters and humanoid robots.

The specific focus of this project is to use data on human opinion – e.g., subjective ratings of whether a gesture looks good visually, or whether it’s a good fit for the character’s speech – to further improve gesture generation. You will not only try to predict human opinion of these aspects, but also use ratings data to further improve models using direct preference optimisation (DPO) or other reinforcement learning from human feedback (RLHF) techniques. There may also be opportunities to pursue improved benchmarking and evaluation practices for gesture motion, sign-language synthesis, or joint multimodal synthesis of speech audio and gesture motion together.

Supervision: Assistant Professor Gustav Eje Henter; co-supervisors Prof. Jonas Beskow and Assistant Prof. Éva Székely

KTH Royal Institute of Technology in Stockholm is Sweden’s largest technical research and learning institution and home to students, researchers and faculty from around the world.

View all positions
We use cookies to personalise content and ads, to provide social media features and to analyse our traffic. We also share information about your use of our site with our social media, advertising and analytics partners. View more
Cookies settings
Accept
Privacy & Cookie policy
Privacy & Cookies policy
Cookie name Active
The WASP website wasp-sweden.org uses cookies. Cookies are small text files that are stored on a visitor’s computer and can be used to follow the visitor’s actions on the website. There are two types of cookie:
  • permanent cookies, which remain on a visitor’s computer for a certain, pre-determined duration,
  • session cookies, which are stored temporarily in the computer memory during the period under which a visitor views the website. Session cookies disappear when the visitor closes the web browser.
Permanent cookies are used to store any personal settings that are used. If you do not want cookies to be used, you can switch them off in the security settings of the web browser. It is also possible to set the security of the web browser such that the computer asks you each time a website wants to store a cookie on your computer. The web browser can also delete previously stored cookies: the help function for the web browser contains more information about this. The Swedish Post and Telecom Authority is the supervisory authority in this field. It provides further information about cookies on its website, www.pts.se.
Save settings
Cookies settings