Postdoc in Machine Learning for human communication

Excited about DALL·E and GPT-3? Join a strong research team across Sweden taking on the challenge of merging NLP, speech processing, and computer vision into one single model.

We are looking for a postdoc to join the newly started WASP NEST project STING – Synthesis and analysis with Transducers and Invertible Neural Generators, working at Umeå University.

Human communication is multimodal in nature, and occurs through combinations of speech, language, gesture, facial expression, and similar signals. STING aims to design models that capture this richness, uniting synthesis and analysis with the help of transducers and deep neural generative models. 

This involves connecting concrete, continuous valued sensory data such as images, sound, and motion, with high-level, predominantly discrete, representations of meaning, which has the potential to endow synthesis output with human understandable high-level explanations, while simultaneously improving the ability to attach probabilities to semantic representations. The bidirectionality also allows us to create efficient mechanisms for explainability, and to inspect and enforce fairness in the models.

The partner research groups bring complementary expertise to the project: KTH has extensive experience with probabilistic deep learning for analysis and synthesis of human verbal and nonverbal communication. Umeå University, on the other hand, are experts on transducer and grammar models for generating semantic graphs, and have recently started to apply these to the task of parsing multimodal data. They also contribute experience with bias analysis and mitigation. Linköping University complements these aspects with in depth knowledge of natural language processing, language being a discrete yet observable signal modality of great interest for bridging the two ends of the project.

In addition to its scientific value, the project is expected to have a substantial societal imprint. The resulting technologies may, e.g., be used to create virtual patients for medical training, to model non-playable characters in video games, and to derive affective states and underlying health issues from human speech and nonverbal behaviour. 

This position will be based at Umeå University, but you will be expected to collaborate closely with the PIs and other researchers in the project at the other sites as well.

Possible research directions

  • Work on the latest deep generative models with diffusion models and normalising flows
  • Combining discrete and continuous methods for synthesis and/or analysis

What we offer

  • Stimulating and collaborative research and environment
  • Smart people with a variety of competences
  • Ability to shape your own research and working conditions

What you bring

  • PhD (or near graduation) in machine learning, or application areas such as computer vision, image or voice synthesis, or NLP
  • The intellectual curiosity to learn about our other application areas
  • Strong analytical and technical skills
  • Strong research track record
  • The will and drive to make a difference

What you should do

If you are interested, please contact us! You can reach us at:

  • Johanna Björklund (, Umeå University)
  • Henrik Björklund (, Umeå University) 
  • Frank Drewes (, Umeå University)

The other PIs involved in the project are:

  • Gustav Eje Henter (KTH Royal Institute of Technology)
  • Hedvig Kjellström (KTH Royal Institute of Technology)
  • Marco Kuhlmann (Linköping University)

We use cookies to personalise content and ads, to provide social media features and to analyse our traffic. We also share information about your use of our site with our social media, advertising and analytics partners. View more
Cookies settings
Privacy & Cookie policy
Privacy & Cookies policy
Cookie name Active
The WASP website uses cookies. Cookies are small text files that are stored on a visitor’s computer and can be used to follow the visitor’s actions on the website. There are two types of cookie:
  • permanent cookies, which remain on a visitor’s computer for a certain, pre-determined duration,
  • session cookies, which are stored temporarily in the computer memory during the period under which a visitor views the website. Session cookies disappear when the visitor closes the web browser.
Permanent cookies are used to store any personal settings that are used. If you do not want cookies to be used, you can switch them off in the security settings of the web browser. It is also possible to set the security of the web browser such that the computer asks you each time a website wants to store a cookie on your computer. The web browser can also delete previously stored cookies: the help function for the web browser contains more information about this. The Swedish Post and Telecom Authority is the supervisory authority in this field. It provides further information about cookies on its website,
Save settings
Cookies settings