PhD student position at the Department of Computing Science at Umeå University.

Project description

We are seeking one PhD student to work in formal language theory, focusing on applying theoretical computer science to the analysis of the transformer neural network architecture.

This project analyses the limits of representability and learnability in transformers, the popular neural network architecture used by many state-of-the-art systems, for example, large language models like ChatGPT and the AI Sweden/RISE/WARA M&L GTP-SW3 model. Transformers are known to be limited, allowing them to be formally analyzed in a principled way. This project leverages recent results, which characterize transformers in terms of Boolean circuits and/or logical descriptions, and aims to apply algebraic decompositions to understand and modify transformer models.

This project is fully funded by WASP. The project will be led by Dr. Martin Berglund as part of the Foundations of Language Processing Group. As a PhD student you would work as part of this group, which has a broad cross-disciplinary focus including formal and natural language processing, related machine learning techniques, and a broad mix of theoretical computer science.

More information and application

View all positions
We use cookies to personalise content and ads, to provide social media features and to analyse our traffic. We also share information about your use of our site with our social media, advertising and analytics partners. View more
Cookies settings
Privacy & Cookie policy
Privacy & Cookies policy
Cookie name Active
The WASP website uses cookies. Cookies are small text files that are stored on a visitor’s computer and can be used to follow the visitor’s actions on the website. There are two types of cookie:
  • permanent cookies, which remain on a visitor’s computer for a certain, pre-determined duration,
  • session cookies, which are stored temporarily in the computer memory during the period under which a visitor views the website. Session cookies disappear when the visitor closes the web browser.
Permanent cookies are used to store any personal settings that are used. If you do not want cookies to be used, you can switch them off in the security settings of the web browser. It is also possible to set the security of the web browser such that the computer asks you each time a website wants to store a cookie on your computer. The web browser can also delete previously stored cookies: the help function for the web browser contains more information about this. The Swedish Post and Telecom Authority is the supervisory authority in this field. It provides further information about cookies on its website,
Save settings
Cookies settings