Rånäs Castle Photo: Anders Fredriksen
The program committee welcomes you to the Wallenberg Advanced Scientific Forum on the theme Measuring What Matters: Evaluation as a Driver of Generative AI.
Deep generative models are advancing the state of the art in artificial intelligence by effectively capturing and recreating complex patterns in high-dimensional, real-world data. Recent breakthroughs in generating text, images, and video have led to the development of new ecosystems and are already making a tangible impact on the world. Applications in other modalities are expected to soon experience similar growth.
However, the evaluation of generative AI systems presents greater challenges compared to traditional machine-learning evaluation, as there is rarely a single correct output. Multimodal generative models introduce additional complexities, requiring assessment of consistency, complementarity, and coherence across different modalities.
This forum will bring together global experts in deep generative modeling—including both problem owners and model developers from industry, academia, and other sectors—across various domains. The goal is to bring about new, effective evaluation methods that will drive future progress in multimodal generative models, alongside practical tools to promote their adoption within the research community.
Date and time
The workshop starts at 12:00 with lunch on September 30th and ends after lunch on October 3rd, all at Rånäs Castle, Stockholm.
Workshop structure
The forum is designed to be highly interactive, consisting nearly exclusively of structured and directed discussion in smaller focus groups, rather than panels, talks, or plenaries. To make the most out of the event, we will use the so-called double-diamond framework for structured problem solving. This organises problem solving into two stages, first selecting the right problem(s) to tackle and then finding solutions for these. Each stage has two phases: first divergence (exploring possibilities), then convergence (narrowing down choices). The in-person forum will also be preceded by one or two short online workshops to kickstart the discussion before we meet up on site.
Program
Tuesday September 30
12:00 Lunch
Location: Main building, the castle
13:00 Intro day 1
Location: Stenkvarn
13:30 Poster session I
Location: Vallonhuset (Vallon building)
14:30 Coffee break and opportunity to check-in
Location: Stenkvarn and Ölstugan
15:30 Presentation of groups
Location: Stenkvarn
15:45 Group discussion and poster creation,
Location: Group rooms
17:30 Mingle, continued group discussions, and free time
Location: Top floor of main build
19:00 Dinner
Wednesday October 1
9:00 Group discussion and poster finalisation
Location: Group rooms
10:00 Coffee and putting up posters
Location: Vallonhuset (Vallon building) and Stenkvarn
10:30 Introduction
Location: Stenkvarn
10:45 Poster presentations
Location: Stenkvarn
11:15 Poster session
Location: Vallonhuset (Vallon building)
12:00 Lunch
Location: Main building
13:00 Group discussions
Location: Group rooms
15:00 Coffee break
Location: Stenkvarn
15:30 Group discussions cont’d
Location: Group rooms
17:00 Guided tour, group 1
17:45 Guided tour, group 2
19:00 Dinner
Location: Main building
Thursday October 2
9:00 Intro day 3
Location: Stenkvarn
9:30 Group discussion and documentation
Location: Group rooms
10:30 Coffee break
Location: Stenkvarn
11:00 9:30 Group discussion and documentation
Location: Group rooms
12:00 Lunch
Location: Main building
13:00 Reporting
Location: Stenkvarn
14:00 Mingle and feedback
Location: Stenkvarn
15:00 Coffee break
Location: Stenkvarn
15:30 Group discussion and documentation
Location: Group rooms
17:30 Break
19:00 Conference dinner
Location: Main building
Friday October 3
9:00 Intro day 4
Location: Stenkvarn
9:45 Implementation planning
Location: Group rooms
10:30 Coffee break
Location: Stenkvarn
11:00 Implementation reporting
Location: Stenkvarn
11:30 Announcements
Location: Stenkvarn
11:40 Free mingle and discussions
Location: Main building
12:00 Lunch
Location: Main building
Pre-workshop sessions
An online preparatory workshop will be held via Zoom. You should have received an email with information about these pre-workshops and will also receive a calendar invitation over email. The purpose of this session is to start the forum work already before we meet. To accommodate different time zones and availability, we are hosting two identical pre-workshop sessions with the same content. You will only attend one of these sessions. The workshops will take place approximately two weeks before the in-person forum in Sweden.
About the pre-workshops
The pre-workshop will last two hours, including a 10-minute break in the middle. It will begin with an introduction to the process and methodology we will be using to guide the discussions at the forum, and then transition into an online workshop session where we all collaborate to make a start on the work that we will continue when we meet in person at the forum.
Accommodation
A single room is booked for each participant. Guaranteed check-in time is 15:00 at the day of your arrival and check-out is 9:00 on the day of your departure.
Transportation
Email anna.bjornemo@liu.se before September 10 if you would like to sign up for transportation by bus between Arlanda Airport and Rånäs Castle. If you choose to arrange your own transportation, please inform Anna as well.
- Bus from Arlanda Airport to Rånäs castle departs September 30th at 11:00.
- Bus from Rånäs castle to Arlanda Airport departs October 3rd at 13:30.
It takes around 40 minutes to go by bus or car between Arlanda and Rånäs Castle.
Please note that if you want to book a taxi from Rånäs to Arlanda you need to book it one day ahead.The cost of the taxi is at your own expens
Contact info to taxi:
- Taxi Stockholm +46 (0)8-15 00 00
Book on Taxi Stockholm website - Sverige Taxi +46 (0)20-20 20 20
Book on Sverige Taxi website
Please see below information regarding how you can reach Rånäs Castle:
Program committee
Jonas Beskow, Professor, Division of speech, music and hearing, KTH Royal Institute of Technology in Stockholm, Sweden
Desmond Elliott, Associate Professor, Department of Computing Science, University of Copenhagen, Denmark
Johanna Björklund, Docent and Associate professor at Department of Computing Science at Umeå University, Sweden
Gustav Eje Henter, Docent and Assistant professor, Division of speech, music and hearing, KTH Royal Institute of Technology in Stockholm, Sweden
Erica Cooper, PhD and Senior Researcher at National Institute of Information and Communications Technology, Japan
Jonas Unger, Professor at Department of Science and Technology at Linköping University
