WASP, through the Research Arena for Media & Language, AI Sweden, and RISE, are working together to develop large-scale generative language models for the Nordic languages. The model now released, GPT-SW3, is the first truly large-scale generative language model for the Swedish language. Based on the same technical principles as the much-discussed GPT-3, GPT-SW3 will help Swedish organizations build a new generation of language applications.
The current GPT-SW3 models are trained on Linköpings University’s supercomputer, Berzelius, using the Nemo Megatron framework from NVIDIA. The pre-release is an important step in the process of knowledge building, validating the model and collecting feedback on both what works well, and what could be improved.
All applicants will have to approve a license and go through manual approval before the model is provided. The pre-release is intended for organizations and individuals in the Nordic NLP ecosystem.
Published: January 26th, 2023