PhD student in Computing Science with focus on Visual Language Grounding

Umeå University is offering a PhD student position in Computer Science with a focus on visual language grounding, in other words, the linking of elements of natural language (words, phrases, or sentences) to visual inputs (such as images or video) in a meaningful way. The position is part of an on-going multisite research project that investigates the possibilities and implications of AI-enabled conversational guides on visitors’ learning and experiences in public educational environments.

The Department of Computing Science has experienced rapid growth in recent years, with a strong emphasis on fostering an inclusive, collaborative, and bottom-up-driven research environment. Our workplace brings together a diverse community of individuals from a wide range of nationalities.

As a PhD student with us, you benefit from comprehensive career development support, opportunities for networking, and access to robust administrative and technical services—all within a setting that offers attractive employment conditions.

To learn more about the department, please visit:

https://www.umu.se/en/department-of-computing-science/

Project description

The advertised position is part of a project funded by the Marcus and Amalia Wallenberg Foundation (MAW). The project explores the possibilities and implications of AI-enabled conversational guides for enhancing visitors’ learning and experiences in public educational environments.

The PhD student will focus on addressing the challenge of visual blindness in large language models (LLMs)—that is, the inability of a model to effectively process or understand visual information. This work involves integrating visual encoders with language models to create multimodal systems. The emphasis is on agentic approaches, where an LLM interacts with visual tools, which may themselves be neural networks. Central challenges include enabling LLMs to reason about visual structures, designing appropriate benchmarks for the target domain, and determining which fusion architectures best balance performance, interpretability, and computational efficiency.

The student will collaborate closely with visualization researchers, AI specialists, research engineers, educational scientists, and pedagogues across Linköping University, Visualization Center C in Norrköping, and Umeå University to investigate the impact of emerging AI-driven educational technologies on learning in public educational spaces. The PhD student is expected to produce research outputs relevant to the fields of visualization and multimodal machine learning.

Admission requirements

The general admission requirements for doctoral studies are a second- cycle level degree, or completed course requirements of at least 240 ECTS credits, of which at least 60 ECTS credits are at second-cycle level, or have an equivalent education from abroad, or equivalent qualifications.

To be admitted to doctoral studies in the field of computer science, the applicant must have completed courses totaling at least 90 higher education credits in computer science or in subjects directly relevant to the specific specialization.

Requirements

We are looking for candidates who have:

• Strong programming skills, particularly in Python
• Solid analytical and mathematical abilities
• Experience with machine learning
• Strong communication skills and proficiency in English
• The ability to work independently as well as collaboratively in an interdisciplinary research environment

Meritorious Qualifications

It is considered an advantage if you have:

• Experience with academic writing
• Familiarity with large language models or multimodal systems
• An interest in visual reasoning, educational technology, or human–AI interaction
• Experience with neural networks for image or video understanding

Personal Qualities

We value candidates who are:

• Intellectually curious
• Self-motivated and well-organized
• Eager to engage with real-world educational contexts

About the position

The position provides you with the opportunity to pursue PhD studies in Computing Science for four years, with the goal of achieving the degree of Doctor in Computing Science. While the position is mainly devoted to PhD studies (at least 80% of the time), it may include up to 20% department service (usually teaching). If so, the total time for the position is extended accordingly, resulting in a maximum of five years.

The procedure for recruitment for the position is in accordance with the Higher Education Ordinance (chapter 12, 2§) and the decision regarding the position cannot be appealed.

The expected starting date is August 11, 2025 or as agreed.

Application

A complete application should contain the following documents:

• A cover letter including a description of your research interests, your reasons to apply for the position, and your contact information
• A curriculum vitae
• Reprints / copies of completed BSc and/or MSc theses and other relevant publications, if any
• Copies of degree certificates, including documentation of completed academic courses and obtained grades
• Documentation and description of other relevant experiences or competences.

The application must be written in English or Swedish. Attached documents must be in pdf format. Applications must be submitted electronically using the e-recruitment system of Umeå University, and be received no later than 2025-05-15.

The Department of Computing Science values gender diversity, and therefore particularly encourages women and those outside the gender binary to apply for the position.

For additional information, please contact Assoc. Prof. Johanna Björklund (johanna@cs.umu.se).