About the project
Objective
Large Multimodal Models (LμMs) have the potential to transform engineering education by supporting hands-on, experiential learning. LμMs can process images, audio, video, and other data types, making them ideal for supporting physical engineering design tasks. However, these tools must be carefully designed to align with educational theories and support, rather than hinder, student learning. This project aims to develop and evaluate a pedagogically-aligned virtual teaching assistant (μTA) powered by LμMs to support problem-solving with physical systems in real-world settings for engineering education. The project addresses the challenges students face when dealing with complex, ill-defined problems in engineering design courses and other experiential learning contexts and the limitations of current AI tools in these settings.
Background
Generative AI tools, like large language models (LLMs), have revolutionized education but remain largely confined to screen-based, text-centric tasks such as programming and writing. Recent advancements in Large Multimodal Models (LµMs) enable processing of diverse inputs, such as text, images, and videos, offering opportunities to extend AI’s benefits to experiential learning environments like workshops and labs. While current research focuses on screen-based applications, little is known about how LµMs can support hands-on, ill-defined problem-solving tasks central to engineering education. This project pioneers the integration of LµMs into these settings, co-designing tools with students and educators to foster skills critical for engineering innovation, their studies, and work success.
Crossdisciplinary collaboration
The project is led by two principal investigators from the KTH Royal Institute of Technology: Associate Professor Olga Viberg (Human Centered Technology/EECS) and Assistant Professor Richard Lee Davis (Learning in Engineering Sciences/ITM). This cross-disciplinary collaboration integrates Viberg’s expertise in the design and evaluation of educational technologies—with a strong focus on AI adoption in STEM education and participatory design methods—with Davis’s experience in designing AI-driven tools for experiential learning, integrating multimodal systems, and advancing pedagogical alignment for generative AI technologies.
About the project
Objective
This project aims to develop a collaborative spatial perception framework that constructs various levels of abstract representations in a city-scale area, incorporating LiDAR point clouds, RGBD images, and remote sensing images collected by various agents in a collaborative autonomous system.
Background
The concept of digital twins, involving the creation of virtual representations or models that accurately mirror physical entities or systems, has garnered growing research attention in the realm of smart cities. However, a critical challenge in realizing digital twins lies in efficiently collecting data and recreating the real world, a task that typically demands substantial human effort. To address this gap, autonomous robots, originally designed to reduce human workload, hold immense potential in shaping the future of digital twinning. These robots can potentially assume a pivotal role in autonomously creating and updating the complete mirroring of the physical world, paving the way for the next generation of digital twinning.
About the Digital Futures Postdoc Fellow
Yixi Cai completed his PhD degree in Robotics at Mechatronics and Robotic Systems (MaRS) Laboratory from Department of Mechanical Engineering, University of Hong Kong. His research focuses on efficient LiDAR-based mapping with applications on Robotics. During his PhD journey, he explored the potential of LiDAR technology to enhance the autonomous capabilities of mobile robots, particularly unmanned aerial vehicles (UAVs). He developed ikd-Tree, FAST-LIO2, and D-Map that have been widely used in LiDAR community. He is deeply interested in exploring elegant representations of the world, which would definitely unlock the boundless possibilities in Robotics.
You might find more information about him from his personal website: yixicai.com
Main supervisor
Patric Jensfelt, Professor, Head of Division of Robotics, Perception, and Learning at KTH Royal Institute of Technology, Digital Futures Faculty
Co-supervisor
Olov Andersson, Assistant Professor at Division of Robotics, Perception, and Learning at KTH Royal Institute of Technology, Digital Futures Faculty
About the project
Objective
The LATEL project aims to harness the potential of data generated by educational technologies to enhance the quality of education. The primary objectives are to identify and retain potential dropout students, motivate learners to achieve their educational goals, and support teachers in refining learning designs. By addressing the practical challenges of implementing learning analytics (LA) in educational institutions, the project seeks to develop a systemic and use-case-based approach to demonstrate how data and evidence can be utilized for informed decision-making. This involves showcasing the application of LA in a real KTH course, exploring its potential in a new KTH program, and examining the legal and ethical frameworks governing data use in learning analytics. Ultimately, the project aims to clarify the legal landscape and promote the value of LA in shaping the future of engineering education, providing a roadmap for data-driven insights and solutions to policy-related obstacles that impede the implementation of LA in universities.
Background
Learning Analytics (LA) is an interdisciplinary field that combines data science, psychology, education, and computer science to optimize learning experiences. By analyzing data from online learning platforms, student information systems, and other sources, LA provides insights into student behavior, learning processes, and institutional performance. Despite its potential to personalize learning and identify at-risk students, the practical application of LA faces significant challenges, particularly related to data identification, curation, and legal and ethical compliance. Many educational institutions struggle with poor student throughput and funding issues, highlighting the need for effective LA solutions.
Current research often focuses on empirical studies, lacking practical applications for implementing LA in academic settings. The LATEL project addresses these gaps by proposing a design-based, iterative approach to demonstrate how data can be used to enhance teaching and learning quality. By exploring legal, ethical, and practical issues, the project aims to provide actionable insights for educators and policymakers, ultimately transforming education through data-driven decision-making.
Cross-disciplinary collaboration
The LATEL project brings together a diverse team of experts from various fields to tackle the complexities of implementing learning analytics in educational settings.
- Dr. Mattias Wiggberg, the principal investigator, holds a PhD in Computer Science Didactics and has extensive experience in digital transformation and the involvement of AI in society. He also contributes with expertise in skills development and policy work in education.
- Dr. Joakim Lilliesköld, an Associate Professor in Systems Engineering Management, contributes his knowledge in engineering education development and digitalization, focusing on legal and system challenges.
- Dr. Olga Viberg, an Associate Professor in Media Technology, specializes in Technology Enhanced Learning and will guide the empirical case study on learning analytics at KTH.
- Dr. Thashmee Karunaratne, an Associate Professor in Digital Learning, brings her background in machine learning and computer science to explore digital transformation and data analytics.
- Dr. Stefan Hrastinski, a Professor with a focus on Digital Learning, offers his extensive research experience in digital learning and learning analytics.
This cross-disciplinary collaboration, supported by the Digital Futures Education Transformation Working Group, ensures a comprehensive approach to addressing the project’s objectives and achieving meaningful educational transformation.
About the project
Objective
Problem statement: Develop future sustainability agendas in an objective and data-driven manner, leveraging large language models (LLMs) to simultaneously address the social, economic and environmental needs of humanity and the planet.
The project has the following goals:
- Effectively utilize the Gemini ecosystem of large language models (LLMs) to analyze thousands of documents, yielding excellent performance in the context of sustainability.
- Perform a detailed analysis of synergies and tradeoffs among the 169 SDG targets, creating a detailed matrix of positive and negative interactions.
- Extend this analysis to find positive and negative connections between the 231 SDG indicators and the 9 Planetary Boundaries.
- Define a new set of goals, addressing the targets from both the SDG and PB frameworks.
Background
The 2030 Agenda for Sustainable Development, adopted by all United Nations Member States in 2015, is an ambitious blueprint for achieving a better and more sustainable future for all. It comprises 17 goals, 169 targets, and 231 measurable indicators that aim to address a wide array of global challenges, including poverty, inequality, climate change, environmental degradation, peace and justice. The intricate and interconnected nature of these goals means that progress in one area can often catalyze advancements in others.
However, this interconnectedness can also lead to unintended consequences where progress on certain goals may inadvertently hinder progress on others. While some of these interactions are straightforward and predictable, many remain complex and difficult to foresee. Nearly a decade after the launch of the 2030 Agenda, we have accumulated a wealth of data and developed new tools that enhance our ability to identify and understand the positive and negative interactions between the Sustainable Development Goals (SDGs). This understanding is crucial as we approach the deadline of the 2030 Agenda and begin to consider the design of goals for the Post-2030 Agenda. This is highlighted in a recent article in Nature by the PIs*. A critical component of this analysis is the Planetary Boundaries (PBs) framework, introduced by the Stockholm Resilience Center in 2009. The PBs provide a science-based framework for monitoring environmental thresholds that define a safe operating space for humanity. These boundaries are quantifiable and offer a valuable tool for assessing environmental sustainability, although they do not directly address the social dimensions that are integral to the SDG Agenda.
*Extending the SDGs to 2050 – a roadmap. Fuso-Nerini et al., Nature (2024)
Crossdisciplinary collaboration
The PIs have a long history of cross-disciplinary collaboration, combining the AI knowledge of Prof. Vinuesa with the sustainability and systems modeling of Prof. Fuso-Nerini.
About the project
Objective
The project aims to develop a system for online rehabilitation of patients with long-term cognitive symptoms after Covid-19.
The project will start with an analysis and concretization of patients’ and clinicians’ demands on the existing pilot system. This information will then be used to develop the system for testing further at the Rehabilitation Medicine University Clinic Stockholm at Danderyd Hospital.
Background
The focus is on patients with severe cognitive problems, e.g. fatigue, concentration problems and memory disorders, after Covid-19. The system to be developed aims to help patients understand more about their symptoms, train to overcome them, and increase their motivation to work with their rehabilitation online.
The project also aims to apply an innovative visualization technology that can support workflows and information flows as well as the integration of applications. The technology can support care personnel so that they can easily receive the results from the patients’ online rehabilitation training and provide support in order to further increase the patients’ motivation.
The project collaborates with the clinical team at Rehabilitation Medicine University Clinic Stockholm, Danderyd Hospital and Visuera Integration.
Crossdisciplinary collaboration
The project partners are Region Stockholm, Danderyd University Hospital, Visuera Integration AB and Telia Company.
Watch the recorded presentation at the Digitalize in Stockholm 2023 event:
About the project
Objective
This seed project aims to support the Digital Futures (DF) Digitalized Industry Working Group by strengthening the international research focus on industrial digitalization, driving sustainability and addressing related complexity from a systems thinking and engineering point of view. The overall objectives of this seed project are (1) to create a European network of leading research institutions, industry and other stakeholders and (2) to write a proposal for a suitable EU project to support this network. We primarily target an EU COST Action but will also consider other instruments.
Background
There are common challenges in sustainability in the manufacturing, transport, energy, water management, and building sectors, as well as related methodologies to deal with the associated socio-technical complexity. There is a fragmentation of competencies in the use of digitalization to support sustainable development and also related to systems thinking/engineering. With the targeted network, we intend to try to overcome this fragmentation.
Cross-disciplinary collaboration
The PI and co-PIs are spread over two different KTH schools. Each PI needs international collaboration as we aim to apply for a proposal to build an EU network. The partners in the ongoing proposal involve eleven countries so far, >50% COSTInclusiveness Target Countries. We have had one in-person meeting at KTH on 12-13 August 2024 (see photo) and plan for the next meeting in person on 14 October 2024. Several online meetings have taken place since the start of the project.
Contacts: Ellen Bergseth, Martin Törngren, Yongkuk Jeong
About the project
Objective
We have two key objectives:
- Understand the extent of diversity at multiple spatial scales in biological systems
- Build an interdisciplinary community in Digital Futures to develop computational models that account for the experimental observed heterogeneities
To this end besides extensive literature surveys we will organize thematic workshops to find synergies among digital future researchers. Besides, we will bring together international experts in the area for a conference at Digital Futures.
Background
Biological systems are shaped by their evolutionary history and adaptation animals acquire throughout their life. This implies that even if two beings exhibit similar behavior the underlying physiological mechanism could be different and vice versa. ‘Degeneracy’ and diversity can be seen at every scale in biological systems: from gene expressions to behavior and can be quantitatively observed in heavy-tailed distribution of physiological parameters. Parameters that describe a biological system usually do not follow the classical Gaussian (or exponential family) distribution.
This biological reality is in stark contrast to how we build computational models of biological systems: Either take a data-driven or normative approach. Data-driven models rely on extensive data but typically we end up with an average model. Normative modeling on the other hand seeks to model biological systems assuming (based on data or some physical/chemical principles) that the system is ‘optimized’ in some sense. Both approaches ignore the fact that cohorts of biological systems do not follow normal distribution and each animal is optimized in their own way.
In this project we want to better understand the structure of the non-normal nature of biological data and how it can be better captured in our computational models.
Cross-disciplinary collaboration
We will focus on three specific spatial scales i.e. molecular level (i.e. genome, transcriptome and proteome), tissue/organ level (i.e. neural networks, cellular physiology, multiscale tissue mechanics, computational human models) and behavior level (i.e. decision-making, motor control). While the data at each level is different we expect there will be some common principles that relate diversity at different levels.
Contacts: Arvind Kumar, Xaiogai Li and Lanie Gutierrez-Farewik