Deep Learning Approaches for Long-term Future Forecasting

September 2020 – August 2022

Objective
This project aims to develop a deep learning-based methodology to enhance the ability to model complicated dynamics for sequential data. With a special focus on the recent progress of transformer-based models, which have shown great potential in modelling very long sequences, we are inspired to integrate them with other state-of-the-art techniques, such as learning dynamic structures and self-supervised learning. By exploring such directions, we expect our results to be applicable to the sequence modelling research and provide good insights for other fundamental deep learning research areas.

Background
Sequence modelling is the fundamental problem of other time series related tasks, including future forecasting. Since being proposed in 2018, transformers have become the de facto choice for most sequence modelling tasks due to their superior performance over traditional RNN-based approaches. However, it appears that transformers usually need a significant amount of training data to achieve their full potential, making them an expensive and impractical option for many real-world scenarios. Thus, it becomes increasingly imperative to develop methods to effectively train transformers with limited labelled data, which is quite common for sequence modelling.

About the Digital Futures Postdoc Fellow
Hao Hu is a postdoc researcher at KTH RPL working with Hossein Azizpour. Before joining KTH, he worked as a research scientist in FX Palo Alto Laboratory (FXPAL), California, United States. Hao got his PhD in Computer Science from the University of Central Florida (UCF) in 2019. His research interests include various topics in machine learning and computer vision, with a special focus on temporal modelling and deep learning.

Main supervisor
Hossein Azizpour, Assistant Professor, Robotics, Perception and Learning at KTH

Co-supervisor
Arne Elofsson, Professor in Bioinformatics at Stockholm University