Data-Limited Learning of Complex Dynamical Systems

About the project

Objective
In the Data-Limited Learning of Complex Dynamical Systems (DLL) project, we develop methods and tools to learn to control complex dynamic systems using limited data samples. In contrast to traditional machine learning techniques that require large amounts of data for training, this project aims to utilize a priori knowledge of the system and combine such structural knowledge reliably with a limited number of data samples.

The project focuses on two application domains: (i) continuous bioprocessing for safer and more efficient production of biopharmaceuticals and (ii) reinforcement learning of cyber-physical systems in general and robotics in particular. In these domains, probing large amounts of data from the system can be expensive or even impossible without wearing out the system.

Background
Recent years have witnessed spectacular successes in applying machine learning tools to the decision-making and control of complex dynamical systems. These techniques typically combine reinforcement learning with large neural networks, thus requiring a tremendous amount of training data, even to learn simple tasks. Their applications have been mainly limited to specific scenarios, such as board and video games, where generating and gathering data is inexpensive. However, in many biological or physical application domains, data is limited, and probing the system for more data can be expensive or even impossible without destroying the system.

Cross-disciplinary collaboration
This project involves co-PIs and researchers from different disciplines, including computer science, automatic control, machine learning, and biotechnology. To make the interaction as fruitful as possible, the project is divided into three sub-projects: continuous bioprocessing, (ii) reinforcement learning in cyber-physical systems, and (iii) theory. The research involves fundamental theory and practical applications, including involvement from industrial partners. The collaboration between the research team at Digital Futures and the Competence Centre for Advanced BioProduction (AdBIOPRO) will establish a strong, visible, and sustainable research environment that will overarch digitalization and life science research at KTH.

Watch the recorded presentation at the Digitalize in Stockholm 2023 event:

Activities & Results

Find out what’s going on!

Activities, awards, and other outputs

• Tutorial at the IEEE International Conference on Signal Processing and Communications (SPCOM) 2022. Tutorial title: Generative models and role of deep neural networks. Lecturer: Saikat Chatterjee.
• Two-day Tutorial at Digital Futures on the Fundamentals of Bayesian Inference using Probabilistic Programming Probabilistic programming. Lecturer: David Broman.
• Distinguished Artifact Award at European Symposium on Programming (ESOP 2022), for the paper Compiling Universal Probabilistic Programming Languages with Efficient Parallel Sequential Monte Carlo Inference. David Broman and co-authors.
• Keynote at the Workshop on Nonlinear System Identification Benchmarks. Speaker: Håkan Hjalmarsson
• Organization of the Data-limited learning workshop (DLL) in fall 2021. Organized by David Broman together with Co-PIs.
• Organization of the MATH, AI, and Neuroscience (MAIN) workshop at Digital Futures in 2021. Organizer: Saikat Chatterjee
• Invited talk at the Stochastic Networks conference (Cornell in 2022). Speaker: Alexandre Proutiere
• Keynote at YEQTIV Eindhoven and in the SNAPP seminar series. Speaker: Alexandre Proutiere
• The organizer of the International Conference on Embedded Software (EMSOFT), the premier venue for embedded software, 2022. PC Chair: David Broman

Results

This project has so far resulted in several fundamental research results. The results include but are not limited to: a new biological modelling approach that includes transcriptional information, new system identification techniques for differential-algebraic equations subject to disturbances, fundamental limits for sample complexity and lower bounds for the regret of deterministic discrete dynamical systems.

The results are published in top-tier journals and conferences such as ICML, NeurIPS, and CDC.

Publications

1. Filippo Vannella, Jaeseong Jeong, and Alexandre Proutiere. Off-Policy Learning in Contextual Bandits for Remote Electrical Tilt Optimization. IEEE transactions on Vehicular Technology, 2022.
2. Filippo Vannella, Alexandre Proutiere, Yassir Jedra, and Jaeseong Jeong. Remote Electrical Tilt Optimization: a contextual bandit approach. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM) 2022.
3. Daniel Lundén, Joey Öhman, Jan Kudlicka, Viktor Senderov, Fredrik Ronquist, and David Broman. Compiling Universal Probabilistic Programming Languages with Efficient Parallel Sequential Monte Carlo Inference. In the Proceedings of 31st European Symposium on Programming (ESOP), 2022.
4. Kévin Colin, Mina Férizbegovic, and Håkan Hjalmarsson, Regret Minimization for Linear Quadratic Adaptive Controllers using Fisher Feedback Exploration, IEEE Control Systems Letters and In Proceedings of the 61th IEEE Conference on Decision and Control (CDC), 2022.
5. Yassir Jedra and Alexandre Proutiere. Minimal expected regret in online LQR. In Proceedings of the International Conference on Artificial Intelligence and Statistics(AISTATS), 2022.
6. Robert Bereza, Oscar Eriksson, Mohamed R.-H. Abdalmoaty, David Broman andHåkan Hjalmarsson. Stochastic Approximation for Identification of Non-Linear Differential-Algebraic Equations with Process Disturbances, In Proceedings of the IEEE Conference on Decision and Control (CDC), 2022.
7. A Ghosh, A.E. Fontcurbeta, Mohamed R. Abdalmoaty, Saikat Chatterjee, Time-varying Normalizing Flow for Generative Modeling of Dynamical Signals, European Signal Processing Conference (EUSIPCO), 2022.
8. S.Das, A.M. Javid, P.B. Gohain, Y.C. Eldar, S. Chatterjee. Neural Greedy Pursuit for Feature Selection, In the Proceedings of the International Joint Conference on Neural Networks (IJCNN), IEEE WCCI 2022.
9. X Liang, AM Javid, M Skoglund, S Chatterjee, Decentralized learning of randomization-based neural networks with centralized equivalence, AppliedSoft Computing, 2022.
10. David Broman. Interactive Programmatic Modeling. In ACM Transactions on Embedded Computing Systems (TECS), Volume 20, Issue 4, Article No 33, Pages 1-26, ACM, 2021.
11. Mohamed R.-H. Abdalmoaty, Oscar Eriksson, Robert Bereza, David Broman and Håkan Hjalmarsson. Identification of Non-Linear Differential-Algebraic Equation Models with Process Disturbances, In Proceedings of the IEEE Conference on Decision and Control (CDC), 2021.
12. Mina Ferizbegovic, Per Mattsson, Thomas Schön, and H. Hjalmarsson. Bayes Control of Hammerstein Systems, 19th IFAC Symposium on System Identification, 2021.
13. Daniel Lundén, Johannes Borgström, and David Broman. Correctness of Sequential Monte Carlo Inference for Probabilistic Programming Languages. In Proceedings of 30th European Symposium on Programming (ESOP), LNCS vol. 12648, Springer, 2021.
14. Viktor Palmkvist, Elias Castegren, Philipp Haller, and David Broman. Resolvable Ambiguity: Principled Resolution of Syntactically Ambiguous Programs. In Proceedings of the 30th ACM SIGPLAN International Conference on Compiler Construction (CC), ACM, 2021.
15. A.M. Javid, S. Das, M. Skoglund and S. Chatterjee, A ReLU Dense Layer to Improve the Performance of Neural Networks, in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021.
16. X. Liang, M. Skoglund, and S. Chatterjee, Feature reuse for a randomization based neural network, in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021.
17. P.G. Jurado X. Liang, A.M. Javid, and S. Chatterjee, Use of deterministic transforms to design weight matrices of a neural network, in European Signal Processing Conference (EUSIPCO), 2021.
18. Díogo Rodrigues, Mohamed R. Abdalmoaty, Elling W. Jacobsen, Véronqiue Chotteau, and Håkan Hjalmarsson. An Integrated Approach for Modeling and Identification of Perfusion Bioreactors via Basis Flux Modes, Computers & Chemical Engineering, 2021.
19. Aymen Al Marjani and Alexandre Proutiere. Adaptive Sampling for Best Policy Identification in Markov Decision Processes. In Proceedings of International Conference on Machine Learning (ICML), 2021.
20. Aymen Al Marjani, Aurélien Garivier, and Alexandre Proutiere: Navigating to the Best Policy in Markov Decision Processes. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), 2021.
21. Damianos Tranos and Alexandre Proutiere: Regret Analysis in Deterministic Reinforcement Learning. In Proceedings of the IEEE Conference on Decision and Control (CDC), 2021.
22. Zhang, L., Wang, M., Castan, A., Hjalmarsson, H. and Chotteau, V., Probabilistic model by Bayesian network for the prediction of antibody glycosylation in perfusion and fed‐batch cell cultures. Biotechnology and Bioengineering, 118(9), pp.3447-3459, 2021.
23. Chotteau, V., Hagrot, E., Zhang, L., Mäkinen, M. Mathematical modelling of cell culture processes. In Cell Culture Engineering and Technology (pp. 467-484). Springer, Cham, 2021.
24. Mingliang Wang, Riccardo S. Risuleo, Elling W. Jacobsen, Véronique Chotteau and Håkan Hjalmarsson. Identification of Nonlinear Kinetics of Macroscopic Bio-reactions Using Multilinear Gaussian Processes, Computers and Chemical Engineering, 123(2), 2020.
25. Stefanie Fonken, Mina Ferizbegovic, and H. Hjalmarsson. Consistent Identification of Dynamic Networks Subject to White Noise Using Weighted Null-Space Fitting, In Proceedings of the IFAC 2020 World Congress, 2020.
26. Riccardo S. Risuleo and Håkan Hjalmarsson. Nonparametric Models for Hammerstein-Wiener and Wiener-Hammerstein System Identification, In Proceedings of the IFAC 2020 World Congress, 2020.
27. Saranya Natarajan and David Broman. Temporal Property-Based Testing of a Timed C Compiler using Time-Flow Graph Semantics. In the Proceedings of the Forum on specification & Design Languages (FDL 2020), IEEE, 2020.
28. Mohamed R. Abdalmoaty and Håkan Hjalmarsson. Identification of Stochastic Nonlinear Models Using Optimal Estimating Functions, Automatica, 119, 2020. Mina Ferizbegovic, Jack Umenberger, Håkan Hjalmarsson and Thomas Schön. Learning robust LQ-controllers using application oriented exploration, IEEE Control Systems Letters, 4(4):19-24 and jointly published in the Proceedings of the IEEE Conference on Decision and Control (CDC), 2020.
29. A.M. Javid, A. Venkitaraman, M. Skoglund, S. Chatterjee, High-dimensional neural feature design for layer-wise reduction of training cost, EURASIP Journal on Advances in Signal Processing, 2020.
30. Yassir Jedra and Alexandre Proutiere. Finite-time Identification of Stable Linear Systems Optimality of the Least-Squares Estimator. In Proceedings of the IEEE Conference on Decision and Control (CDC), 2020.
31. Filippo Vannella, Jaeseong Jeong, and Alexandre Proutiere. Off-policy Learning for Remote Electrical Tilt Optimization. VTC-Fall, 2020.
32. Jack Umenberger, Mina Ferizbegovic, Thomas Schön and Håkan Hjalmarsson. Robust exploration in linear quadratic reinforcement learning, In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), 2019 (Spotlight paper).
33. Yassir Jedra, Alexandre Proutiere. Sample Complexity Lower Bounds for Linear System Identification. In Proceedings of the IEEE Conference on Decision and Control (CDC), 2019.