Defence of dissertation in the field of computer science, Alexander Grigorevskiy, M.Sc. (Tech.)

In the dissertation "Advances in Randomly-Weighted Neural Networks and Temporal Gaussian Processes" it was discovered that back-propagation is not the only way to train neural networks. Randomly assigning the weights of the first layers and using the ordinary least-squares often provides much faster but comparable alternative. In addition, the computational speed of modeling uncertainty for the time series prediction problem has been significantly improved.

Neural networks are the most popular machine learning models these days. They are used extensively in industry and attract a large number of researchers. However, they are still trained by the slow back-propagation algorithm or its newer versions. In this dissertation, the Randomly-Weighted Neural Networks (RWNN) have been investigated. They are also known as Extreme Learning Machines (ELM). The vanilla RWNN consists of two layers. The weights of the first layer are randomly generated and never updated. The weights of the second layer may be computed by the simple algorithm - ordinary least-squares. This algorithm is much faster than back-propagation. In many problems, the accuracy of RWNNs is close to the state-of-the-art. In this dissertations the new RWNN algorithm has been proposed which is slightly slower than vanilla RWNN, but the resulting neural network is much smaller and more accurate.

This new algorithm has been applied to the time series prediction (TSP) problem. Several strategies for time series prediction using RWNNs have been studied and it has been found that DirRec strategy is the best one on average. Time series prediction and modeling have been considered also from the probabilistic point of view. Probabilistic modeling has an advantage that it easily allows to estimate uncertainties. The convenient machine learning models for these tasks are Gaussian Processes and State-Space models. The connection between them has been elucidated, which can help researchers to better apply these models. Finally, the contribution has been made to speed up the Gaussian Process models for time series data.

The contributions of this dissertation are useful in several ways. Randomly weighted neural networks are favorable in situations when fast training time is required. So, they are a nice addition to the collection of machine learning models. Time series modeling and prediction is a very relevant topic with applications in industry, ecology, economy and so forth. Modeling the uncertainties of time series prediction facilitate better decision making. 

Opponent: Professor Tommi Kärkkäinen, University of Jyväskylä

Custos: Professor Aki Vehtari, Aalto University School of Science, Department of Computer Science

Contact information: Alexander Grigorevskiy, Department of Computer Science, [email protected]