Pablo Montero Manso, professor at the University of Sydney and currently a visiting researcher at Google, will give a lecture on July 11 at 12:00 at the UDC Faculty of Computer Science, which can also be followed through this link. In his lecture, promoted by the MODES research group, Montero Manso will present results on how large pre-trained neural networks can become near-optimal statistical estimators, with significant applications in time series forecasting.

Montero Manso will analyze how these networks, trained on an extensive number of simulations derived from data generation processes, can outperform the very families of models that generate such data. For example, a neural network trained on ARIMA (AutoRegressive Integrated Moving Average) simulations can become more accurate in predicting new observations of true ARIMA processes than a canonical Maximum Likelihood + AIC estimator for ARIMA. 

Montero Manso proposes in one of the examples in the study an approximate solution to a problem that has challenged the time series community for decades: the puzzle of combining forecasting models, or how to find the optimal weights for the combination of models.

These “artificial estimators” have demonstrated remarkable performance on real data. A “simple” network trained on combinations of ARIMA and Exponential Smoothing simulations with Student-t innovations achieved Top-1 accuracy on many benchmark data sets, outperforming even extremely sophisticated ad-hoc neural network architectures and other Machine Learning models.

Montero Manso’s study also addresses the limitations of the paradigm and the results of the No-Free-Lunch theorem. Although training these large models is expensive, it is affordable with commodity hardware and inference is fast. 

About Pablo Montero Manso

Pablo Montero Manso is a senior lecturer at the University of Sydney and currently a visiting researcher at Google. His research focuses on machine learning, artificial intelligence and statistical tools for time series analysis, covering forecasting, classification, clustering and data visualization. Pablo has developed award-winning predictive models in major forecasting competitions such as M4 and M6, and his models have been adopted by industry.

During the COVID-19 pandemic, Montero Manso contributed with accurate models and predictions on the evolution of the pandemic, which were part of the decision-making process in Australia, Spain and the European Union. In addition, he is a member of the advisory board of the WHY project, which analyzes European household energy consumption for policy making, and is an author and contributor to several open-source tools and datasets in Python and R, including the popular TSclust package for time series clustering.