Back to news

Active model combination for time series forecasting

19 May 2013

Devon Barrow obtains his PhD in Forecasting with his thesis entitled 'Active model combination for time series forecasting: an empirical evaluation and extension of Bagging and Boosting'.

Devon successfully completed the Viva Voce defence of his thesis in early December 2012 under the supervision of Dr Sven Crone. The abstract of his thesis is given below:

Since the seminal work by Bates and Granger (1969), the practice of combining two or more models, rather than selecting the single best, has consistently been shown to lead to improvements in accuracy. In forecasting, model combination aims to find an optimal weighting given a set of precalculated forecasts. In contrast, machine learning includes methods which simultaneously optimise individual models and the weights used to combine them. Bagging and boosting combine the results of complementary and diverse models generated by actively perturbing, reweighting and resampling training data. Despite large gains in predictive accuracy in classification, limited research assesses their efficacy on time series data.

This thesis provides a critical review of the combination literature, and is the first literature survey of boosting for time series forecasting. The lack of rigorous empirical evidence on forecast accuracy of Bagging and boosting is identified as a major gap. To address this, a rigorous evaluation of Bagging and boosting adhering to recommendations of the forecasting literature is performed using robust error measures on a large set of real time series, exhibiting a representative set of features and dataset properties. Additionally there is a narrow focus on marginal extensions of boosting, and limited evidence of any gains in accuracy.

A novel framework is proposed to explore the impact of varying boosting meta-parameters, and to evaluate the empirical accuracy of the resulting 96 boosting variants. The choice of base model and combination size are found to have the largest impact on forecast accuracy. Findings show that boosting overfits to noisy data, however no existing study investigates this crucial issue. New noise robust boosting methods are developed and evaluated for time series forecast models. They are found to significantly improve accuracy above current boosting approaches and Bagging.