Explainability of time series models
Aito, Helena (2022)
Aito, Helena
2022
All rights reserved. This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:amk-2022122831651
https://urn.fi/URN:NBN:fi:amk-2022122831651
Tiivistelmä
The lack of interpretability of machine learning models is a drawback of their use. To better understand how the model works, how data affects its performance, how the model could be improved, and to gain trust in the model, investigating the model in more detail is necessary.
This thesis consists of two parts, model development and explainability analysis using S&P 500 index data. To find out what the baseline prediction accuracy is for S&P 500 index forecasting and to keep data preparation works as simple as possible, models were kept simple and features and responses were derived from S&P index data only. In total, five time series regression models were developed to predict S&P 500 index. Explainability of all five models was investigated both at a global and local level by using permutation importance, local interpretable model-agnostic explanations (LIME), and Shapley additive explanation (SHAP).
The best model was gradient boost. The prediction accuracy of the best model was considered sufficient both for a baseline version and explainability analysis. Model explainability was investigated using permutation importance, LIME, and SHAP. Structures of selected models were also visualized. Results from permutation importance, LIME, and SHAP were also compared to find out what the most important features are both on average and at a specific timestep. Potential next steps of this thesis could focus on using deep learning, such as long short-term memory network as well as investigating the usability and necessity of additional explainability methods.
This thesis consists of two parts, model development and explainability analysis using S&P 500 index data. To find out what the baseline prediction accuracy is for S&P 500 index forecasting and to keep data preparation works as simple as possible, models were kept simple and features and responses were derived from S&P index data only. In total, five time series regression models were developed to predict S&P 500 index. Explainability of all five models was investigated both at a global and local level by using permutation importance, local interpretable model-agnostic explanations (LIME), and Shapley additive explanation (SHAP).
The best model was gradient boost. The prediction accuracy of the best model was considered sufficient both for a baseline version and explainability analysis. Model explainability was investigated using permutation importance, LIME, and SHAP. Structures of selected models were also visualized. Results from permutation importance, LIME, and SHAP were also compared to find out what the most important features are both on average and at a specific timestep. Potential next steps of this thesis could focus on using deep learning, such as long short-term memory network as well as investigating the usability and necessity of additional explainability methods.