Bayesian Hyperparameter Optimisation for Neural Networks

Abstract

As the popularity of artificial intelligence increases, the amount of resources needed to train the machine learning models for it also correspondingly grows. To develop a high performing model, an important step is to choose an optimal set of hyperparameters. However, the process of choosing and testing them can take significant amount of time and hence resources. Therefore, the purpose of this thesis was to apply the Bayesian hyperparameter optimisation method for a neural network aiming to classify breast cancer ultrasound images. This method explores the hyperparameter search space by learning from previous trails and hence reduces computational cost and environmental impact. The core research questions for this project were how the Bayesian hyperparameter optimisation method performs regarding model accuracy, timing and which hyperparameters hold the highest impact on model’s performance.

Firstly, this thesis provides theoretical background on neural networks, hyperparameters and hyperparameter optimisation methods. Secondly, the practical part of applying the Bayesian hyperparameter optimisation method is described and shown by code screen captures for key steps. Lastly, the results are evaluated and explained through various figures and summarised for final conclusions.

The results of this study showed that the Bayesian hyperparameter optimisation is an efficient method that takes significantly less time per training trial whilst also delivering reliable model performance. Furthermore, the size of the first fully connected layer was the most important hyperparameter regarding the model’s performance, showing the importance of model’s capacity to learn complex patterns for successful classification of ultrasound images. Additionally, further suggestions for development are discussed in the summary section of this thesis.