Title
Automatic Optimization of Hyperparameters for Deep Convolutional Neural Networks: Grid Search Enhanced with Coordinate Ascent
Authors
Qingqing Song
Faculty of Applied Mathematics and Computer Science, Belarusian State University, Belarus
Shaoliang Xia
Faculty of Applied Mathematics and Computer Science, Belarusian State University, Belarus
Zhen Wu
Nanjing University of Aeronautics and Astronautics, China
Published In
MIDA'24: Proceedings of the 2024 International Conference on Machine Intelligence and Digital Applications
Published Date
03 August 2024
Abstract
Using the same depth convolutional neural network model will result in significantly different results due to different combinations of hyperparameters. By adjusting the configuration of hyperparameters, we can enhance the performance of the model. However, hyperparameter optimization typically requires a significant amount of computational resources and time. Therefore, improving the efficiency of hyperparameter optimization is crucial. In this study, we utilized the coordinate ascent method, which only offered initial candidate values for each hyperparameter. By changing only the hyperparameter that had the greatest effect on the model in each iteration, we gradually expanded the search grid until we achieved convergence in accuracy. This method enables us to effectively and automatically find hyperparameter combinations that can improve model accuracy. The experimental results show that using the MWD dataset, the model optimized through hyperparameters achieved an accuracy of 95.71% on the validation set, and this hyperparameter combination can be considered an approximate global optimal solution. In addition, the performance of the hyperparameter combination in its neighborhood is stable, which further proves the robustness of our hyperparameter optimization strategy.
Identifiers
DOI: 10.1145/3662739.3664743
WoS: 001304539800024
EI-Compendex: 20243416902562
Scopus: 2-s2.0-85201279150
OCLC: 10326197620
Links
ACM Digital Library
https://dl.acm.org/doi/abs/10.1145/3662739.3664743
Comment