Almulla Khalaf, Maysa Ibrahem and Gan, John Q (2019) A three-stage learning algorithm for deep multilayer perceptron with effective weight initialisation based on sparse auto-encoder. Artificial Intelligence Research, 8 (1). p. 41. DOI https://doi.org/10.5430/air.v8n1p41
Almulla Khalaf, Maysa Ibrahem and Gan, John Q (2019) A three-stage learning algorithm for deep multilayer perceptron with effective weight initialisation based on sparse auto-encoder. Artificial Intelligence Research, 8 (1). p. 41. DOI https://doi.org/10.5430/air.v8n1p41
Almulla Khalaf, Maysa Ibrahem and Gan, John Q (2019) A three-stage learning algorithm for deep multilayer perceptron with effective weight initialisation based on sparse auto-encoder. Artificial Intelligence Research, 8 (1). p. 41. DOI https://doi.org/10.5430/air.v8n1p41
Abstract
A three-stage learning algorithm for deep multilayer perceptron (DMLP) with effective weight initialisation based on sparse auto-encoder is proposed in this paper, which aims to overcome difficulties in training deep neural networks with limited training data in high-dimensional feature space. At the first stage, unsupervised learning is adopted using sparse auto-encoder to obtain the initial weights of the feature extraction layers of the DMLP. At the second stage, error back-propagation is used to train the DMLP by fixing the weights obtained at the first stage for its feature extraction layers. At the third stage, all the weights of the DMLP obtained at the second stage are refined by error back-propagation. Network structures and values of learning parameters are determined through cross-validation, and test datasets unseen in the cross-validation are used to evaluate the performance of the DMLP trained using the three-stage learning algorithm. Experimental results show that the proposed method is effective in combating overfitting in training deep neural networks.
Item Type: | Article |
---|---|
Uncontrolled Keywords: | Sparse auto-encoder, Deep learning, Feature learning, Effective weight initialization |
Divisions: | Faculty of Science and Health Faculty of Science and Health > Computer Science and Electronic Engineering, School of |
SWORD Depositor: | Unnamed user with email elements@essex.ac.uk |
Depositing User: | Unnamed user with email elements@essex.ac.uk |
Date Deposited: | 06 Aug 2019 12:43 |
Last Modified: | 23 Sep 2022 19:34 |
URI: | http://repository.essex.ac.uk/id/eprint/25114 |
Available files
Filename: 14835-54049-1-PB.pdf