Research Repository

A three-stage learning algorithm for deep multilayer perceptron with effective weight initialisation based on sparse auto-encoder

Almulla Khalaf, Maysa Ibrahem and Gan, John Q (2019) 'A three-stage learning algorithm for deep multilayer perceptron with effective weight initialisation based on sparse auto-encoder.' Artificial Intelligence Research, 8 (1). 41 - 41. ISSN 1927-6982

[img]
Preview
Text
14835-54049-1-PB.pdf - Published Version

Download (728kB) | Preview

Abstract

A three-stage learning algorithm for deep multilayer perceptron (DMLP) with effective weight initialisation based on sparse auto-encoder is proposed in this paper, which aims to overcome difficulties in training deep neural networks with limited training data in high-dimensional feature space. At the first stage, unsupervised learning is adopted using sparse auto-encoder to obtain the initial weights of the feature extraction layers of the DMLP. At the second stage, error back-propagation is used to train the DMLP by fixing the weights obtained at the first stage for its feature extraction layers. At the third stage, all the weights of the DMLP obtained at the second stage are refined by error back-propagation. Network structures and values of learning parameters are determined through cross-validation, and test datasets unseen in the cross-validation are used to evaluate the performance of the DMLP trained using the three-stage learning algorithm. Experimental results show that the proposed method is effective in combating overfitting in training deep neural networks.

Item Type: Article
Uncontrolled Keywords: Sparse auto-encoder, Deep learning, Feature learning, Effective weight initialization
Divisions: Faculty of Science and Health > Computer Science and Electronic Engineering, School of
Depositing User: Elements
Date Deposited: 06 Aug 2019 12:43
Last Modified: 06 Aug 2019 13:15
URI: http://repository.essex.ac.uk/id/eprint/25114

Actions (login required)

View Item View Item