Comparison of Regularization and Optimization Methods for Process Recognition with Use of Deep Neural Network

Abstract

This paper brings the topic of comparison of optimization techniques using gradient descent, gradient descent with momentum, Adam and learning rate decay in combination with previous optimization algorithms for face recognition using deep neural network. This paper compares several settings of these techniques as well as techniques among themselves with combination of regularization techniques varying in using no regularization, L2 regularization and dropout to successfully recognize the sex of a person captured. The result of the evaluation is taken in a matter of successfully recognized face rate for training data and test data, cost and deep neural network learning time.