The physical correlates of local minima

Abstract

The training of neural-net classifiers is often hampered by the occurrence of local minima, which results in the attainment of inferior classification performance. We study the problem of local minima in order to devise means of alleviating it. In order to establish a better understanding of the problem, the nature of the physical states of neural nets stuck in local minima, is investigated. We show that the occurrence of a local minimum in the crite­rion function can often be related to specific patterns of defects in the classifier. In particular, three main causes for local minima are identified. Such an understanding of the physical correlates of local minima is important, since it suggests sensible ways of choosing the weights from which the training process is initiated.