Architectural defect in the Cascade Correlation architecture
by
Parma Nand
University of the South Pacific
Coauthors: Dr. Anthony Adams (University of the South Pacific, Suva, Fiji)

Cascade Correlation is the name given to an artificial neural network architecture, which has been designed for classification problems. In terms of what it can classify, it is one of the most powerful architectures in existence. Its other main strength is the speed with which it can be trained, which is often at least an order of magnitude faster than alternative neural network methods.

A different class of problems requires analogue outputs and it is well known that Cascor performs very poorly on such problems.

This paper brings to light an architectural problem with Cascade Correlation which emerged as part of a research dedicated to finding a solution to analogue problems using the Cascaded architecture. Apart from getting steppy approximation to analogue problems there is an unusual spiky oscillation on the LHS of the test peak. With different starting values the oscillation can also appear on the RHS and, as will become clear, it is theoretically possible for such oscillations to appear multiple times on a peak. An analysis of where this comes from brings to light a problem of the Cascor architecture itself when applied to analogue problems. This problem although exists does not pose any problem for classification problems. Adams Waugh showed that the Cascor architecture is capable of fitting functions such as the test peak. However, in their work although the spike is clearly seen, its presence was not explained nor its relation to the architecture explored. A careful analysis shows that the Cascor architecture can indeed fit such functions but under certain circumstances there will always be the possibility of undesirable oscillations occurring. This paper shows that the spiky oscillation is caused by large magnitude weights, which also prevents smooth approximation of analogue problems. The architecture was successfully modified to eliminate the spikes but it was found that the approximation although smooth as desired, deteriorated in terms of RMS error. Reasonable solution to analogue problems were found by using various ways of constraining weights and it was found that if the weights were constrained within certain absolute magnitudes the problem of spiky oscillation was also eliminated.

Date received: September 5, 2000