Nonlinear Modeling and Bayesian Information Criteria
by
Sadanori Konishi
Graduate School of Mathematics, Kyushu University, Fukuoka 812-8581, Japan

Modeling and model evaluation are crucial issues in various fields of natural and social sciences. In practice Akaike's (1973, 1974) information criterion AIC and Schwarz's (1978) Bayesian information criterion SIC have been widely used as criteria for evaluating models estimated by the maximum likelihood method. Now the wide availability of fast and inexpensive computers enables us to construct various types of nonlinear models for analyzing data with complex structure. Nonlinear models are generally characterized by a large number of parameters. In such cases the maximum likelihood method yields unstable parameter estimates and leads to overfitting.

The regularization techniques provide alternative estimation method that may be used to advantage in nonlinear modeling based on neural networks and splines. Then the parameter estimates may depend on hyperparameters that have to be determined from the observed data. We consider the model evaluation problem for nonlinear modeling from a Bayesian viewpoint and also an information-theoretic approach. We present criteria for evaluating various types of nonlinear models constructed by the regularization techniques.

Date received: October 13, 2000