Comparing Adaptive Designs of Computer Experiments for Global Fit of Response Surfaces
by
Chen Quin Lam
The Ohio State University
Coauthors: William Notz

Computer simulations have become increasingly popular for representing the complex physical reality in a set of computer codes. Due to limited computing capabilities to carry out these simulations at very fine grids, computer experiments have often been performed to approximate the unknown response surface given sparse observations. While space-filling designs, such as Maximin distance and Latin Hypercube designs, are useful for initial exploratory purposes, they do not allow the addition of designs points iteratively while still maintaining the space-filling property. Other complex designs based on certain optimality criteria (e.g., mean squared prediction error and maximum entropy) can be easily converted into sequential designs and the selection of new design points are based on optimizing the criterion at each stage. Alternative approaches, such as cross validation, have also been proposed. The basic idea of the cross validation approach is to leave one observation out (as an example), predict the response at this point based on the remaining observations and hence provide a ``semi-parametric'' form of the prediction error which is then used to select the design points.

While many sequential designs have been proposed, it is not clear how the performance of these methods might be affected by the type of response surface, choice of the correlation function, size of initial starting designs etc. We propose some new criteria for the cross validation approach and a modified Expected Improvement criterion for global fit (which is originally proposed for global optimization). An empirical study is conducted to compare the predictive performance of the various methods against one another and also against a fixed point design (e.g., Maximin Latin Hypercube design). Preliminary results reveal that sequential designs are potentially superior but careful implementation of these methods needs to be considered.

PDF

Date received: August 16, 2007