“Real world” statistical data and the role of the consultant statistician
by
Jean Thompson
Statistical Consultant, JAD Associates, Wellington

At the beginning of 1962 when I joined the Applied Maths Laboratory of the DSIR in Wellington, the statistical work I got involved in substantially comprised helping various agricultural, biological and ecological scientists analyse their data. I was to learn quickly that unless the statistician had a part in designing the data collection system, even exhaustive and highly-sophisticated analysis could often yield little new information. Sadly, in essence, not much has changed in the intervening 39 years.

There is still a perception in many quarters that the statistician is the “necessary evil” to get the data analysed and frequently they are not consulted until long after an investigation is well under-way. This is a great shame, for while huge advances have been made in the methods we can use to analyse data and the facility with which it can be done, there is still the vexatious issue of data quality. In the ideal world one would be part of the whole process but it is more usual for a consultant to be brought in late in the day often after much damage has been done. However, we must be careful not to necessarily blame the investigator for hard-to-handle data. Some areas of application simply do not produce nice clean data no matter how one tries, even with good input from a statistician.

It was a good 20 years ago that the Applied Maths Division of the DSIR started commercial consulting and thereby opened the floodgates for highly variable data. After its demise in 1992 I continued as a private consultant doing the same sort of things I had been doing for my public sector employers, helping researchers and people in business understand the processes they cared about.

In this talk I will summarize some of the situations I have encountered over the last 5 years and focus on what I see as important issues for applied statisticians and their teachers.

The philosophy and material I will present was given in a one-hour seminar at the University of Canterbury in April 2001.

Date received: August 30, 2001

Copyright © 2001 by the author(s). The author(s) of this document and the organizers of the conference have granted their consent to include this abstract in Atlas Conferences Inc. Document # cahg-49.