Identifying unintended responses in Census questionnaires
by
Allyson Seyb
Survey Methods Division Statistics New Zealand

On Tuesday 6th March 2001 everyone in New Zealand will fill in a Population Census questionnaire-or have one filled in for them if they are too young -or too old- to fill one in themselves. The questionnaires are collected shortly after that and brought to Christchurch where the process of reading and recording the responses begins. The forms are scanned and the responses "recognised" using a process called Optical Mark Recognition (OMR). OMR can recognise a mark in a tick box with over 99.9However, OMR cannot differentiate between true responses and unintended responses.

Unintended responses are marks in tick boxes such as a crossed out dash or a mark made by a pen resting in the box or ink leaking through from the back of the form. All of these marks are recognised as true responses Most unintended marks show up as multiple responses to a question. How a multiple response is dealt with depends on how important the question is deemed to be. Multiple responses in questions that are important-we call them foremost questions-like gender or ethnicity- are seen by an operator. This means that they are the key outputs and every effort is being made toensure they are of the highest quality possible.

In the 1996 Population Census tick box recognition couldn't distinguish between ticks, crossings out and unintentional marks. Some of these marks were then subjected to automated edits, adding errors and a bias to the census results. The Quality Management Strategy for the 2001 Population Census states that all multiple responses in important questions will be brought to the attention of the operators. Multiple responses in less important questions will be subject to automated edits.

This project aimed to investigate if it is possible to identify unintended responses using the standard output fields from recognition. And if this was possible, then it aimed to investigate the effects on data quality.

The study was carried out using data from two of the "dress rehearsal" tests of our systems that happen in the lead-up to a census and data from a sample of forms from the census in 1996.

Date received: August 23, 2000

Copyright © 2000 by the author(s). The author(s) of this document and the organizers of the conference have granted their consent to include this abstract in Atlas Conferences Inc. Document # cadt-34.