Wednesday, October 19, 2011

Self-administered questionnaires

In survey research, especially when questions related to sensitive topics are being asked, there are debates between which form of questionnaire administration is best: a) interviewer-administration or b) self-administration. More often that not, questionnaires are administered by a trained interviewer; however, there are times that some people feel its best that the respondent completes the questionnaire without the assistance of an interviewer (self-administered).

Currently, I'm dealing with survey data from a youth study that used a self-administered questionnaire and the data contain many "missing" cases, nonsensical responses, and numerous cases of Errors of Commission and Errors of Omission. An Error of Commission is one where the person responds where they should not and an Error of Omission is one where the person fails to respond when they should.

A questionnaire designed for an interviewer administered survey cannot be used for a self-administered survey! Interviewers are trained in understanding the questions and how to navigate through the questionnaire; however, a questionnaire that is designed for someone who has never seen it before and for them to understand the questions as well as navigate through the questionnaire requires special attention to many factors. Using pg.6 from the 2008 National Survey of College Graduates, conducted by the US Census Bureau to illustrate, some critical factors to consider for a self-administered questionnaire are:

  • Language - the instructions and questions need to be written in a vocabulary that is slightly lower than lowest education level of any respondent.
  • Section Heading - every section/topic needs a heading that is short, in bold font, slightly different color than the rest of the questionnaire, such as Part B - Past Employment.
  • Question Numbering - question numbers should carry the section lettering/numbering as well as the question number and should be in a slightly larger font than the question text and in bold font, such as B1.
  • Verbal navigation - instructions next to certain responses that tell the respondent where to clearly go next. In the example above, if the respondent answers "No" in question B1 there is a verbal instruction, in bold font, telling them both 1) the page and 2) the question # to go to.
  • Symbol navigation - these are generally arrows showing a respondent where to go next if they answer a certain response. Above, if a respondent answers "Yes" in question B1 the arrow shows them to go to question B2.
  • Adequate spacing - all to often to save printing costs, a questionnaire is too cluttered but generally this is ok for a trained interviewer but not for self-administration. A self-administered questionnaire should have adequate spacing between questions to reduce eye fatigue and confusion.
  • Coloring - if posible, use slightly different grays or colors to highlight different sections and responses, such as in the example above.