Chapter 14 Summary

Identify and discuss the critical factors that can contribute to directly improving the accuracy of surveys, and explain why questionnaire development is not a simple process.

Questionnaire development is much more than just writing a set of questions and asking people to answer them. Designing good surveys goes beyond just developing reliable and valid scale measurements. There are a number of design factors, systematic procedural steps, and rules of logic that must be considered in the development process. In addition, the process requires knowledge of sampling plans, construct development, scale measurement, and types of data. It is important to remember that a questionnaire is a set of questions/scales designed to generate enough raw data to allow the researcher and decision maker to generate information to solve the business problem.

Discuss the theoretical principles of questionnaire design, and explain why a questionnaire is more than just asking a respondent some questions.

Many researchers, unaware of the underlying theory, still believe that questionnaire designing is an art rather than a science. Questionnaires are, however, hierarchial structures consisting of four different components: words, questions, formats, and hypotheses. Most surveys are descriptive instruments that rely heavily on the collection of state-of-being or state-of-behavior data; others are predictive instruments that focus on collecting state-of-mind and state-of-intention data that allow for predicting changes in people's attitudes and behaviors as well as testing hypotheses.

Identify and explain the communication roles of questionnaires in the data collection process.
Good questionnaires allow researchers to gain a true report of the respondent's attitudes, preferences, beliefs, feelings, behavioral intentions, and actions/reactions in a holistic manner, not just a fragment. Through carefully worded questions and clear instructions, a researcher has the ability to control a respondent's thoughts and ensure objectivity. By understanding good communication principles, researchers can avoid bad questioning procedures that might result in either incomprehensible information requests, unanswerable questions, or leading questions that obscure, prohibit, or distort the meaning of a person's responses.

Explain why the type of information needed to address a decision maker's questions and problems will substantially influence the structure and content of questionnaires.

Once research objectives are transformed into information objectives, determining the specific information requirements plays a critical role in the development of questionnaires. For each information objective, the researcher must be able to determine the types of raw data (state of being, mind, behavior, or intentions); types of question/scale measurement formats (nominal, ordinal, interval, or ratio); types of question structures (open-ended and closed-ended); and the appropriate selection of scale point descriptors. Researchers must be aware of the impact that different data collection methods (personal, telephone, self-administered, computer-assisted, etc.,) have on the wording of both questions and response choices.

List and discuss the 11 steps in the questionnaire development process, and tell how to eliminate some common mistakes in questionnaire designs.
Using their knowledge of construct development and scale measurement development (Chapter 12) and attitude measurement (Chapter 13), researchers can follow an 11-step process to develop scientific survey instruments. Refer back to Exhibit 14.4 which lists these steps.

Discuss and employ the "flowerpot" approach in developing scientific questionnaires. 

The flowerpot approach serves as a unique framework or blueprint for integrating different sets of questions and scale measurements into scientific structure for collecting high-quality raw data. This ordered approach helps researchers make critical decisions regarding (1) construct development (2) the appropriate dimensions and attributes of objects, (3) question/scale measurement formats, (4) wording of actual questions and directives, and (5) scale points and descriptors. Following the flowerpot approach assures that the data flow will correctly go from general information level down to a more specific level.

Discuss the importance of cover letters, and explain the guidelines to help eliminate common mistakes in cover letter designs.

While the main role of any cover letter should be that of winning over a prospective respondent, a set of secondary roles ranges from initial introduction with a person to communicating the legitimacy and other important factors about the study. There are 10 critical factors that should be included in most, if not all, cover letters. Including these will help the researcher counteract the three major reasons that prospective respondents use to avoid participating in self-administered survey and personal interviews. A lottery-based incentive or compensation system can significantly improve a prospective respondent's willingness to participate.