University of FloridaSolutions for Your Life

Download PDF
Publication #AEC397

The Savvy Survey #6b: Constructing Open-Ended Items for a Questionnaire1

Jessica L. Gouldthorpe and Glenn D. Israel2

Introduction

There are five publications in the Savvy Survey Series that provide an introduction to important aspects of developing items for a questionnaire. This publication provides an overview of constructing open-ended items for a questionnaire. As a reminder, there are also several sources that give in-depth information on developing a questionnaire that may be useful during the construction phase (de Leeuw, Hox, and Dillman 2008; Dillman, Smyth, and Christian 2009; Groves et al. 2009).

Remember, there are two major question types that can be used to create a questionnaire. Each collects certain types of information. These question types are

    • open-ended—a blank answer space provided for a description of explanation, a list of items, numbers, or dates; and

    • closed-ended—response choices provided (scale, ordered, unordered, or partial).

Though both can be used in a single questionnaire, each type has unique characteristics that may be of use within a survey. This publication will examine the characteristics of the open-ended question.

What Are Open-Ended Questions?

Open-ended questions provide the survey participant the opportunity to respond to a question using his or her own words.

An open-ended question might ask the person being surveyed to give a description or explanation of a particular problem they are having. It may ask them to provide a list of items they believe are important for the future of a certain community program. However, open-ended does not necessarily just suggest words; it may also request a number-based answer, such as how old the respondent is or how long he or she has lived in a home. Altogether, researchers have identified five types of open-ended questions (Couper et al. 2010; see Figure 1).

Figure 1. 

Examples of open-ended questions.


[Click thumbnail to enlarge.]

It is interesting to note that words used to phrase the question stem, as well as the size of the space provided for responses, have been found to impact the length of the response that may be generated for description questions (Israel 2010; Smyth et. al. 2009). Larger spaces tend to generate longer responses than smaller spaces; however, these answers may or may not contain increased substance (Israel 2010). Therefore, if a longer, more detailed response is preferred, then be sure to provide a space that visually suggests that to the survey taker.

Additionally, often survey takers are unmotivated to respond to questions in a survey, especially descriptive open-ended questions. In order to motivate the survey taker to provide an answer, the survey designer can include additional wording to impress upon the survey taker the importance of his or her response (Smyth et al. 2009). Using phrases such as this question is very important and please take your time answering this question have been found to increase the length of the response (i.e., the number of words) and the time spent answering the question (Smyth et al. 2009). However, it is best to use this technique only when needed; though everything in the survey is important to the study being conducted, the overuse of these phrases will only reduce their effectiveness.

Guidelines for Creating Open-Ended Questions

Several guidelines for creating open-ended items have been suggested by Dillman, Smyth, and Christian (2009).

1. When asking for numerical responses (see Figure 2), be sure to

a. ask for the specific unit desired in the question stem,

b. provide answer spaces that are sized appropriately for the response task, and

c. provide unit labels with the answer spaces.

2. When asking for a list of items (see Figure 3), be sure to

a. specify the number and type of responses desired in the question stem,

b. design the answer spaces to support the type and number of responses desired, and

c. provide labels with answer spaces to reinforce the type of responses requested.

3. When asking for descriptions or explanations (see Figure 4), be sure to

a. provide statements to increase motivation to respond,

b. provide adequate space for respondents to completely answer the question, and

c. Use scrollable boxes and consider programing additional questions to probe open-ended responses for online surveys.

Figure 2. 

Poor and better examples of a numerical open-ended question


[Click thumbnail to enlarge.]

Figure 3. 

Poor and better examples of a list open-ended question


[Click thumbnail to enlarge.]

Figure 4. 

Poor and better examples of a description open-ended question


[Click thumbnail to enlarge.]

Limitations of Open-Ended Questions

Though useful, open-ended questions do have limitations. People taking surveys often skip open-ended questions, especially the descriptive type, because they often require more mental work to answer. If they do respond, it is possible that the respondent will provide a short answer that may or may not provide the depth this type of question hopes to gain. Finally, the data collected through open-ended questions have to be entered and analyzed in a different way than their closed-ended counterparts. This limitation may create a challenge for someone unfamiliar with this type of analysis.

Furthermore, there are limitations even within different audiences and their ability and/or willingness to answer open-ended questions. Audiences with language barriers (e.g., non-English speaking groups) or low educational levels may not complete open-ended based on confusion about what is being asked, insecurity in providing an adequate response, or the perceived difficulty in formulating an answer. Additionally, there may be physical issues, such as eye- or hand-coordination problems, that create limitations in ability to respond. Since the goal is to get responses from all groups within the surveyed audience, consider these issues when asking open-ended questions.

In Summary

Developing a high-quality questionnaire is a critical step for collecting useful data for assessing program needs and evaluating the outcomes of programs. The items that are used to collect information must be carefully constructed so that those taking the survey have the ability to answer as easily and accurately as possible. This publication provided an overview of the guidelines for constructing and using open-ended questions in a questionnaire. Additional publications in this series address best practices for constructing closed-ended questions and sets of items for measuring constructs.

References

Couper, M. P., Kennedy, C., Conrad, F. G., and Tourangeau, R. (2011) “Designing Input Fields for Non-narrative Open-ended Responses in Web Surveys,” Journal of Official Statistics, 27, 65-85.

de Leeuw, E. D., Hox, J. J., and Dillman, D. A., (Eds.). (2008). International handbook of survey methodology. New York, NY: Lawrence Erlbaum Associates.

Dillman, D. A., Smyth, J. D., and Christian, L. M. (2009). Internet, mail, and mixed-mode surveys: The tailored design method. (3rd ed.). Hoboken, NJ: John Wiley and Sons.

Groves, R. M., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., and Tourangeau, T. (2009), Survey methodology (2nd ed.). Hoboken, NJ: Wiley.

Israel, G. D. (2010). Effects of answer space size on responses to open-ended questions in mail surveys. Journal of Official Statistics, 26(2), 271-285.

Smyth, J. D., Dillman, D. A., Christian. L. M., and McBride, M. (2009). Open-ended questions in web surveys: Can increasing the size of answer boxes and providing extra verbal instructions improve response quality? Public Opinion Quarterly, 73(2), 325-337.

Footnotes

1.

This document is AEC397, one of a series of the Agricultural Education and Communication Department, UF/IFAS Extension. Original publication date April 2014. Visit the EDIS website at http://edis.ifas.ufl.edu.

2.

Jessica L. Gouldthorpe, doctoral candidate; and Glenn D. Israel, professor, Agricultural Education and Communication Department; UF/IFAS Extension, Gainesville, FL 32611. The authors wish to thank Cheri Brodeur, Laura Sanagorski, Marilyn Smith, and Pam Sigler for their helpful suggestions on an earlier draft.


The Institute of Food and Agricultural Sciences (IFAS) is an Equal Opportunity Institution authorized to provide research, educational information and other services only to individuals and institutions that function with non-discrimination with respect to race, creed, color, religion, age, disability, sex, sexual orientation, marital status, national origin, political opinions or affiliations. For more information on obtaining other UF/IFAS Extension publications, contact your county's UF/IFAS Extension office.

U.S. Department of Agriculture, UF/IFAS Extension Service, University of Florida, IFAS, Florida A & M University Cooperative Extension Program, and Boards of County Commissioners Cooperating. Nick T. Place, dean for UF/IFAS Extension.