MENU

AskIFAS Powered by EDIS

The Savvy Survey #6b: Constructing Open-Ended Items for a Questionnaire

Jessica L. O'Leary and Glenn D. Israel

Introduction

There are five publications in the Savvy Survey Series that provide an introduction to important aspects of developing items for a questionnaire. This publication provides an overview of constructing open-ended items for a questionnaire. As a reminder, there are also several sources that give in-depth information on developing a questionnaire that may be useful during the construction phase (de Leeuw, Hox, and Dillman, 2008; Dillman, Smyth, and Christian, 2014; Groves et al., 2009).

Remember, there are two major question types that can be used to create a questionnaire. Each collects certain types of information. These question types are:

  • open-ended—a blank answer space provided for a description of explanation, a list of items, numbers, or dates; and
  • closed-ended—response choices provided (scale, ordered, unordered, or partial).

Though both can be used in a single questionnaire, each type has unique characteristics that may be of use within a survey. This publication will examine the characteristics of the open-ended question.

What are open-ended questions?

Open-ended questions provide the survey participant the opportunity to respond to a question using his or her own words.

An open-ended question might ask the person being surveyed to give a description or explanation of a particular problem they are having. It may ask them to provide a list of items they believe are important for the future of a certain community program. However, open-ended does not necessarily just suggest words; it may also request a number-based answer, such as how old the respondent is or how long he or she has lived in a home. Altogether, researchers have identified five types of open-ended questions (Couper et al., 2010; see Figure 1).

Figure 1. Examples of open-ended questions.
Figure 1.  Examples of open-ended questions.

It is interesting to note that words used to phrase the question stem, as well as the size of the space provided for responses, have been found to impact the length of the response that may be generated for description questions (Israel, 2010; Smyth et. al., 2009). Larger spaces tend to generate longer responses than smaller spaces; however, these answers may or may not contain increased substance (Israel, 2010). Therefore, if a longer, more detailed response is preferred, then be sure to provide a space that visually suggests that to the survey taker.

Additionally, survey takers are often unmotivated to respond to questions in a survey, especially descriptive open-ended questions. In order to motivate the survey taker to provide an answer, the survey designer can include additional wording to impress upon the survey taker the importance of his or her response (Smyth et al., 2009). Using phrases such as this question is very important and please take your time answering this question have been found to increase the length of the response (i.e., the number of words) and the time spent answering the question (Smyth et al., 2009; Kumar Chaudhary & Israel, 2016). However, it is best to use this technique only when needed; though everything in the survey is important to the study being conducted, the overuse of these phrases will only reduce their effectiveness.

Guidelines for Creating Open-Ended Questions

Several guidelines for creating open-ended items have been suggested by Dillman, Smyth, and Christian (2014).

  1. When asking for numerical responses (see Figure 2), be sure to

a. ask for the specific unit desired in the question stem,

b. provide answer spaces that are sized appropriately for the response task, and

c. provide unit labels with the answer spaces.

2. When asking for a list of items (see Figure 3), be sure to

a. specify the number and type of responses desired in the question stem,

b. design the answer spaces to support the type and number of responses desired, and

c. provide labels with answer spaces to reinforce the type of responses requested.

3. When asking for descriptions or explanations (see Figure 4), be sure to

a. provide statements to increase motivation to respond,

b. provide adequate space for respondents to completely answer the question, and

c. use scrollable boxes and consider programming additional questions to probe open-ended responses for online surveys.

Figure 2. Poor and better examples of a numerical open-ended question
Figure 2.  Poor and better examples of a numerical open-ended question

 

Figure 3. Poor and better examples of a list open-ended question
Figure 3.  Poor and better examples of a list open-ended question

 

Figure 4. Poor and better examples of a description open-ended question
Figure 4.  Poor and better examples of a description open-ended question

 

Limitations of Open-Ended Questions

Though useful, open-ended questions do have limitations. People taking surveys often skip open-ended questions, especially the descriptive type, because they often require more mental work to answer. If they do respond, it is possible that the respondent will provide a short answer that may or may not provide the depth this type of question hopes to gain. Finally, the data collected through open-ended questions have to be entered and analyzed in a different way than their closed-ended counterparts. This limitation may create a challenge for someone unfamiliar with this type of analysis.

Furthermore, there are limitations even within different audiences and their ability and/or willingness to answer open-ended questions. Audiences with language barriers (e.g., non-English speaking groups) or low educational levels may not complete open-ended questions based on confusion about what is being asked, insecurity in providing an adequate response, or the perceived difficulty in formulating an answer. Additionally, there may be physical issues, such as eye- or hand-coordination problems, that create limitations in ability to respond. Since the goal is to get responses from all groups within the surveyed audience, consider these issues when asking open-ended questions.

In Summary

Developing a high-quality questionnaire is a critical step for collecting useful data for assessing program needs and evaluating the outcomes of programs. The items that are used to collect information must be carefully constructed so that those taking the survey have the ability to answer as easily and accurately as possible. This publication provided an overview of the guidelines for constructing and using open-ended questions in a questionnaire. Additional publications in this series address best practices for constructing closed-ended questions and sets of items for measuring constructs.

References

Couper, M. P., Kennedy, C., Conrad, F. G., and Tourangeau, R. (2011). Designing Input Fields for Non-narrative Open-ended Responses in Web Surveys. Journal of Official Statistics. 27(1), 65–85.

de Leeuw, E. D., Hox, J. J., and Dillman, D. A. (Eds.). (2008). International handbook of survey methodology. New York, NY: Lawrence Erlbaum Associates.

Dillman, D. A., Smyth, J. D., and Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method. (4th ed.). Hoboken, NJ: John Wiley and Sons.

Groves, R. M., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., and Tourangeau, T. (2009). Survey methodology (2nd ed.). Hoboken, NJ: Wiley.

Israel, G. D. (2010). Effects of answer space size on responses to open-ended questions in mail surveys. Journal of Official Statistics, 26(2), 271–285.

Kumar Chaudhary, A., and Israel, G. D. (2016). Influence of importance statements and box size on response rate and response quality of open-ended questions in web/mail mixed-mode surveys. Journal of Rural Social Sciences, 31(3), 140–159.

Smyth, J. D., Dillman, D. A., Christian. L. M., and McBride, M. (2009). Open-ended questions in web surveys: Can increasing the size of answer boxes and providing extra verbal instructions improve response quality? Public Opinion Quarterly, 73(2), 325–337.

Publication #AEC397

Release Date:October 29, 2020

Reviewed At:December 11, 2023

Related Collections

Fact Sheet

About this Publication

This document is AEC397, one of a series of the Department of Agricultural Education and Communication, UF/IFAS Extension. Original publication date April 2014. Revised August 2017. Visit the EDIS website at https://edis.ifas.ufl.edu for the currently supported version of this publication.

About the Authors

Jessica L. O'Leary, former doctoral candidate; and Glenn D. Israel, professor emeritus, Department of Agricultural Education and Communication; UF/IFAS Extension, Gainesville, FL 32611. The authors wish to thank Cheri Brodeur, Laura Sanagorski, Marilyn Smith, and Pam Sigler for their helpful suggestions on an earlier draft.

Contacts

  • Glenn Israel