Introduction
There are five publications in the Savvy Survey Series that provide an introduction to important aspects of developing items for a questionnaire. This publication provides an overview of constructing open-ended items for a questionnaire. As a reminder, there are also several sources that give in-depth information on developing a questionnaire that may be useful during the construction phase (de Leeuw, Hox, and Dillman, 2008; Dillman, Smyth, and Christian, 2014; Groves et al., 2009).
Remember, there are two major question types that can be used to create a questionnaire. Each collects certain types of information. These question types are:
- open-ended—a blank answer space provided for a description of explanation, a list of items, numbers, or dates; and
- closed-ended—response choices provided (scale, ordered, unordered, or partial).
Though both can be used in a single questionnaire, each type has unique characteristics that may be of use within a survey. This publication will examine the characteristics of the open-ended question.
What are open-ended questions?
Open-ended questions provide the survey participant the opportunity to respond to a question using his or her own words.
An open-ended question might ask the person being surveyed to give a description or explanation of a particular problem they are having. It may ask them to provide a list of items they believe are important for the future of a certain community program. However, open-ended does not necessarily just suggest words; it may also request a number-based answer, such as how old the respondent is or how long he or she has lived in a home. Altogether, researchers have identified five types of open-ended questions (Couper et al., 2010; see Figure 1).
It is interesting to note that words used to phrase the question stem, as well as the size of the space provided for responses, have been found to impact the length of the response that may be generated for description questions (Israel, 2010; Smyth et. al., 2009). Larger spaces tend to generate longer responses than smaller spaces; however, these answers may or may not contain increased substance (Israel, 2010). Therefore, if a longer, more detailed response is preferred, then be sure to provide a space that visually suggests that to the survey taker.
Additionally, survey takers are often unmotivated to respond to questions in a survey, especially descriptive open-ended questions. In order to motivate the survey taker to provide an answer, the survey designer can include additional wording to impress upon the survey taker the importance of his or her response (Smyth et al., 2009). Using phrases such as this question is very important and please take your time answering this question have been found to increase the length of the response (i.e., the number of words) and the time spent answering the question (Smyth et al., 2009; Kumar Chaudhary & Israel, 2016). However, it is best to use this technique only when needed; though everything in the survey is important to the study being conducted, the overuse of these phrases will only reduce their effectiveness.
Guidelines for Creating Open-Ended Questions
Several guidelines for creating open-ended items have been suggested by Dillman, Smyth, and Christian (2014).
- When asking for numerical responses (see Figure 2), be sure to
a. ask for the specific unit desired in the question stem,
b. provide answer spaces that are sized appropriately for the response task, and
c. provide unit labels with the answer spaces.
2. When asking for a list of items (see Figure 3), be sure to
a. specify the number and type of responses desired in the question stem,
b. design the answer spaces to support the type and number of responses desired, and
c. provide labels with answer spaces to reinforce the type of responses requested.
3. When asking for descriptions or explanations (see Figure 4), be sure to
a. provide statements to increase motivation to respond,
b. provide adequate space for respondents to completely answer the question, and
c. use scrollable boxes and consider programming additional questions to probe open-ended responses for online surveys.
Limitations of Open-Ended Questions
Though useful, open-ended questions do have limitations. People taking surveys often skip open-ended questions, especially the descriptive type, because they often require more mental work to answer. If they do respond, it is possible that the respondent will provide a short answer that may or may not provide the depth this type of question hopes to gain. Finally, the data collected through open-ended questions have to be entered and analyzed in a different way than their closed-ended counterparts. This limitation may create a challenge for someone unfamiliar with this type of analysis.
Furthermore, there are limitations even within different audiences and their ability and/or willingness to answer open-ended questions. Audiences with language barriers (e.g., non-English speaking groups) or low educational levels may not complete open-ended questions based on confusion about what is being asked, insecurity in providing an adequate response, or the perceived difficulty in formulating an answer. Additionally, there may be physical issues, such as eye- or hand-coordination problems, that create limitations in ability to respond. Since the goal is to get responses from all groups within the surveyed audience, consider these issues when asking open-ended questions.
In Summary
Developing a high-quality questionnaire is a critical step for collecting useful data for assessing program needs and evaluating the outcomes of programs. The items that are used to collect information must be carefully constructed so that those taking the survey have the ability to answer as easily and accurately as possible. This publication provided an overview of the guidelines for constructing and using open-ended questions in a questionnaire. Additional publications in this series address best practices for constructing closed-ended questions and sets of items for measuring constructs.
References
Couper, M. P., Kennedy, C., Conrad, F. G., and Tourangeau, R. (2011). Designing Input Fields for Non-narrative Open-ended Responses in Web Surveys. Journal of Official Statistics. 27(1), 65–85.
de Leeuw, E. D., Hox, J. J., and Dillman, D. A. (Eds.). (2008). International handbook of survey methodology. New York, NY: Lawrence Erlbaum Associates.
Dillman, D. A., Smyth, J. D., and Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method. (4th ed.). Hoboken, NJ: John Wiley and Sons.
Groves, R. M., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., and Tourangeau, T. (2009). Survey methodology (2nd ed.). Hoboken, NJ: Wiley.
Israel, G. D. (2010). Effects of answer space size on responses to open-ended questions in mail surveys. Journal of Official Statistics, 26(2), 271–285.
Kumar Chaudhary, A., and Israel, G. D. (2016). Influence of importance statements and box size on response rate and response quality of open-ended questions in web/mail mixed-mode surveys. Journal of Rural Social Sciences, 31(3), 140–159.
Smyth, J. D., Dillman, D. A., Christian. L. M., and McBride, M. (2009). Open-ended questions in web surveys: Can increasing the size of answer boxes and providing extra verbal instructions improve response quality? Public Opinion Quarterly, 73(2), 325–337.
