University of FloridaSolutions for Your Life

Download PDF
Publication #FCS6010

Using Qualitative Research in Planning and Evaluating Extension Programs1

Lisa A. Guion, Lamont Flowers, David C. Diehl, and Debra McDonald2

This paper provides a brief overview of qualitative research and discusses the ways in which it can be used in planning and evaluating extension programs. While quantitative information is highly valued in extension, qualitative information can help faculty gain a rich understanding of the issues they are addressing. Qualitative research is a useful tool for understanding extension programs and can complement quantitative methods.

What is qualitative research?

Qualitative research is a method of research that yields nonnumeric information (e.g., words, text) generated by examining phenomenon that are not easily translated into numbers and/or are not quantifiable (Schwandt, 1997). Qualitative data are left in the narrative state in order to glean a deeper, fuller understanding of the phenomenon under study (Miller and Dingwall, 1997). Also, most qualitative research involves direct, first-hand study of naturally occurring situations or events in their natural settings (Schwandt, 1997).

Berg (2007) describes several ways to collect qualitative data, including interviews, focus groups, ethnography, observation, and case studies.

Interviews: “Interviewing may be defined simply as conversation with a purpose” (Berg, 2007, p. 89). The three types of interviews commonly used in qualitative research are (1) structured, in which a specific set of questions is asked, (2) semi-structured, in which a specific set of questions is asked, but other, unplanned questions are also asked, and (3) unstructured, in which the natural flow of conversation determines the questions asked.

Figure 1. 

Conversation with a purpose.


Credit:

UF/IFAS Photo by Tyler Jones.


[Click thumbnail to enlarge.]

Focus groups: Focus group interviews are conducted with small groups of people on a particular topic and are facilitated by a moderator. They are designed to collect information in a format that allows for discussion and group interaction.

Ethnography: “Ethnography is the art and science of describing a group or culture” (Berg, 2007, p. 1). Data are gathered through an array of means—direct observation, interviews, surveys, and questionnaires—with the final goal being a more personal, and therefore, more valid, experience of that culture.

Observation: Direct observation is a non-intrusive research tool that is used to evaluate participants as they go about their daily affairs. Observation typically takes place in naturalistic settings and relies on the researchers’ abilities to be unobtrusive and to observe the behaviors or practices of interest.

Case studies: Case studies are a method for “systematically gathering enough information about a particular person, social setting, event, or group to permit the researcher to effectively understand how the subject operates or functions” (Berg, 2007, p. 283). The goal is to understand specific instances in great detail such that the information might be used to generalize to other situations.

Qualitative Research for Program Planning and Evaluation

Qualitative research is a tool for program planning and evaluation that is frequently focused on understanding the stories behind particular individuals or groups. Qualitative methods are especially well suited for asking open-ended questions that can inform program delivery or explain evaluation findings. In extension, qualitative approaches may be appropriate in several situations, including the following:

  • Conducting interviews or focus group(s) to determine program needs or to better understand program needs that have been identified through a survey

  • Conducting structured observations of practitioners (e.g., farmers, parents, etc.) to understand their current practices and behaviors

  • Conducting exit interviews of participants to explore what program elements are working and not working and why

  • Conducting interviews to elicit success stories from participants

  • Using focus groups with program participants to help interpret findings from a program evaluation

For program planning purposes, qualitative data can provide a deeper understanding of community needs and issues. Dialogues with potential participants and other informants can be used to inform key issues such as: high-priority community issues, specific target audiences, audience interests, and preferred educational methods. When conducting needs assessments, qualitative interviews or focus groups can be used alone or in combination with quantitative methods and survey data. Examples of qualitative approaches to planning include the following:

  1. Auburn University conducted open-ended interviews to identify sources of resistance to organizational change. Themes from the interviews were organized into categories, which were then used to create survey items that could be quantified. The qualitative and quantitative data were the foundation for a systematic process addressing extension faculty’s concerns, engaging stakeholders in the process, and creating a plan for implementing recommendations (Washington & Fowler, 2005).

  2. Oregon State University Extension utilizes qualitative data collection to inform a “program snapshot process,” designed to be a quicker approach to strategic planning. The process takes about 6-8 weeks and is focused on midcourse corrections for extension programs (Simon-Brown, 2008).

  3. Pennsylvania State University Extension conducted qualitative research to inform its efforts on environmental policy education. The researchers conducted key informant interviews focused on policy challenges, performance indicators, and future directions. The report generated was shared with numerous stakeholders and was used to guide future work on environmental issues (Dodd & Abdalla, 2004).

For program evaluation purposes, qualitative methods can be used to: develop program outcomes when necessary, elicit qualitative findings and success stories, and clarify evaluation findings. Qualitative research can be used to provide deeper understanding of contextual variables and explanations of how and why program outcomes are achieved (Bamberger, 2000). Examples include the following:

  1. Iowa State University Extension conducted a qualitative evaluation of the Expanded Food and Nutrition Education Program (EFNEP). The researchers used focus groups to identify participants’ perceptions of the overall program, barriers to participation, the usefulness of incentives, and the value of experiential activities. Structured observations were used to assess how well strategies were being implemented by participants (Hoover, Martin, & Litchfield, 2009).

  2. Researchers from Ohio State University Extension used qualitative methods to identify motivational factors and retention characteristics for 4-H volunteers. Because they did not know the possible factors prior to conducting their research, they used open-ended questions to generate responses and then categorized these responses (Culp & Pilat, 1998).

  3. Rutgers University Extension evaluated its Individual Development Account (IDA) Financial Education by analyzing notes taken during sharing sessions and anecdotes written by participants. Three researchers coded the data, coming to consistency on major themes related to the qualitative impact of the IDA program (O’Neill, 2006).

Summary

The strengths of qualitative research include the:

  • ability to gain a deeper understanding of what is being evaluated;

  • perception that the data collection is more user-friendly and personal; and

  • flexibility in evaluation design and implementation (Bamberger, 2000).

However, qualitative research also has weaknesses that should be considered, including the:

  • challenge of generalizing findings to the larger population, because typically very few individuals are studied;

  • inability to arrive at statistics and precise computation; and

  • potentially more time-consuming nature of inquiry (Bamberger, 2000).

As with any method, researchers should seek to maximize the strengths and minimize the weaknesses. In short, qualitative research can provide rich information that can be used to not only show what worked and what did not, but also why it did or did not work. Qualitative data can contribute to a deeper and more meaningful program planning and evaluation process.

Further Reading

Creswell, J.W. (2006). Qualitative Inquiry and Research Design: Choosing Among Five Approaches. Thousand Oaks, CA: Sage Publications.

Gillam, A. (n.d.). Evaluation of HIV prevention programs using qualitative methods. Centers for Disease Control and Prevention. Available at http://www.cdc.gov/HealthyYouth/publications/hiv_handbook/pdfs/hiv_booklet09.pdf.

Maxwell, J.A. (2004). Qualitative Research Design: An Interactive Approach. Thousand Oaks, CA: Sage Publications.

Patton, M.Q. (2001). Qualitative Research and Evaluation Methods. Thousand Oaks, CA: Sage Publications.

Summers, S.J. (2009). Using qualitative data in program evaluation: Telling the story of a prevention program. Available at http://friendsnrc.org/direct-download-menuitem/doc_download/160-using-qualitative-data-in-program-evaluation-telling-the-story-of-a-prevention-program.

References

Bamberger, M. (Ed.). (2000). Integrating Quantitative and Qualitative Research in Development Projects. Washington, DC: World Bank.

Berg, B.L. (2007). Qualitative Research Methods for the Social Sciences (4th Ed.). Boston, MA: Allyn & Bacon.

Culp, K., & Pilat, M. (1998). Converting qualitative feedback into quantifiable categories. Journal of Extension, 36(5). http://www.joe.org/joe/1998october/iw3.php.

Dodd, A., & Abdalla, C. (2004). Strengthening environmental policy education through qualitative research: Experience with Pennsylvania’s Nutrient Management Act regulatory review. Journal of Extension, 42(5). http://www.joe.org/joe/2004october/a2.php

Fetterman, D. M. (2009). Ethnography: Step-by-Step. Thousand Oaks, CA: Sage Publications.

Hoover, J., Martin, P., & Litchfield, R. (2009). Qualitative tools to examine EFNEP curriculum delivery. Journal of Extension, 47(3). http://www.joe.org/joe/2009june/a3.php.

Miller, G. & Dingwall, R. (1997). Context & Method in Qualitative Research. London: Sage Publications.

O’Neill, B. (2002). IDA financial education: Qualitative impacts. Journal of Extension, 44(6). http://www.joe.org/joe/2006december/rb7.php.

Schwandt, T.A. (1997). Qualitative Inquiry. Thousand Oaks, CA: Sage Publications.

Simon-Brown, V. (2008). Taking program snapshots: Planning strategies when there isn’t time for strategic planning. Journal of Extension, 46(4). http://www.joe.org/joe/2008august/tt1.php.

Washington, R., & Fowler, S. (2005). Systematic assessment of resistance to Extension organizational change: Evidence from the Alabama Cooperative Extension System.Journal of Extension, 43(2). http://www.joe.org/joe/2005april/rb2.php

Footnotes

1.

This document is FCS6010, one of a series of the Department of Family, Youth and Community Sciences, Florida Cooperative Extension Service, Institute of Food and Agricultural Sciences, University of Florida. Original publication date August 2001. Revised August 2005, September 2011. Original written by Lisa A. Guion, former faculty member, and Lamont Flowers, former faculty member; revised by David C. Diehl and Debra McDonald. Visit the EDIS website at http://edis.ifas.ufl.edu.

2.

Lisa A Guion, former faculty member; Lamont Flowers, former faculty member; David C. Diehl, assistant professor; Debra McDonald, project coordinator; Department of Family, Youth and Community Sciences; Florida Cooperative Extension Service; Institute of Food and Agricultural Sciences; University of Florida; Gainesville, FL 32611.


The Institute of Food and Agricultural Sciences (IFAS) is an Equal Opportunity Institution authorized to provide research, educational information and other services only to individuals and institutions that function with non-discrimination with respect to race, creed, color, religion, age, disability, sex, sexual orientation, marital status, national origin, political opinions or affiliations. For more information on obtaining other UF/IFAS Extension publications, contact your county's UF/IFAS Extension office.

U.S. Department of Agriculture, UF/IFAS Extension Service, University of Florida, IFAS, Florida A & M University Cooperative Extension Program, and Boards of County Commissioners Cooperating. Nick T. Place, dean for UF/IFAS Extension.