The Savvy Survey #10: In-Person-Administered Surveys

Introduction

This publication provides an overview of in-person-administered surveys as part of the Savvy Survey Series. In-person-administered surveys are a type of face-to-face interview that collects mainly quantitative data from a number of individuals. In-person-administered surveys are appropriate for a number of situations and purposes, including collecting information from low-literacy audiences or obtaining very detailed information from agricultural producers or business owners/managers about needs or program outcomes (for example, see Farnsworth et al., 2019; Monaghan et al., 2011). These surveys can also be used to "intercept" individuals at locations such as marinas, state or national parks, and farmers' markets because lists of users from which to draw a sample are usually unavailable.

This publication presents the construction, development, and implementation of in-person survey instruments. It also includes how to prepare an introductory script, train interviewers, and manage the survey process.

Developing the In-Person-Administered Questionnaire

The construction of every survey questionnaire requires careful attention to detail. However, for questionnaires that will be administered by an interviewer, special attention must be given to account for possible bias from the interviewer, who might ask questions that result in answers that are different in some way from when people answer the questionnaire on their own. Since there is no interviewer present for mail and online surveys, visual modes alone are used to present questions and display response options. In contrast, in-person-administered surveys rely on an interviewer to ask the questions and to either state the options for answering or provide either show cards printed with the response options or a tablet device to display the response options and record the answer. More importantly, the interviewer can observe respondent reactions and facilitate the data collection process. However, the presence of an interviewer opens the door for him or her to knowingly (or unknowingly) interject bias through behavior, word choices, or tone (Loosveldt, 2008).

A well-developed questionnaire is the foundation for collecting useful data from interviews. It is common to structure survey questions to include the responses in the stem of the question (see Table 1) to take advantage of the interviewer-respondent interaction. Notice that the list of response options and associated code numbers (for interviewer use only) includes the categories "Don't Know" and "Refused" to cover those situations even though the interviewer does not read those categories to the respondent. In contrast, the mail version includes instructions for marking an answer and omits response options for "Don't know" and "Refused." For more information about creating specific questions, see Savvy Survey Series publications #6a-6c.

The questionnaire can be created in a paper format. This format allows interviewers to mark or write participant responses, as well as any notes regarding the exchange, as they move through the question series. The questionnaire can also be created using survey software, such as Qualtrics. Using an electronic form, the interviewer enters the data into an offline version of the survey using a laptop computer or a tablet device (e.g., an iPad) while talking to the respondent and, when a connection to the Internet is available, the data are uploaded to the online survey database. If an electronic form is used, the questions should be formatted for oral administration so that interviewers ask the questions with the response options in the stem. Alternatively, the interviewer might hand a tablet device to the respondent to complete the survey while the interviewer facilitates the process or troubleshoots problems.

The Process for In-Person-Administered Surveys

In addition to developing the questionnaire, the survey process includes several important activities. These are:

• Preparing a script for making appointments for surveying individuals and associated materials
• Recruiting and training interviewers
• Scheduling and conducting the survey

Preparing the Introductory Script and Support Materials

In addition to tailoring the questionnaire for interviewer administration, an introductory script and other support materials must be prepared before implementing the survey. The introductory script is used by the interviewer to schedule the date, time, and location for the interview (See Table 2). Paul Lavrakas (2010) notes that interviewers have about 20–30 seconds to obtain a respondent's agreement to participate; this means that scripts should be as brief as possible while also communicating critical information about the purpose and importance of the survey.

In addition to drafting the introductory script, a call sheet (or its electronic equivalent) needs to be prepared. The call sheet is used to keep track of the date and time that a given phone number is called to schedule an interview. It is also used to record what happened (referred to as the disposition) during each instance that the number was dialed. Table 3 shows an example of a call sheet used for a customer satisfaction survey. Note that the call sheet includes the name and phone number, as well as space for recording notes about the dialing process. Alternatively, email messages might be used as a substitute for the phone is scheduling interviews and a spreadsheet could be used for keeping track of these messages.

A third item to prepare is the Interviewee Information Sheet. The information sheet explains the purpose of the interview, how it will be conducted, what will be done with the collected data, the rights of the interviewee, and the contact information of the survey director. If the survey protocol has been reviewed and approved by the Institutional Review Board (IRB, see Savvy Survey Series publication #9), it may include additional information. Note that surveys that will be used for research and result in conference presentations or journal articles are generally required to have an IRB review. A sample information sheet is included in Appendix A.

If other people will be helping to conduct the survey, then an interviewer orientation guide should be created. The guide should fully explain the purpose of the survey, its procedures, and the expectations for interviewer conduct. It might also include scenarios for interviewers to review and practice, the set of fallback statements, and a copy of the questionnaire. Fallback statements are examples of ways that interviewers can answer a question or address a concern or problem. For example, if a respondent expresses concern about how his or her answers will be used, an interviewer might say, "Your answers will be combined with those of other people to provide a summary of what people think about Extension's services. Only grouped responses will be used in reports or public presentations." Additional examples of fallback statements are included in Appendix B.

Finally, equipment needed to conduct the interviews must be acquired. If a paper questionnaire is being used "out in the field," then each interviewer will need a clipboard, as well as pens/pencils, copies of the questionnaire and the interviewee information sheet, a notebook for record keeping, and an official ID or name tag that identifies the person and organization (Vaske, 2008).

Recruiting and Training Interviewers

Depending on the scope of the survey, a number of interviewers may be needed. The number needed will depend, in part, on the desired number of interviews to be completed and the timeline for conducting the survey. Small-scale surveys conducted over one or two months can be completed with just 1 to 2 interviewers while large-scale surveys might need 10 to 20 interviewers. For Extension surveys, office staff and volunteers may be suitable to fill interviewer roles. People who might have a conflict of interest with the survey results should avoid being an interviewer. For example, an Extension agent who conducted an educational program should not interview the attendees for an evaluation of outcomes for that program because many people are likely to follow social norms and give a more positive assessment rather than share their true assessment of the program.

It is important for interviewers to understand that the in-person-administered survey represents a "structured conversation" with a respondent. There are both similarities and differences between a structured conversation and an everyday conversation and between in-depth interviews used for qualitative studies.

When the interview guide is prepared and interviewers have been recruited, a training session should be scheduled. The session should include a discussion of the purpose of the survey and the expectations for interviewers (see Appendix C for suggestions about expected roles), as well as a practice interview and a review of survey scenarios and fallback statements (Israel, 2000). Spending time with interviewers to build a common understanding of the survey procedures and to develop skills in probing and giving feedback is critical to conducting a quality survey and obtaining useful data.

Scheduling and Conducting Interviews

To properly implement an interviewer-administered survey, a supervisor or survey manager must complete the following steps while paying careful attention to details.

Step 1. Use an advance contact, if appropriate. When the sample list contains both telephone numbers and postal/email addresses, an advance letter/email message can be sent to alert potential respondents about the survey or a call can be made to schedule an appointment. An advance letter can reduce the number of refusals by introducing the survey and establishing its legitimacy.

In some situations, the individuals who are targeted for interviews are unknown (e.g., shoppers at a farmers' market or clients at a food pantry). It is helpful, however, to notify and gain the cooperation of organizational leaders, such as the manager of the farmers' market or the food pantry, respectively. Procedures for selecting specific individuals for the interviews may need to be used, such as systematically selecting the n^th person arriving at the interview location.

Step 2. Initiate the interview. The interview should begin with the interviewer introducing him- or herself and reiterating the purpose of the survey. It is important to establish the interviewer's legitimacy by connecting the survey with the university and/or county Extension office. In addition, reviewing the purpose of the survey helps to establish its importance in the mind of the respondent. If an incentive is being offered to build trust, it will be most effective if offered when the interviewer introduces him- or herself and if given without requiring a completed interview (i.e., a non-contingent incentive).

Step 3. Provide instructions to help the respondent understand the task. Do not assume that the respondent understands the necessary type and specificity of information (Loosvelt, 2008). Include some general instructions at the beginning of the interview to enhance clarity about how to respond; this will facilitate the question and answer process. For example, these general instructions might include:

We ask everyone who is participating in the survey to think carefully about each question and search their memory so that we get accurate and complete answers. Please give as much detail as you can because we have found that this helps us to better understand your situation. Also, when a question is not clear, I can read it again.

In addition to general instructions at the beginning of the interview, specific instructions might be needed for some questions. For example, a survey of homeowners' landscaping practices might ask "What fertilizer did you apply to your lawn?" and include the instruction "Please be as specific as possible and tell us the N-P-K formulation, if you know it."

Because the interviewer's ability to motivate respondents is a major advantage of this survey mode, phrases to reinforce and provide feedback on a respondent's performance should be used. Reinforcing phrases include "Okay" and "Thank you." Positive feedback can also be provided when the respondent gives answers that are specific. For example, when a client says, "I ate two servings of fruit on each of the last three days," a Family Nutrition Program agent might respond, "That's exactly what I need to know." Clarifying feedback is required when a respondent's answer is incomplete or does not use the prescribed answer categories (e.g., "You said that you ate fruit during the last three days. Can you tell me how many servings each day?").

Step 4. Collect the information. Because the in-person-administered survey is designed to collect primarily quantitative data (in contrast to in-depth interviews that focus on qualitative analysis), the questions must be read as worded and asked in the prescribed order. When interviewers omit words or make subtle changes in the wording, this can affect how respondents answer, increasing measurement error (Schuman and Presser, 1981).

When a respondent asks for clarification, the question should be restated using the same words. When interviewers try to clarify the question on the fly, these ad hoc explanations can change the meaning of the question. This problem can be avoided by careful development of the questionnaire and pretesting the instrument.

Although clarifying questions should be avoided, it is acceptable for interviewers to ask probing questions. This is done more often for open-ended questions than for close-ended questions. Typical probing questions include, "I'm not sure that I understand. What do you mean exactly?" and "Can you give me more details on that?" (Loosveldt, 2008).

Finally, the interviewer's responsibilities are not limited to asking questions. Recording answers accurately and completely is equally as important as asking the questions correctly. When using a paper questionnaire, the answers must be legible so that the information can be transferred to an electronic format. In addition, care must be used to ensure the correct response is checked on the form or the appropriate format is used for recording numbers or dates.

Step 5. Close the interview. When the last question has been asked and answered, the interviewer should thank the respondent for his or her help.

Step 6. Enter the data. When the interview has resulted in a completed questionnaire, the survey manager arranges for the data entry process to begin (see Savvy Survey Series publication #15). Alternatively, interviewers may use Qualtrics survey software to enter the data as the interview is being conducted.

In Summary

This publication has provided a brief overview of how to develop and conduct an interviewer-administered survey. When volunteers or staff assist with the survey by interviewing respondents, this data collection method can be economical and effective. Careful attention is needed for developing the questionnaire and supporting materials, as well as for training interviewers in order to obtain credible survey data. In the right situation, these surveys can be a valuable tool for Extension agents and specialists.

References

Farnsworth, D., Clark, J. L., & Cothran, H. M. (2019). Business Retention and Expansion (BRE) Programs: Developing a Business Retention and Expansion Survey. FE654. Gainesville: University of Florida Institute of Food and Agricultural Sciences. https://edis.ifas.ufl.edu/publication/FE654.

Dillman, D. A. (2007). Mail and Internet Surveys: The Tailored Design Method. 2007 Update with New Internet, Visual, and Mixed-Mode Guide. New York: Wiley.

Israel, G. D. (2000). Conducting a Customer Satisfaction Survey. AEC356. Gainesville: University of Florida Institute of Food and Agricultural Sciences. https://ufdcimages.uflib.ufl.edu/IR/00/00/24/74/00001/WC03900.pdf

Lavrakas, P. J. (1993). Telephone Survey Methods: Sampling, Selection, and Supervision, 2nd ed. Thousand Oaks, CA: Sage Publications.

Loosveldt, G. (2008). "Face-to-face interviews." In International Handbook of Survey Methodology edited by D. de Leeuw, J. J. Hox, and D. D. Dillman, 201-220. New York, NY: Lawrence Erlbaum Associates.

Monaghan, P. F., S.L. Forst, J.A. Tovar-Aguilar, C.A. Bryant, G.D. Israel, S. Galindo-Gonzalez, Z. Thompson, Y. Zhu, & R.J. McDermott. (2011). "Preventing eye injuries among citrus harvesters: The community health worker model." American Journal of Public Health, 101(12), 2269-2274. doi:10.2105/AJPH.2011.300316.

Schulman, H. & S. Presser. (1981). Questions and Answers in Attitude Surveys: Experiments in Question Form, Wording, and Context. New York: Academic Press.

Vaske, J. J. (2008). Survey Research and Analysis: Applications in Parks, Recreation and Human Dimensions. State College, PA: Venture Publishing, Inc.

Appendix A: Sample Information Sheet

Appendix B: Sample Fallback Statements

IF IN DOUBT

You were chosen as part of a random sample of people in order to obtain their opinions. It is very important to learn how people feel about our services and the usefulness of the information that we provide.

I DON'T HAVE TIME FOR THIS

I understand that you have a busy schedule but it is very important that we speak to busy people like yourself in order to get an accurate cross-section of people. We can do the interview at your convenience. Is there a time that would be best for you in the next week?

HOW CAN I BE SURE THAT THIS IS LEGITIMATE?

I would be glad to give you the telephone number of the faculty member in charge of the survey, who will provide you with more information. His/her name is______________. I am sure (s)he would be happy to talk with you. The number to call him(her) at is _____-_______.

RESPONDENT WANTS TO BREAK OFF THE INTERVIEW

We have only a few more questions, if you will just bear with me.

TOO BUSY

This should take only a few minutes. Sorry to have caught you at a bad time, I would be happy to come back. When would be a good time in the next day or two?

BAD HEALTH

I am sorry to hear that. I would be happy to come back in a day or two. Would that be okay?

TOO OLD

Older people's opinions are just as important in this particular survey as anyone else's. In order for the results to be representative of all the people that we serve, we have to be sure that older people have as much chance to give their opinion as anyone else does. We really do want your opinion.

DON'T KNOW ENOUGH TO ANSWER

The questions are not all difficult. Some of the people we have already interviewed have had the same concern you have, but once we got started, they did not have any difficulty answering the questions. Maybe I could read just a question or two to you and you can see what they are like.

NOT INTERESTED

It is awfully important that we get the opinions of everyone in the sample, otherwise the results will not be very useful. So, I would really like to talk to you.

Appendix C: Interviewer Expectations

When you agree to be an interviewer, you must also agree to follow some simple but important guidelines. The first and most important is to honor confidentiality. In order for surveys to be successful, respondents must be assured that their answers will be kept confidential. However, in a small community it may be possible that you know the person you are contacting. If this occurs, you must not disclose any information about that person to anyone.

As an interviewer, you should remain neutral. It is important that the respondent answer the questions as honestly as possible. There are no right or wrong answers for opinions and attitudes. Thus, you must be careful not to leave the impression that one answer is better than another, or that a respondent's answer is silly or dumb. Be as neutral as possible so that we can accurately record the respondent's opinion. Here are some tips to help do that:

• Read questions slowly and clearly
• Read questions exactly as written
• Ask all appropriate follow-up questions, including ones which ask the respondent to explain or clarify an earlier response
• Give no opinions
• Never suggest an answer
• Do not be mechanical (rather, be conversational)

It is important that you, the interviewer, manage the survey process in an orderly manner. Above all, you must maintain your composure. You will come in contact with many different types of people. Most people will be easy to interview, but others will be shy or intimidated. Still others will be argumentative and combative. You cannot get into an argument with the respondent. Suggestions of what to say to difficult or reluctant respondents are included in the fallback statements.

As in most things, your initial contact is very important. Read the introductory information at the beginning of the questionnaire to introduce yourself and the survey. Be as natural and smooth as possible (practice helps). You do not have to apologize; it is better to make the person feel that they are contributing to something important rather than suggesting it is unimportant with an apology. Remember that a person can refuse to give an interview. Suggestions on how to encourage people to respond are provided in the fallback statements, but do not be pushy or hostile. If a person refuses to cooperate, thank him or her and make note of it. Likewise, it is alright for someone to refuse to answer a particular question or not to have an opinion. In these situations, reread the question for the person or gently encourage him or her to respond, but ultimately the person does not have to answer if he or she does not want to. Don't force anything. Finally, end the interview by thanking the respondent for his or her help.