The Savvy Survey #12: Telephone Surveys

Introduction

This publication is one of several in the Savvy Survey Series that highlights procedures for collecting data from respondents. The focus here is on using telephone surveys, which includes both landline and cellular phones. In addition, our focus is limited to conducting telephone surveys using an available list because conducting telephone surveys using a random-digit dialing sample of the general public (i.e., a random sample of adults in a county or the entire state) requires specialized resources. Although a telephone survey of the general population might be desirable for a county- or state-wide needs assessment, this is a complex task that is better suited for professional survey contractors. In addition, telephone surveys have faced a number of technological and cultural challenges in recent years, resulting in very low response rates for many surveys (Dutwin and Lavrakas, 2016; Kennedy and Hartig, 2019; Lavrakas, 2010).

When a list of phone numbers is available, telephone surveys can be very useful for conducting follow-up surveys or evaluating program outcomes. For example, client contact lists have been used to conduct a customer satisfaction survey (Terry and Israel, 2004). Telephone surveys often can be completed faster than other survey modes. Telephone surveys also can be more cost effective than mail surveys when a volunteer pool or staff are available to be interviewers. On the other hand, hiring an external organization to conduct a telephone survey can be expensive.

In some cases, list-based phone surveys can also be used for conducting a needs assessment of a specific population. Organizations with membership lists that contain phone numbers are suitable for a telephone survey to assess needs. Such groups or organizations might include members of a local Farm Bureau, a Nursery Growers and Landscape Association, alumni from a leadership institute program, Master Naturalist program alumni, and many others.

Finally, while telephone surveys can be combined with other survey modes, such as mail or online surveys, these mixed-mode surveys combining telephone and self-administered approaches involve added complexity of combining aural and visual modes of communication. Consequently, this publication focuses on developing and conducting stand-alone telephone surveys. This fact sheet highlights the development of an instrument using the telephone. Also, preparing an introductory script, training interviewers, and managing the survey process are discussed.

Developing the Telephone Questionnaire

The construction of a questionnaire to be administered over the phone requires careful attention to detail but, in particular, accounts for differences in the senses used by respondents to complete the survey. While mail and online surveys use visual modes to present questions and display response options, telephone surveys rely on an interviewer to orally ask the question and provide the options for answering. Thus it is a poor practice to read a mail or online survey to a respondent over the phone. Instead it is a common and recommended practice for phone surveys to include the responses in the stem of the question (see Table 1). Notice also that the list of response options and associated code numbers includes "Don't Know" and "Refused" categories to cover these situations, even though the interviewer does not read them to the respondent. In contrast, the mail version includes instructions for marking an answer and omits responses options for "Don't know" and "Refused."

Finally, the telephone questionnaire can be created in a paper format whereby interviewers will mark or write the responses. It can also be created using survey software, such as Qualtrics, in which the interviewer enters the data into the online version of the survey while talking to the respondent. If Qualtrics is used, it is important that the questions are formatted for oral administration so that interviewers ask the questions with the response options in the stem.

Developing the Introductory Script and Support Materials

In addition to tailoring the questionnaire for the telephone, an introductory script and other support materials must be prepared before implementing the survey. The introductory script is used by the interviewer at the beginning of the interview to obtain the respondents' cooperation. Paul Lavrakas (2010) notes that interviewers have about 20–30 seconds to obtain a respondent's agreement to participate, so this means that scripts should be as brief as possible while communicating critical information (Table 2). This includes the purpose of the survey and language for informed consent that might be required by an institutional review board.

In addition to drafting the introductory script, a call sheet (or its electronic equivalent) needs to be prepared. The call sheet is used to keep track of the date and time that a given number is called. It also is used to record what happens (referred to as the disposition) each time the number is dialed. Common disposition categories include No Answer, Non-working Number, Incorrect Number, Respondent Refusal, Other Refusal, Call Back Scheduled, Ineligible Respondent, Partial Interview, and Completed Interview (see also Lavrakas 1993). Table 3 shows an example of a call sheet used for a recent customer satisfaction survey. Note that the call sheet includes a Customer Satisfaction Survey ID number (e.g., CSSID = 057) which is matched with a copy of the questionnaire, as well as a name and phone number. It also includes space for recording notes about the dialing and interview process.

Other materials to be prepared include fallback statements and an interviewer orientation guide. Fallback statements are used by interviewers to provide an answer to a question by a respondent or to address a concern or problem. For example, if a respondent expresses concern about how his or her answers will be used, an interviewer might say, "Your answers will be combined with those of other people to provide a summary of what people think about Extension's services. Only grouped responses will be used in reports or public presentation." Additional examples of fallback statements are included in Appendix A (see also Israel, 2000; Lavrakas, 1993).

Finally, an interviewer orientation guide should be developed for use in training and supporting the persons who will be conducting the telephone interviews, especially when volunteers or support staff will be filling this role. The guide should include a few pages which fully explain the purpose of the survey and expectations for interviewer conduct. It might also include scenarios for interviewers to review and practice, the set of fallback statements, and a copy of the call sheet and questionnaire.

Conducting Interviewer Orientation

A number of interviewers will be needed to conduct the telephone survey. The number needed will depend, in part, on the desired number of interviews to complete (see Savvy Survey #3) and the timeline for conducting the survey. Small-scale surveys conducted over a month or two can be completed with just a few interviewers (e.g., 4–6) while larger surveys might need 10–20 interviewers. For Extension surveys, office staff and volunteers may be suitable to fill interviewer positions. Persons who might have a conflict of interest with the survey results should avoid being an interviewer. For example, an Extension agent who conducted an educational program should not interview clients as part of an evaluation of outcomes because many people are likely to feel some pressure to give a more positive assessment.

When the interview guide is prepared and interviewers have been recruited, an orientation session should be scheduled. Within this session, the survey manager should engage the interviewers in a discussion of the purpose of the survey, expectations for the interviewers (see Appendix B for suggestions about expected roles), and any survey scenarios and fallback statements (Israel, 2000; Lavrakas, 1993). The manager should also conduct a mock interview where interviewers can practice reading through the associated scripts and questions, as well as record-keeping using a call sheet (Israel, 2000; Lavrakas, 1993). An example of two survey scenarios and possible solutions are shown in Table 4. Spending time with interviewers to build a common understanding of the survey procedures and develop necessary skills is critical to conducting a quality survey and obtaining useful data.

Implementing the Survey

Implementing a telephone survey involves a number of steps and requires high attention to detail by a supervisor or manager.

Step 1. Use an advance contact, if appropriate.

When the sample list contains both telephone numbers and postal addresses, a letter can be sent before initiating the telephone calls. Response rates have been shown to be increased by sending a pre-letter in the mail to notify individuals about the upcoming telephone survey (Lavrakas, 2010). The advance letter can increase contacts by getting more people to answer the phone and can reduce refusals by introducing the survey and establishing its legitimacy. A sample letter is illustrated in Appendix B.

Step 2. The survey manager (or appropriate support staff) should pre-fill information on the questionnaire (only an ID number) and the call sheet (ID number, respondent's name, telephone number, and other relevant information).

Step 3. The survey manager should ensure that each interviewer has a complete set of materials (e.g., script, questionnaires, call sheets, fallback statements, etc.) and review record-keeping procedures with interviewers before starting the calling process.

Step 4. Interviewers begin dialing assigned phone numbers, using the procedures outlined in the interviewer expectations (see Appendix C). In processing phone numbers, the following is recommended:

• Non-working numbers should be re-dialed immediately (a second time) to confirm that the number is ineligible
• Check call sheets for a call back to a person at a specific time
• Vary the day of the week and time of day that follow-up dialings are made, e.g., if a number is not answered during the afternoon, then try an evening or Saturday morning dialing
• Each number is "retired" after an appropriate number of attempts is made (typically 6–8 attempts), except when a call back is scheduled
• Check that follow-up dialings have been conducted for each number

Step 5. The survey manager should check call sheets to ensure completeness of the record keeping and determine which numbers should be dialed during the next calling session. Likewise, completed questionnaires should be reviewed for completeness, and if data are missing, a call back to the respondent can be scheduled to collect the missing information.

Step 6. When all of the telephone numbers have resulted in a completed questionnaire or been retired, the survey manager arranges for the data-entry process to begin (see Savvy Survey Series #15: Survey Responses and Data Entry). Alternatively, interviewers may use Qualtrics survey software to enter the data as the interview is being conducted.

Language Considerations

Increasingly, Extension clients and the population at large includes people who are not native English speakers or they prefer to speak another language. For example, many clients in the landscaping industry have Spanish as their primary language. In this situation, it is important to translate all survey materials (questionnaires, introductory scripts, fallback statements, etc.) into the language needed for that client group. In situations where just a few clients might be non-English speakers, preparing an appropriate fallback statement (about having a translator call back) and having a person who can administer the survey by translating "on the fly" can be helpful in obtaining a completed questionnaire from these persons.

In Summary

This publication provides a brief overview of how to develop and conduct a telephone survey to collect data. It assumed that a list of phone numbers, such as program registration lists, is available for conducting program evaluations or assessing needs. When volunteers or staff assist with the survey by interviewing respondents, this data collection method can be economical and effective. Careful attention is needed when developing the questionnaire and supporting materials and when orienting interviewers in order to obtain credible survey data. In the right situation, telephone surveys can be a valuable tool for Extension agents and specialists.

References

Dutwin, D., and Lavrakas, P. (2016). “Trends in Telephone Outcomes, 2008-2015.” Survey Practice 9(3). https://doi.org/10.29115/SP-2016-0017

Israel, G. D. (2000). Conducting a Customer Satisfaction Survey. AEC356. Gainesville: University of Florida Institute of Food and Agricultural Sciences.

Kennedy, C., and Hartig, H. (2019). Response rates in telephone surveys have resumed their decline. Pew Research Center. https://pewrsr.ch/2XqxgTT

Lavrakas, P. J. (1993). Telephone Survey Methods: Sampling, Selection, and Supervision, 2nd ed. Thousand Oaks, CA: Sage Publications.

______. (2010). “Telephone Surveys.” in Marsden, P. V., & Wright, J. D. (eds.), Handbook of Survey Research, 2nd ed., 471-498. Bingley, UK: Emerald Group.

Terry, B. D. and Israel, G. D. (2004). “Agent Performance and Consumer Satisfaction.” Journal of Extension 42(6). https://archives.joe.org/joe/2004december/a4.php

Appendix A: Sample Fallback Statements

IF A NATIVE SPANISH SPEAKER

Usted solamente habla español? Puedo conseguir que alguien que habla español le llame de regreso en unos pocos minutos. Eso le parece bien?

(Do you only speak Spanish? I can have someone who speaks Spanish call you back in a few minutes. Would that be okay?) Note: when non-English speakers are called, it is helpful to have a scripted fallback statement for a language that is common in the area.

IF IN DOUBT

You were chosen as part of a random sample of people in order to obtain their opinions. It's very important to learn how people feel about our services and the usefulness of the information that we provide.

I DON'T HAVE TIME FOR THIS

I understand that you have a busy schedule but it's very important that we speak to busy people like yourself in order to get an accurate cross-section of people. We can do the interview at your convenience. How about tomorrow afternoon at 3 o'clock, or tomorrow evening at 8 p.m.?

HOW CAN I BE SURE THAT THIS IS LEGITIMATE?

I would be glad to give you the telephone number of the faculty member in charge of the survey, who will provide you with more information. His/hers name is _____________________. I am sure (s)he would be happy to talk with you. The number to call him(her) is ________-________.

RESPONDENT WANTS TO BREAK OFF THE INTERVIEW

We have about ______ more questions. If you would like, I can read the questions a little faster.

OR

We have only a few more questions, if you'll just bear with me.

CALL BACKS

INTRODUCTION FOR A SCHEDULED CALL BACK

I'm _____________________ calling from _____________________. You spoke with one of our interviewers (a few days/a week ago) and scheduled a call back to complete the interview. As you recall, we left off at the question on _____________________.

IF THE RESPONDENT IS RELUCTANT TO FINISH THE CALL BACK

We'd really appreciate your cooperation in (completing/ doing) the interview we started. It will just take a few more minutes of your time.

REFUSALS

Try to get every valid interview, so, if possible, try to persuade each person to respond. A few suggested responses to reasons for refusing to give an interview are listed below.

TOO BUSY

This should take only a few minutes. I'm sorry to have caught you at a bad time. I would be happy to call you back. When would be a good time for me to call in the next day or two?

BAD HEALTH

I'm sorry to hear that. I would be happy to call you back at another time. Would it be okay to wait a few days and call back to see if you feel up to talking to me?

TOO OLD

Older people's opinions are just as important in this particular survey as anyone else's. In order for the results to be representative for all the people that we serve, we have to be sure that older people have as much chance to give their opinion as anyone else does. We really do want your opinion.

DON'T KNOW ENOUGH TO ANSWER

The questions are not all difficult. Some of the people we have already interviewed have had the same concern you have, but once we got started, they didn't have any difficulty answering the questions. Maybe I could read just a question or two to you and you can see what they are like.

NOT INTERESTED

It's important that we get the opinions of everyone in the sample, otherwise the results won't be very useful. So, I'd really like to talk to you.

Appendix B: Sample Advance Letter

Appendix C: Interviewer Expectations

When you agree to be an interviewer, you must also agree to follow some simple but important guidelines. The first and most important is to honor confidentiality. In order for surveys to be successful, respondents must be assured that their answers will be kept confidential. This means that you cannot identify the answers with a name or address. However, in a small community it may be possible that you know the person you are contacting. If this occurs, you must be committed not to disclose any information about that person to anyone.

As an interviewer, it is also important that you remain neutral. It is important that the respondent answer the questions as honestly as possible. In particular, there are no right or wrong answers for opinions and attitudes. Thus you must be careful not to leave the impression that one answer is better than another, or that a respondent's answer is silly or dumb. Be as neutral as possible so that we can accurately record the respondent's opinion. Here are some tips to help do that:

• Read questions slowly and deliberately
• Read questions precisely as written
• Ask all appropriate follow up questions, including ones which ask the respondent to explain or clarify an earlier response
• Give no opinions
• Avoid reinforcements, but show interest
• Never suggest an answer
• Do not be mechanical (rather, be conversational)

It is important that you, the interviewer, manage the interview in an orderly manner. Above all, it is important that you maintain your composure. You will come in contact with many different types of people. Most people will be easy to interview, but others will be shy or intimidated. Still others will be argumentative and combative. You cannot get into an argument on the phone. Nor can you hang up on the respondent. Suggestions of what to say to difficult or reluctant respondents are included in the fallback statements.

As in most things, your initial contact is very important. Read the introductory script at the beginning of the questionnaire to introduce yourself and the survey. Be as natural and smooth as possible (practice helps). You do not have to apologize for the call; it is better to make the person feel they are contributing to something important rather than suggesting it is unimportant with an apology. It is important to remember that a person can refuse to give an interview. Suggested ways to encourage people to respond are provided in the fallback statements, but do not be pushy or hostile. If a person refuses to cooperate, thank him or her and make note of it on the call sheet provided. Likewise, it is okay for someone to refuse to answer a particular question, or not to have an opinion. In these situations, reread the question for the person or gently encourage him or her to respond, but the person does not have to answer if he or she does not want to. Do not force anything. Finally, end the interview by thanking the respondent for their help.

Record keeping is important to keeping the survey process organized. Because consistent procedures must be used, a record should be made of what is done each time a telephone number is dialed. Call sheets are used to keep track of when a number is dialed, who makes the call, and what happens. This information is used, for example, to decide when to call again or to retire the number. When using the call sheet and survey forms, you should do the following:

• Write the date, time, and your name on the call sheet before dialing a number.
• Write the disposition status on the call sheet after dialing a number.
• Confirm that the ID number on the call sheet matches the ID number on the questionnaire.
• Fill out the call back section on the call sheet if you started an interview but were not able to complete it.
• Fill out the call back section on the call sheet if the person cannot conduct the interview at the time you call.
• If the person does not want to give an interview, write the reason for the refusal in the notes section of the call sheet.