Extension faculty recognize the need for effective communication and distribution methods that best inform, motivate, and serve their clients (Telg, Irani, & Varvorines, 2008). While previous Extension documents discuss the need for successful Extension marketing campaigns, persuasive communications, and public relations (Gorham, Telg, & Irani, 2013; Telg et al., 2008; Hicks, Telg, & Irani, 2013), little information has been distributed about evaluating communication materials and campaigns.
Extension faculty and agricultural communication professionals should be designing their communication messages based on how their target audience uses and values the information they find in the communication materials (Ann, 2014; Dimas & Redish, 1999). Usability testing is an evaluation technique measuring how easily a target audience member can learn the information in the communication material and then make use of the information to accomplish a task (Dumas & Redish, 1999; Goodwin, Davis, & Telg, 2014). These tests evaluate users' perceptions of ease and value in order to find out the usefulness of the communication material and identify potential improvements that could increase effectiveness and efficiency for both stakeholders and the public (Dumas & Redish, 1999; Goodwin et al., 2014; Rubin & Chisnell, 2008).
One way Extension faculty can measure the usability of their communication materials is by using heat maps embedded in online surveys. Heat maps are used to enhance communication materials. For example, when creating web pages, heat maps are used to determine which parts of a page visitors click on the most, which may lead developers to assess what images or text draw people's attention more on that web page. A number of survey software programs provide heat map functions, such as Qualtrics (2014a), AirMagnet Survey (2015), Survey Analytics (2015), and SurveyGizmo (2013). In an online survey with heat map function, the image of a piece of communication material can be presented to the respondents with a heat map question. The question can request that the respondent click on an image on the computer screen to determine numerous usability functions (Qualtrics, 2014a). The survey also can record how long the task takes to complete in order to figure out the degree of functionality associated with the website (Qualtrics, 2014a). For example, in Figure 1 the heat map shows that more people click on the banana image than anywhere else on the computer screen, followed by the "Don't Pack a Pest" button at the top of the screenshot. The heat map allows Extension faculty to gather information and determine the ease of respondent use of the communication material. Researchers can use this information to improve communication strategies targeted toward communication material use and value.
Using Heat Maps
In order to use heat maps to perform a usability test, the Extension faculty member will need access to a piece of communication material, access to an online survey platform such as Qualtrics, and access to a sample of people that would be most likely to view the communication material (the target audience). Heat maps work inside an online survey platform such as Qualtrics, a survey software available to all University of Florida faculty, staff, and students (UF e-Learning Help, 2014). Note that other online survey tools also utilize heat maps and the specific procedure may vary; refer to your online survey's Help menu to learn how to use heat maps in another survey application. The steps for developing a heat map question in Qualtrics are described below.
Developing your Question
Develop user- or audience-based questions to determine how well the user finds information
Example statements for your questions:
Please click on the area of the following image where your eye went first when you looked at the page.
Please click on the area where you would most likely click first were you visiting this page.
Please click where you would go to find information about...
Please click on the area that attracts your eye the most.
Please click on the area you think needs improvement.
2. Take a photograph or screenshot of your communication material to be used in the question. We will use the mobile version of the "Don't Pack A Pest" website as an example.
Incorporating Your Question into Qualtrics
Before developing your questions in Qualtrics and using heat maps, it is recommended that you read the EDIS publications in the "Savvy Survey Series" (https://edis.ifas.ufl.edu/topic_series_savvy_survey). It is also recommended that those interested in using heat maps in Qualtrics should refer to lynda.com tutorials, available for free to University of Florida employees.
Select heat map as the question "item type" inside the survey software.
Select choose graphic to upload the screenshot or photograph into the survey software.
Enter the question in the question box.
Enter the number of clicks on the page that the respondent is allowed to make on the screen shot.
Evaluating Timing and Value
Add a timing question before the heat map question to evaluate how long it took the participant to click on the communications tool. While the researcher will be able to see how long it took a respondent on this page, the timing question will not be displayed to the survey respondents (Qualtrics, 2014b).
Add a value question after the heat map question to evaluate the value of the communication tool. You may incorporate Likert-type, semantic differential, and multiple-choice questions to evalute the value of the tool.
Likert-type questions are used to assess the respondents' levels of agreement or satisfaction with statements.
Please indicate your level of agreement or disagreement with the following statements about the website.
Please indicate your level of satisfaction with the following statements about the website.
Semantic Differential Question
Semantic differential questions should be used to determine the respondents' attitudes toward a specific concept. Semantic differential questions are used to determine where respondents fall between two opposite ends of a spectrum.
Please respond to the following items by marketing the circle most closely aligned with how you feel based on the two options provided:
I found the information presented on the website to be:
Not complete, complete
Hard to understand, easy to understand
Multiple-choice questions should be used to gather specific points of information about the respondents' experiences with the communication tool.
Please select the location on the website where bringing garlic from Puerto Rico was found.
Analyzing a Heat Map
Data will be recored as the respondents complete the survey.
The heat map can be observed to discover areas where the respondent clicked the most frequently. The information will be displayed in a heat map.
The higher the frequency of clicks in a specific location, the more red the area is or the "hotter" the map is.
In this example, respondents were most likely to click on Pest and on the Can I Bring It Logo.
Custom region shapes may be added or drawn around certain images within the communications tool.
By creating custom regions, the researcher is able to see how many people clicked in a specific area.
In this example, a region was drawn around the "Can I Bring It?" button.
Qualtrics will then output a summary of the number of people who clicked in a specific region such as "Can I Bring It."
In this example, there were 53 responses. Twenty-four of the respondents, or 45 percent, clicked in the "Can I Bring It" logo while 29 (54 percent) clicked in other areas of the communication tool.
An Example Application and Implications
Heat maps were recently used in a study to evaluate the usefulness of a website designed to provide educational resources regarding invasive species transferred on agricultural products to international travelers. The study used heat maps to determine the usefulness of the website on mobile, tablet, and desktop computer platforms. The perceived value of the website was measured through questions about users' general perceptions of the website, perceived navigation experience, and perceived quality and quantity of information.
Results indicated that respondents generally focused on the highlighted website elements, such as buttons with larger text and highlighted objects. The results showed click timing varied among the mobile, tablet, and desktop computer platforms, with desktop using the most amount of time for respondents to make a decision. These results show areas where communicators need to focus their attention to improve usability, such as emphasizing important areas with larger buttons, text, and highlights. Additionally, respondents took less time on platforms that had less information displayed on the website. While there is value in the information on the homepage, the homepage should be concise and simple to attract users to the most important place in a shorter amount of time. These results also show communicators how website information should be even more concise when it is displayed on mobile and tablet devices than it is on desktop computers.
By using a heat map as a tool to evaluate communication materials, Extension faculty and agricultural communication professionals can understand how their target audiences use the information found in specific communication materials, such as websites. By understanding how a piece of communication material is used by a target audience, the Extension faculty member or agricultural communicator can suggest recommendations or change the communication material to fit the needs of the target audience. Usability evaluation tools, such as heat maps, have the ability to improve communication messages and strategies between information senders and receivers; therefore these usability tools can help Extension faculty and agricultural communication professionals provide communication strategies and services that are more targeted to the usability needs of their target audiences.
AirMagnet Survey. (2015). http://www.flukenetworks.com/products/airmagnet-survey
Ann, J. (2014, January 29). "Why usability is important for web design." Retrieved from http://fatcow.com/blog/2014/01/29/why-usability-is-important-for-web-design/
Dumas, S. C., & Redish, J. C. (1999). A practical guide to usability testing. Portland: Intellect Books.
Goodwin, J. N., Davis, A., & Telg, R. W. (2014). Communication audits: Adding value to social impact to agricultural communications. Journal of Applied Communications, 98(1), 8-16.
Gorham, L. M., Telg, R. W., & Irani, T. (2013). Persuasion as a communication tool. WC148. Gainesville, FL: University of Florida Institute of Food and Agricultural Sciences.
Hicks, R., Telg, R. W., & Irani, T. (2013). Event marketing and public relations. WC144. Gainesville, FL: University of Florida Institute of Food and Agricultural Sciences.
Qualtrics (2014a). Heat map. Retrieved from http://www.qualtrics.com/university/researchsuite/basic-building/editing-questions/question-types-guide/heat-map/
Qualtrics. (2014b). Timing. Retrieved from http://www.qualtrics.com/university/researchsuite/basic-building/editing-questions/question-types-guide/timing/
Rubin, J., & Chisnell, D. (2008). Handbook of usability testing: How to plan, design, and conduct effective tests (2nd ed.). Indianapolis, IN: Wiley Publishing, Inc.
Survey Analytics (2015). Heat map analysis/question type. https://www.surveyanalytics.com/system/heatmap.html
Surveygizmo (2013). Using highly interactive questions in online surveys – Pros and cons. http://www.surveygizmo.com/survey-blog/using-highly-interactive-questions-in-online-surveys-pros-and-cons/
Telg, R., Irani, T., & Varvorines, J. (2008). Event marketing: Building a successful local Extension marketing campaign. AEC396. Gainesville, FL: University of Florida Institute of Food and Agricultural Sciences.
UF e-Learning Help. (2014). Qualtrics. Retrieved from https://lss.at.ufl.edu/help/Qualtrics
Wright, D. K., Burson, H., & Hinson, M. D. (2014). An updates examination of social and emerging media use in public relations practice: A longitudinal analysis between 2006 and 2014. Public Relations Journal, 8(2). Retrieved from www.prsa.org/intelligence/prjournal/documents/2014wrighthinson.pdf