Small-to-Medium-Scale Sensory Evaluation of Horticultural Crops: Standard Sensory Practices

Sean M. Campbell and Charles A. Sims

Introduction

Before starting any sensory evaluation procedures, it is important to address the protocols and practices for accurate and reliable testing, often referred to as “standard sensory practices.” While most testing is done in controlled environments designed to limit panelists’ biases, these facilities are not always available, leading sensory evaluation operators to seek alternative accommodations. Implementing these standard sensory practices can improve testing in any environment, while helping the operator determine the most appropriate sensory evaluation test for their needs. Figure 1 shows an example of a directional paired comparison test using hop pellets, with the appropriate score sheet and sample preparations.

This publication is the third in a series designed to assist producers in the small-to-medium-sized sensory evaluation of their horticultural crops, summarizing the standard sensory evaluation practices established for testing environment, sample ballots, panelist recruitment, and sample preparation, with the guidelines outlined in this publication taken from the 2nd edition of Sensory Evaluation of Food: Principles and Practices (Lawless and Heymann 2010).

A picture containing text, several, vegetable

Description automatically generated

Figure 1. Directional paired comparison test example using hop pellets. Sean M. Campbell, UF/IFAS

Testing Environment

The sensory testing environment will vary depending on the operator, but whether using an indoor, climate-controlled room or the back of a hop yard, some standard practices in location selection should be followed to make sure panelist bias is limited. Evaluation should be done away from excessive noise or smells, especially if they can interfere with testing. For example, you would not want to evaluate hops in an active brewhouse, where the smell of beer is predominant. The testing location should also be convenient for panelists and operators; having 50 panelists walking through an active farm or processing facility might not be feasible during production, and instead a more accessible location should be chosen.

Make sure panelists do not have access to any information that may induce biases. If panelists see how the products they are assessing are being made or prepared, this can cause them to evaluate samples incorrectly if they attempt to make assumptions based on that information. Panelists must also be restricted from seeing others’ results, because this can unconsciously influence their own decision-making. Formal sensory-testing facilities will have panels with constructed booths, isolating the panelists and allowing the operator to distribute samples from a prep area through hinged doors. In nonconventional testing environments, panelist separation can be accomplished with temporary booths made out of plywood or trifold poster boards, or by isolating people with distance.

Finally, the testing location should have access to adequate preparation facilities. Depending on the crop or product being evaluated, this includes access to water, refrigeration, sample preparation space, and storage for all of the necessary supplies and equipment. Depending on the intensity of testing, established panels will have enough room to set up multiple sessions of samples at once, but this is less necessary for small-to-medium-scale testing. Testing environment requirements can be found listed with the other parameters outlined by this publication in the sensory evaluation checklist (Figure 2).

Table

Description automatically generated

Figure 2. Sensory evaluation checklist. Credits: Sean M. Campbell, UF/IFAS

Blind Labeling/Randomization

In sensory evaluation, blind labeling is the process of giving samples a randomized 3-digit label to serve as an identifier. This is done to keep panelists from being able to discern any sort of pattern or order in the service of samples, rather than being presented in a sequential order (Sample 1, 2, 3…). Random number selection should be done using tables or random number generators found online, because numbers chosen by people are never truly random and will inherently contain patterns. Depending on the testing method used, sample order should also be randomized among panelists. That is, an equal number of panelists should receive Product A and Product B as their first sample. Randomization and selection of appropriate testing methods is discussed in more detail in the next publication in the series, ENH1339, Small-to-Medium-Scale Sensory Evaluation of Horticultural Crops—Sensory Testing Methods (https://edis.ifas.ufl.edu/ep603).

Sample Ballot

Once an appropriate testing method for the crop or product being evaluated has been identified and randomized 3-digit labels have been assigned to each of the samples, a sample ballot should be created. While details are specific to the testing method used, Figure 3 illustrates some sample ballot essentials.

Table

Description automatically generated

Figure 3. Directional paired comparison test sample ballot example. Credits: Sean M. Campbell, UF/IFAS

  1. Title: The title can include the name of the testing method, a brief description of the product, or both. Make sure the title contains enough information to allow panelists to make an informed decision about whether to participate, but without revealing anything that could cause a bias. Figure 4 is an example of a title for a test being done on a commercial beverage containing alcohol. The alcohol content is an important warning for potential panelists because paneling was done on a university campus and some panelists might be underage or have aversions to alcohol. Depending on the testing objective, it can also indicate the flavor of the product being tested. Finally, it is crucial that the title hide any brand names, which might cause bias during evaluation.
  2. Panelist instructions: Instructions should be clear and concise, and they should contain all the information panelists could need to properly execute the evaluation. Some testing methods are simpler than others and require less detail, but at the minimum all instructions should include directions for sample evaluation and sample order, and if applicable, they should identify the attribute being tested.
  3. Panelist response area: Having evaluated the samples according to the instructions, panelists need a way of recording their response. Some testing methods have panelists write in or circle the 3-digit code of the sample(s) they have chosen or indicate their response on a line or scale.
Text, whiteboard

Description automatically generated

Figure 4. Example title for lemon-flavored beverage alcohol testing. Credits: Sean M. Campbell, UF/IFAS

Panelist Recruitment

With an appropriate testing method chosen and the basic parameters set, panelists should be selected who are convenient to the testing location and familiar with the types of products being tested. They could be regular customers and consumers of your crop or product, repeat patrons of a brewery or distillery, or members of the local community. Recruitment can be done in a variety of ways, but most operators who plan on conducting testing on more than one occasion prefer to set up an email listserv system. This consists of collecting the contact information of a large pool of potential panelists who can then be contacted automatically with the details about any potential paneling for which they might qualify. Other effective methods include flyers and social media postings.

Informed Consent

With a pool of potential panelists, the primary consideration is making sure they have sufficient information to make an informed decision about whether to participate. Anytime human subjects are used for study, their health and safety must be of the highest priority. In academic settings, this is regulated by the institution’s Human Subjects’ Institutional Review Board (IRB). Because the protocols listed in this series are intended for small-to-medium-scale producers to use on horticultural crops and other related products that are regularly consumed by the public, and because the results should not be considered research or publication worthy, it is the authors’ recommendation that as long as these guidelines are followed, any resulting studies would pose no hazard “above the ordinary risks of daily life” to panelists.

Screening

Before being accepted, panelists should also be screened for some minimum qualifications. These primarily consist of ensuring that panelists have a base-level familiarity with the product or attribute being evaluated. This is easily accomplished when using regular customers or consumers but becomes more important if using members of the local community. Using the example in Figure 4, panelists could be screened using the questions in Figure 5. It is important that the questions obtain the desired information without revealing testing details. You would not want to ask if they regularly consume alcohol, but instead make alcoholic beverages some of the several answers in a multiple choice. As long as they indicate alcohol as one of their answers, they would qualify for that question. It is also important to note that the testing title should not contain any details being screened for, because this would also bias panelists’ responses for the screening questions.

Graphical user interface, table

Description automatically generated

Figure 5. Example screening questions for lemon-flavored beverage alcohol testing. Credits: Sean M. Campbell, UF/IFAS

Panelist Incentive

In order to encourage panelist participation, it is common to offer some sort of incentive or compensation, i.e., a monetary payout or a complimentary product. While this incentive is necessary in most cases, it is important to make sure it does not influence testing. Panelist involvement should be completely voluntary, rather than being an obligation for employment, membership, etc. It is also important that the incentive is enough that ample people participate, but not enough that it is their only reason for participating. This can cause panelists to lack motivation during testing; instead, it is better to find panelists who have an interest in the panel outside of compensation, such as brand loyalty or interest in seeing an improved product.

Sample Preparation

The most important consideration for sample preparation is standardization, namely, ensuring all variables between samples are the same except the one being tested. This is done to ensure panelists use that tested attribute to distinguish between samples, rather than trying to make assumptions based on other observations.

Serving Container

The container used to serve samples is highly product specific, but the first consideration is reusable vs. disposable. Glass and plastic containers like wine glasses can be washed and reused multiple times but require the labor and facilities necessary during and after testing. A good example is the glass wine glasses with plastic covers in Figure 6. Stemmed glasses are more indicative of how wine is normally consumed, and the plastic covers retain volatile compounds that would otherwise be lost as the panelists move through samples, but both are reusable.

Merlot sensory evaluation example.
Figure 6. Merlot sensory evaluation example.
Credit: Bradley Cooper, Wikimedia Commons

Alternatively, disposable containers like the 2 oz plastic souffle cups in Figure 1 are convenient and can be written on directly, but purchase costs and environmental impact may prohibit their use. Which category of container to use is largely dependent on the operator, but sensory specialists will often use a combination, choosing the most appropriate for the product being tested. Regardless of reusable or disposable, serving containers should be free of any tastes, odors, or markings that would interfere with panelists’ evaluations. Using markers to label containers works well, especially with disposable cups, but the cups must be marked well in advance so the marker smell can dissipate before being used for paneling. Alternatively, most sensory specialists employ labeling guns like those used to add prices to supermarket goods. These labels are cheap, are easily removable for reusable containers, are much faster than writing by hand, and have no offending odors to bias panelists.

Sample Size/Shape

If the sample appearance is not the attribute being assessed it should be standardized across samples. One method is to define a standard size, then choose samples that fall within ±10% (Figure 7). Furthermore, sample size and shape should be tailored to the crop or product and to the test objective. What is a normal portion size for this product? Should it be evaluated (visual appearance, odor, etc.) or consumed; and if consumed, should it be done in portions or as a whole? Can this product be made uniform easily, or is there large variability between individual units?

A picture containing fruit, custard apple

Description automatically generated

Figure 7. Sample size example using hop cones. Credits: Sean M. Campbell, UF/IFAS

Overall, operators should strive to give panelists a “representative” or “composite” sample of the product being tested. With larger items, this involves taking a representative sample, such as a slice, cube, or other portion. For smaller products, like hemp flower trimmings or hop cone pellets, it is better to include multiple units of the product to give panelists a composite sample, reducing variation between individual units. Note that when in doubt, it is better to give panelists too much of a sample rather than too little, because giving too little can cause limited interaction with the product.

Serving Temperature

Horticultural crop samples are most commonly served at ambient temperature, but other products necessitate either a raised or lowered temperature. This requires the samples to not only be taken to that temperature in a uniform fashion, whether cooked or refrigerated, but also that they be served at a uniform temperature. For example, hop cones that have been stored in the freezer should be allowed to come to room temperature before being served alongside freshly harvested cones, because the temperature difference would be easily detected by panelists. To help ensure uniformity, samples that are served at non–room temperature should be verified with a thermometer against a standard value before being served.

Serving Time

Serving, preparation, and holding times are also important and should have set standards because they can affect other variables within the samples. The amount of time taken between hemp flower trimmings being removed from storage and being assessed by panelists should be standardized among samples, because this can affect volatile oil concentrations and therefore overall smell intensity. If serving a carbonated beverage, the amount of time between opening the container and serving should be uniform, because this can cause a differential in carbonation. If serving a beverage with ice, a uniform number of pieces should be added with the same amount of time before consumption, reducing variation in melting.

Palate Cleansing

A standard practice across sensory evaluation, palate cleansing is the removal of residual materials, tastes, smells, and perceptions of one sample before moving on to the next. Lucak and Delwiche (2009) assessed a variety of oral cleansers across foods representing sweet (jelly beans), bitter (coffee), fatty (sausage), etc. and found that table water crackers were the only cleanser effective with all foods tested. A water and cracker tray is the common oral palate cleanser, preferably using distilled or deionized water to remove any conflicting flavors. When evaluating samples by smell, an effective way to cleanse is by smelling the back of the hand or the forearm. A technique pioneered in the fragrance industry, smelling your own skin helps reset the palate back to base levels (Rogers 2017).

Conclusions

Regardless of the horticultural crop or product being tested, following the standard sensory practices outlined in this publication for the testing environment, sample ballots, panelist recruitment, and sample preparation can reduce panelist biases and increase testing accuracy. Combined with the selection of an appropriate testing methods, as discussed in the next publication in the series, ENH1339, Small-to-Medium-Scale Sensory Evaluation of Horticultural Crops—Sensory Testing Methods (https://edis.ifas.ufl.edu/ep603), these procedures are designed to allow small-to-medium-scale operators to conduct their own sensory evaluations.

Literature Cited

Lawless, H. T., and H. Heymann. 2010. Sensory Evaluation of Food: Principles and Practices. New York: Springer. https://doi.org/10.1007/978-1-4419-6488-5_3

Lucak, C., and J. Delwiche. 2009. "Efficacy of Various Palate Cleansers with Representative Foods." Chemosensory Perception 2 (1): 32–39. https://doi.org/10.1007/s12078-009-9036-6

Rogers, L. 2017. Sensory Panel Management: A Practical Handbook for Recruitment, Training and Performance. Sawston, UK: Woodhead Publishing.

Peer Reviewed

Publication #ENH1338

Date: 2021-05-26
Campbell, Sean M
Sims, Charles
Environmental Horticulture
UF/IFAS Mid-Florida Research and Education Center, Apopka (MREC)

Related Topics

Fact Sheet Commercial

About this Publication

This document is ENH1338, one of a series of the Environmental Horticulture Department, UF/IFAS Extension. Original publication date May 2021. Visit the EDIS website at https://edis.ifas.ufl.edu for the currently supported version of this publication.

About the Authors

Sean Michael Campbell, postdoctoral scholar, Department of Viticulture and Enology, UC Davis; and Charles A. Sims, professor, Food Science and Human Nutrition Department; UF/IFAS Extension, Gainesville, FL 32611.

Contacts

  • Charles Sims