Trainee Reactions
What are trainee reactions?
Trainee reactions refer to “subjective evaluations learners make about their training experiences” (Sitzmann et al., 2008, p. 280). Commonly referred to as “level 1” evaluations (e.g., Kirkpatrick, 1998), reactions are the most commonly used type of training evaluation and are measured with post-training surveys. Though measures are occasionally included in published research, there are no universal or even widely used measures. Instead, questions are developed by each organization or trainer based on their particular goals and interests. As a result, there is significant variation in what is measured. Originally, reactions referred only to how well a trainee liked a training program (Kirkpatrick, 1959), and measures still often tap satisfaction with various aspects of the trainer, the training, and other contextual factors. Among others, these include things like trainer preparedness, clarity, knowledge, and effectiveness; training materials, methods, organization, duration, and technology; and facility, food, or administrative aspects like enrollment processes. These kinds of reactions have come to be known as affective reactions (Alliger, 1997). In addition, another category of reactions is utility reactions, which refer to the perceived value or usefulness of the training (Alliger, 1997). These include perceptions about things like relevance, applicability, improvements in knowledge and skills, and potential to improve performance.
Are trainee reactions valuable?
Though reactions data can be used for a variety of purposes, meta-analytic research has focused on the extent to which reactions are related to learning and transfer outcomes. Results show that reactions are strongly connected to post-training motivation and moderately connected to self-efficacy (confidence in the ability to perform), even when controlling for pre-training motivation and self-efficacy (Sitzmann et al., 2008). Reactions are only modestly related to immediate knowledge outcomes and unrelated to delayed knowledge (Sitzmann et al., 2008). The connection with skill development is unknown. There is also no difference between affective and utility reactions in terms of their connections with learning outcomes (Sitzmann et al., 2008). Looking at the longer-term outcome of training transfer (i.e., when learning transfers to the job and improves performance), utility reactions have a modest connection, and affective reactions are not related (Blume et al., 2010).
What contributes to trainee reactions?
Meta-analytic research on trainee reactions thus far has focused on assessing factors that are merely associated with reactions, not on causal relationships. Those factors have included a small set of trainee characteristics and situational characteristics, leaving many other potential factors in question. More specifically, trainee reactions are more positive when trainees have positive motivation to learn during training, are low in anxiety (either as a general tendency or as a temporary state during training), or have an agreeable personality (i.e., tendency to be likable, friendly, flexible, cooperative; Sitzmann et al., 2008). The only other trainee characteristic that has been studied is having a mastery goal orientation (i.e., tendency to aspire to learn and build competence), which does not predict reactions (Sitzmann et al., 2008).
Only three situational characteristics have been explored, and all are positively related to training reactions. The trainer’s instructional style has a strong connection, such that trainee reactions are more positive when the trainer uses nonverbal and verbal communication that creates connections with trainees (Sitzmann et al., 2008). Example behaviors include making eye contact, moving around the room, addressing participants by name, and using personal examples and humor. Similarly, reactions are more positive when there is more human interaction involved in the training, either with the trainer or with other participants (Sitzmann et al., 2008). Finally, perceived organizational support has a modest connection to reactions, such that they are more positive when trainees believe their employer values their contributions and cares about their well-being (Sitzmann et al., 2008).
How can trainee reactions be useful?
Given the evidence here, an important piece of advice is to know the strengths and limitations of reaction measures. Reactions do predict changes in knowledge as a result of training, so they are not merely reflections of satisfaction, but that is not their best use. On the other hand, reactions are fairly powerful predictors of changes in motivation and self-efficacy, both of which are important in their own right. Self-efficacy is predictive of both knowledge acquisition in training and transfer (Blume et al., 2010; Sitzmann et al., 2008), making it a useful perception to include in a post-training survey.
Moving beyond the use of reactions as leading indicators of learning and transfer, there are other uses that are common but less conducive to empirical examination. Another important evaluation purpose is process evaluation; that is, trainee reaction data can be used to provide ongoing feedback about what is working and what needs improvement. Particularly in the case of asynchronous training, trainee feedback can be a critical mechanism for identifying issues that might not otherwise be detected. Reactions evaluations can also be used as one form of performance evaluation for trainers and training developers. For new workers who spend significant time in training before beginning their work in the office, training satisfaction may even serve as a temporary form of job satisfaction, given that attending training is their primary duty.
To make the best use of a trainee reaction measure, it is important to consider the purpose for having it. Consider the relevant stakeholders, what their needs are, and whether and how those needs can be met through gathering trainee reactions. One approach for thinking about potential purposes suggests that all reasons for doing training evaluation can be boiled down to three purposes: decision making, feedback, and marketing (Kraiger, 2002). Consider what decisions may need to be made, what feedback could be useful, and what information would reflect or support promotional efforts. After settling on a purpose and aligning survey questions accordingly, it is important to also ensure that the data are used in a way that best meets the identified goals.
QIC-WD Takeaways
- There are no universal or even widely used measures of trainee reactions; instead, questions are developed by each organization or trainer based on their particular goals and interests.
- Trainee reactions are strongly connected to post-training motivation and moderately connected to self-efficacy.
- Trainee reactions are only modestly related to immediate knowledge outcomes and unrelated to delayed knowledge; the connection with skill development is unknown.
- Utility reactions have a modest connection to training transfer, and affective reactions are not related to transfer.
- Trainee reactions are more positive when trainees have positive motivation to learn during training, are low in anxiety, or have an agreeable personality. Having a mastery goal orientation is not related to training reactions.
- Trainee reactions are more positive when trainers use nonverbal and verbal communication that creates connections with trainees; when there is more human interaction involved in the training, either with the trainer or with other participants; and when trainees believe their employer values their contributions and cares about their well-being.
- Beyond their use as leading indicators of learning and transfer, trainee reactions can provide valuable feedback for process evaluation.
- To make the best use of a trainee reaction measure, it is important to identify the purpose for having it, align questions accordingly, and ensure that the data are analyzed and distributed in a way that best meets the identified goals.
References
Alliger, G. M., Tannenbaum, S. I., Bennett, W., Jr., Traver, H., & Shotland, A. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50, 341–358.
Blume, B. D., Ford, J. K., Baldwin, T. T. & Huang, J. L. (2010). Transfer of training: A meta-analytic review, Journal of Management, 39, 1065–1105.
Kirkpatrick, D. L. (1959). Techniques for evaluating training programs. Journal of the American Society for Training Directors, 13(11), 3–9.
Kirkpatrick. D. L. (1998). Evaluating training programs: The four levels (2nd ed.). Berrett-Koehler Publishers.
Kraiger, K. (2002). Decision-based evaluation. In K. Kraiger (Ed.), Creating, implementing, and managing effective training and development: State-of-the-art lessons for practice (pp. 331–375). Jossey-Bass.
Sitzmann, T., Brown, K. G., Casper, W. J., Ely, K., & Zimmerman, R. D. (2008). A review and meta-analysis of the nomological network of trainee reactions. Journal of Applied Psychology, 93, 280–295.
Author(s)
Megan Paul, PhD, University of Nebraska‐Lincoln
Suggested Citation
Paul, M. (2022, February 9). Umbrella summary: Trainee reactions. Quality Improvement Center for Workforce Development. https://www.qicwd.org/umbrella/trainee-reactions
For general information about Umbrella Summaries, visit https://www.qic-wd.org/umbrella-summaries-faq