We’ve discussed the importance of collecting treatment fidelity data and using those data to inform future supervision efforts in previous posts. This post discusses ways to track provider fidelity over time to assist in the decision-making process to ensure high-quality services are delivered.
Graphing Software
One way to track fidelity is using graphing software. Some common graphing software include Excel and GraphPad Prism. Data tables and graphs can be created to track fidelity, similar to how clinical data are graphed for clients. Although this is a great starting point for organizations and BCBAs, it takes time to create and upkeep this system. Additionally, the results are rarely shared with the providers(to my knowledge).
Performance Scorecards
Another way to track fidelity is using scorecards. Scorecards are a way for organizations to link fidelity data to organizational benchmarks or standards (Abernathy, 1996; Daniels & Bailey, 2014). Similar to graphs, they allow one to easily see fidelity over time. However, they aren’t just a mere reflection of the fidelity for a particular procedure. Scorecards allow organizations to link fidelity to important outcomes that they want employees to demonstrate. When creating scorecards, it is important to determine the influence of certain weights. Recent research from Dr. Sharlet Rafacz’ lab found that weights may communicate that some items aren’t as important as others.
How to create scorecards
There are many ways to create scorecards. DiGennaro Reed et al. (2022) discuss the steps, in detail, outlined below in their chapter. I recommend reading their chapter if you are interested in learning more about creating scorecards at your organization. Creating scorecards includes:
Identifying the behavioral measure,
Writing an operational definition,
Determining a scale,
Determining a weight, and
Calculating the score.
Benefits of scorecards
Although scorecards, too, take time to develop, they allow organizations the ability to ensure providers have the appropriate training, support, and resources to effectively do their job. Scorecards can be used as a measure of quality improvement. This means that the behaviors measured can be linked to standards set forth to demonstrate high-quality service delivery. Additionally, scorecards are a mechanism to provide feedback. The data are commonly shared with relevant parties so fidelity is known and all can work together toward organizational standards.
There are a few behavior-analytic studies that have discussed scorecards and how their results can be linked to pay. Check out Szabo et al. (2012) and Griffin et al. (2019) for more information!
References
Abernathy, W. B. (1996). The sins of wagers: Where the conventional pay system has led us, and how to find a way out. Performance Management Publications.
Daniels, A. C., & Bailey, J. S. (2014). Performance management: Changing behavior that drives organizational effectiveness (5th Ed.). Performance Management Publications.
DiGennaro Reed, F.D., Pellegrino, A. J., Blackman, A. L., Erath, T. E., Ruby, S. A., Harbison, M.J., & Maguire, H. (2022). Advancing OBM practice and research. In J.K. Luiselli, R. M. Gardner, F. L. Bird, & H. Maguire (Eds.), Organizational Behavior Management Approaches for Intellectual and Developmental Disabilities.Routledge.
Griffin, M., Gravina, N. E.,Matey, N., Pritchard, J., & Wine, B. (2019) Using Scorecards and a Lottery to Improve the Performance of Behavior Technicians in Two Autism Treatment Clinics. Journal of Organizational Behavior Management, 39(3-4), 280-292. https://doi.org/10.1080/01608061.2019.1632241
Szabo, T. G., Williams, W. L., Rafacz, S. D., Newsome, W., & Lydon, C. A. (2012). Evaluation of the service review model with performance scorecards. Journal of Organizational Behavior Management, 32(4), 274–296. https://doi.org/10.1080/01608061.2012.729408
Comments