“Effects of Non-Normal Performance Distributions on the Accuracy of Utility Analysis”, 2018-07-09 ():
[abstract-only; doesn’t full paper ever published] Utility analysis estimates the dollar value of human resource management programs (ie. utility gain). We investigated whether a violation of the performance normality assumption affects the accuracy of the following 4 popular utility analysis procedures, all of which assume normality: 40% of mean salary, 70% of mean salary, global, and modified global.
To assess the accuracy of results derived from using these normality-based procedures, we compared them against results derived from a utility analysis procedure that uses observed performance distributions rather than assuming normality. We used 206 samples of individual performance encompassing 824,924 workers, consisting of researchers, entertainers, athletes, lawyers, managers, laborers, financial advisors, tellers, salespeople, medical doctors, programmers, recruiters, fundraisers, call center employees, typists, and operators.
Results: showed that the greater a procedure’s sensitivity to departures from normality as well as the more positively skewed the performance distribution, the greater is the underestimation of utility. In particular, we found that lower values of power law curve’s parameter α (denoting heavier positive skew) correspond to greater underestimation.
Results: also support the substantive explanation that normality-based utility analysis procedures underestimate utility by ignoring the presence of star performers. Specifically, underestimation was greater in contexts where more star performers exist (ie. longer time frames to allow more stars to emerge and higher performance ceilings).
Overall, our study suggests that utility analysis cannot automatically assume normally distributed performance because inaccuracy of results increases with greater deviations from normality.