The Association between Structured Professional Judgment Measure Total Scores and Summary Risk Ratings: Implications for Predictive Validity

Structured professional judgment (SPJ) instruments are used by mental health professionals to assess risk for future violence and assess treatment needs. Current literature tends to examine SPJ instruments in a way that is not congruent with how the instruments are designated to be used in the field. Specifically, studies often leave out analyses of the structured professional judgment piece of the instrument (summary risk rating, SRR) and, when the analysis is included, authors rarely compare the SRR to the actuarially derived total score. This study sought to provide practitioners with a comparison of the SPJ measure total scores and SRRs. I conducted a review of the literature to find studies that measured the predictive validity of SPJ measure total scores and SRRs. I requested additional data from corresponding authors in order to compare the two scores using varying statistical methods. In total I included 69 samples (n = 10,871). I performed several meta-analyses to determine if a) the predictive validity of the total score and SRR were similar, and b) if the SRR adds any additional predictive power to the total score. Findings suggest that the total score and SRR have similar predictive abilities. The small difference between the mean weighted SRR (AUC = .701) and total score (AUC = .698) effect sizes was not significant. I also calculated a z-score to test the difference between the SRR and total score effect size in each sample, and found a statistically significant difference in only 8 of the 69 samples. However, a meta-analysis of odds ratio values from logistic regression models including effects for both total scores and SRRS revealed a consistent incremental validity effect for SRRs (OR = 1.96, p < .001) over total scores. Overall, this review provides evidence to suggest that the total score and SRR provide similar predictive effects, but also reveals that using the SRR is worthwhile for practicing clinicians. Implications for both research and practice are discussed.