OVERALL AGREEMENT FOR MULTIPLE RATERS WITHOverall Agreement • June 9th, 2020
Contract Type FiledJune 9th, 2020CP, TDI and RAUCPC are all originally defined for two raters. However, there may be some competing new raters developed at the same time that need to be compared to each other or to an existing rater. Here we are interested in the interchangeability among more than two raters. For example, in the data set published in the Bland and Altman’s paper [5], the blood pressures of 85 patients were measured by two human observers and one device. We are interested in whether these three raters (two human observers and one device) can be used interchangeably. Therefore, it is desirable to extend the CP, TDI and RAUCPC to measure agreement among multiple raters while preserving the intuitive interpretation of pairwise version. Lin et al.[6] first extended the concept of CP and TDI to multiple raters using two-way mixed model and later Jang et al.[7] proposed a new set of definitions based on the root mean square of pairwise differences(RMSPD). Although these overall agreement indices give some in