THE KAPPA COEFFICIENT. Introduced by Xxxxx (1960), the Kappa Coefficient is an index for rater agreement on categorical variables. It has since become widely used when analysing data from the medical and health sciences. In sport science research, Kappa has been used to measure the reliability, or more accurately the agreement, of researchers conducting video analysis xxxxxxx00–37 and skill assessment studies38. Along with the popularity of Kappa, has come discussion about its appropriateness, relevance and reporting practices39–46. Though often used interchangeably, there is a difference between “agreement” and “reliability”43,47. Agreement should refer to the extent to which observations are identical, whereas reliability should refer to the extent of variability and error inherent within the measure43,47. This is pertinent in the case of the Kappa Coefficient, as categorical variables are not analysed with the traditional statistics used for measuring the reliability of continuous data. Kappa is derived from the proportion of observed agreement (Po) and the proportion of chance agreement (Pc), and is an index which represents the extent of agreement that can be achieved beyond chance (eq. 1)42,43,48. Kappa has been adapted to account for several situations which require different approaches to establishing rater agreement. The most common and basic situation requires the use of Xxxxx’x Kappa to determine true agreement between two raters making categorical ratings on players or participants. Weighted Kappa is an extension of Xxxxx’x Kappa for use with ordinal variables46. Fleiss, (1971) adapted Kappa for the measurement of agreement for multiple categorical variables, among any constant number of independent raters. The concepts of agreement and reliability, and the kappa statistics are relevant to the research described in chapter 4 because of the categorical nature of the data.
