Significance tests for the measure of raw agreement are proposed. First, it is shown that the measure of raw agreement can be expressed as a proportionate reduction-in-error measure, sharing this characteristic with Cohen's Kappa and Brennan and Prediger's Kappa_n. Second, it is shown that the coefficient of raw agreement is linearly related to Brennan and Prediger's Kappa_n. Therefore, using the same base model for the estimation of expected cell frequencies as Brennan and Prediger's Kappa_n, one can devise significance tests for the measure of raw agreement. Two tests are proposed. The first uses Stouffer's Z, a probability pooler. The second test is the binomial test. A data example analyzes the agreement between two psychiatrists' diagnoses. The covariance structure of the agreement cells in a rater by rater table is described. Simulation studies show the performance and power functions of the test statistics. (author's abstract) / Series: Research Report Series / Department of Statistics and Mathematics
| Identifer | oai:union.ndltd.org:VIENNA/oai:epub.wu-wien.ac.at:epub-wu-01_ac9 |
| Date | January 2006 |
| Creators | von Eye, Alexander, Mair, Patrick, Schauerhuber, Michael |
| Publisher | Department of Statistics and Mathematics, WU Vienna University of Economics and Business |
| Source Sets | Wirtschaftsuniversität Wien |
| Language | English |
| Detected Language | English |
| Type | Paper, NonPeerReviewed |
| Format | application/pdf |
| Relation | http://epub.wu.ac.at/1336/ |
Page generated in 0.0018 seconds