23/08/2008 HJCL, COLING Workshop, 2008 19 Computation of P e (Contd….) The probability that two arbitrary annotators agree on a category pair for all assignment combinations is The chance agreement is P e = 0.46 A m = 0.33 A-DA-SA-HD-SD-HS-H 0.3750.50.250.50.3750.623
{}