Cohen's Kappa Calculator
Chance-corrected agreement for two raters and a yes or no decision.
What it does
Cohen's kappa measures how much two raters agree on a categorical label after accounting for agreement that could happen by chance. It is common in chart review, diagnosis, content coding, and any study where two people classify the same items.
The input is a 2x2 table: both raters said yes, one said yes and the other said no in each direction, and both said no. Observed agreement comes from the diagonal. Expected agreement comes from each rater's yes and no totals.
The defaults describe 100 items with 85 on the diagonal. Observed agreement is 85.0%, expected agreement is 50.0%, and kappa is 0.700, which falls in the substantial band on the Landis and Koch scale.
The correction matters when one answer is much more common than the other. Two raters can agree 85% of the time and still have a modest kappa if their marginal totals make that much agreement likely by chance.
How to use
- Count the items both raters labeled yes.
- Count the items rater 1 labeled yes and rater 2 labeled no.
- Count the items rater 1 labeled no and rater 2 labeled yes.
- Count the items both raters labeled no.
- Press Calculate to see observed agreement, expected agreement, and kappa.
- Report the kappa with the table and the rating instructions.
Formula
- a
- Both raters said yes.
- b
- Rater 1 said yes and rater 2 said no.
- c
- Rater 1 said no and rater 2 said yes.
- d
- Both raters said no.
- Po
- Observed agreement, the share on the diagonal.
- Pe
- Agreement expected from the row and column totals alone.
If both raters use only one category, Pe is 100% and kappa is undefined; the tool returns an error. The calculation treats both raters as fixed and both categories as equally important. It does not weight near misses, which matters when the labels have a natural order.
Worked example
Example: 40 both yes, 10 and 5 disagreements, 45 both no
Defaults: both yes 40, rater 1 yes and rater 2 no 10, rater 1 no and rater 2 yes 5, both no 45.
- Total items: 40 + 10 + 5 + 45 = 100.
- Observed agreement: (40 + 45) / 100 = 85.0 %.
- Rater 1 yes share: 50 / 100 = 0.50; rater 2 yes share: 45 / 100 = 0.45.
- Rater 1 no share: 0.50; rater 2 no share: 0.55.
- Expected agreement: 0.50 x 0.45 + 0.50 x 0.55 = 0.500, or 50.0 %.
- Kappa: (0.85 - 0.50) / (1 - 0.50) = 0.700, Substantial.
Reference table
| Kappa | Band |
|---|---|
| 0.81 to 1.00 | Almost perfect |
| 0.61 to 0.80 | Substantial |
| 0.41 to 0.60 | Moderate |
| 0.21 to 0.40 | Fair |
| 0.01 to 0.20 | Slight |
| 0.00 or below | Poor or worse than chance |
| 0.700 | The example: substantial |
Limits
- This version handles two raters and two categories only.
- Kappa can fall when one category is much more common, even if agreement looks good.
- Unweighted kappa treats every disagreement the same, which is wrong for ordered scales.
- The bands are conventions from one 1977 paper, not clinical thresholds.
Common mistakes
- Reporting raw percent agreement and calling it kappa.
- Building the table from totals instead of item-by-item ratings.
- Ignoring very uneven yes and no counts when reading the result.
- Using unweighted kappa for an ordinal scale such as mild, moderate, severe.
- Treating 0.61 as a guarantee of useful agreement in a high-stakes setting.
Try it yourself
Quick checks. Answers sit tucked under each question.
What is kappa for 20 both yes, 5 rater 1 only yes, 10 rater 2 only yes, and 65 both no?
Observed agreement is 85.0 %, expected agreement is 60.0 %, and kappa is 0.625, in the Substantial band.
Both raters agree on 90 of 100 items, with 45 both yes and 45 both no. What is kappa?
Observed agreement is 90.0 %, expected agreement is 50.0 %, and kappa is 0.800.
Both raters marked every item yes. What happens?
Chance agreement is 100 %, kappa is undefined, and the tool returns an error.
Questions
What is a good kappa value?
Many studies call 0.61 to 0.80 substantial and 0.81 or higher almost perfect. The right cutoff depends on the cost of disagreement.
Why is kappa lower than percent agreement?
Kappa subtracts the agreement expected from the raters' own yes and no shares. Raw agreement gives credit for that chance agreement.
Can kappa be negative?
Yes. Negative kappa means agreement worse than chance, though small negative values can appear from sampling noise.
Does kappa measure correlation?
No. It measures agreement on labels. Two raters can correlate strongly and still disagree on the absolute category.
What if my scale has three or more categories?
Use a multi-category kappa or a weighted kappa. This page is limited to a 2x2 table.
Should I report a confidence interval?
Yes for a study estimate. This tool reports the point estimate only, so a separate interval is still needed.