I continue to study the problem I have to confront in the near future,

Best Regards

Markku

]]>That’s an interesting question. Theoretically, kappa would work in this case, but you’d have to be careful with the interpretation. I’ve never seen kappa with so many categories. You can inflate free-marginal kappa by adding superflous categories that aren’t theoretically defensible. With 130 categories the probability of change agreement is .008. However, the probability of chance agreement wouldn’t actually be .008 if many of the 130 categories are “throw away” categories that you or the other raters wouldn’t really pick. My suggestions is to only use as many categories as a theoretically justifiable . . . all of your 130 cats might be theoretically justifiable and you’d just have to make that argument. You might report the fixed marginal kappa in this case (or both).

With that said, I’ve never tried the OKC with 130 cats. Let me know what happens.

]]>Hi Gina,

I don’t think that I saw this before because it’s in the about section.

I think that the OKC will work for you. I’d put the data in a spreadsheet first and try to cut and paste. You don’t want to input all of that data by hand in case there’s an error!

]]>I started to wonder if you or somebody else using this website could help me with the followning question.

I have conducted face to face interviews and from the transcribed text two independent researchers have identified meaning units that were linked to ICF (international classification of functioning disability and health).No I shoud calculate the kappa statistics on inter rater agreement. The problem is that the ICF core set that we have used has 130 categories. So everytime I choose a category I choose among 130 categories as well as the other rater.

Is it possible to calculate Kappa statistics of this? ]]>

i came to your online calculator as i have trouble doing Kappa in SPSS and need a calculator that can digest my dataset. Most online calculators only do 8 or 12 categories but not more.

I have a dataset with 2 coders, 34 categories, and 135 cases (which will grow up about 400 cases for each kappa i need to calculate); and as i have 6 2-way combinations of 4 raters i will have to compute 6 different kappas.

Of course, so many categories mean i get a lot of zero’s in the category-by-category square, which might cause my SPSS not to calculate it. If i enter all of these data into your calculator (format is different than the SPSS output i get, so i would have to do it all by hand, no C&P) do you think your calculator will be able to digest this? Or do you have any other recommendations how i could best do this?

Thank you!

Gina

]]>Hi Emmanuel,

I just checked and it is working from two of my computers. You need to have the latest version of java installed to get the calculator to work in your browser. If you already have Java, try to uninstall it and reinstall it. That has worked for others who have had the same problem. If worse comes to worse, you might have to use the calculator from a different computer.

Justus

]]>THANKS SO MUCH FOR THE ON LINE CALCULATOR. BY THE WAY AM WRITING FROM NIGERIA WEST AFRICA.

EACH TIME I OPEN THE WEB PAGE OF THE CALCUTOR, I CAN ONLY SEE THE INSTRUCTIONS BUT THE TABLE NEVER APPEARS FOR ME TO LOAD DATA. I’VE SPENT SEVERAL HOUR TRYING BUT SAME RESULT.

PLS TELL ME WHAT TO DO NEXT.

THANKS.

EMMMANUEL.

]]>Hi Shivani,

I tried out a hypothetical data set with those parameters and it

seemed to work. Be sure that all of the rows sum to 2, because that’s

how many raters you have. ‘m assuming that raters can only assign one

category to each case. Also be sure that all of the empty cells have

a zero in them. Don’t leave any cells blank. If that doesn’t work, let

me know what the error is and post your dataset and I’ll see if I can

figure out what’s happening.

Take care,

Justus

]]>