About

Justus Randolph was born and raised in beautiful Ogden, Utah, U.S.A. He currently lives in Joensuu, Finland.

11 comments

May 13, 2009 at 12:36 am

Shivani

Hello, I am trying to use your online calculator for my dissertation – I have a qualitative study with 17 categories, 6 subjects and 2 raters. But every time I try to enter the data and press calculate – I get different error messages. Any way you can help me figure out what I’m doing wrong? I’d greatly appreciate it! Thank you! Shivani

May 13, 2009 at 2:16 am

jrandolp

Hi Shivani,

I tried out a hypothetical data set with those parameters and it
seemed to work. Be sure that all of the rows sum to 2, because that’s
how many raters you have. ‘m assuming that raters can only assign one
category to each case. Also be sure that all of the empty cells have
a zero in them. Don’t leave any cells blank. If that doesn’t work, let
me know what the error is and post your dataset and I’ll see if I can
figure out what’s happening.

Take care,

Justus

May 21, 2009 at 7:32 pm

EMMANUEL

HI JUSTUS,

THANKS SO MUCH FOR THE ON LINE CALCULATOR. BY THE WAY AM WRITING FROM NIGERIA WEST AFRICA.

EACH TIME I OPEN THE WEB PAGE OF THE CALCUTOR, I CAN ONLY SEE THE INSTRUCTIONS BUT THE TABLE NEVER APPEARS FOR ME TO LOAD DATA. I’VE SPENT SEVERAL HOUR TRYING BUT SAME RESULT.

PLS TELL ME WHAT TO DO NEXT.

THANKS.

EMMMANUEL.

May 22, 2009 at 6:54 am

jrandolp

Hi Emmanuel,

I just checked and it is working from two of my computers. You need to have the latest version of java installed to get the calculator to work in your browser. If you already have Java, try to uninstall it and reinstall it. That has worked for others who have had the same problem. If worse comes to worse, you might have to use the calculator from a different computer.

Justus

June 25, 2010 at 6:16 pm

Gina

Hi Justus,

i came to your online calculator as i have trouble doing Kappa in SPSS and need a calculator that can digest my dataset. Most online calculators only do 8 or 12 categories but not more.
I have a dataset with 2 coders, 34 categories, and 135 cases (which will grow up about 400 cases for each kappa i need to calculate); and as i have 6 2-way combinations of 4 raters i will have to compute 6 different kappas.
Of course, so many categories mean i get a lot of zero’s in the category-by-category square, which might cause my SPSS not to calculate it. If i enter all of these data into your calculator (format is different than the SPSS output i get, so i would have to do it all by hand, no C&P) do you think your calculator will be able to digest this? Or do you have any other recommendations how i could best do this?

Thank you!

Gina

February 27, 2012 at 6:20 pm

Justus

Hi Gina,

I don’t think that I saw this before because it’s in the about section.

I think that the OKC will work for you. I’d put the data in a spreadsheet first and try to cut and paste. You don’t want to input all of that data by hand in case there’s an error!

February 27, 2012 at 1:07 am

Markku Paanalahti

HI!
I started to wonder if you or somebody else using this website could help me with the followning question.
I have conducted face to face interviews and from the transcribed text two independent researchers have identified meaning units that were linked to ICF (international classification of functioning disability and health).No I shoud calculate the kappa statistics on inter rater agreement. The problem is that the ICF core set that we have used has 130 categories. So everytime I choose a category I choose among 130 categories as well as the other rater.
Is it possible to calculate Kappa statistics of this?

February 27, 2012 at 6:32 pm

Justus

Markku,

That’s an interesting question. Theoretically, kappa would work in this case, but you’d have to be careful with the interpretation. I’ve never seen kappa with so many categories. You can inflate free-marginal kappa by adding superflous categories that aren’t theoretically defensible. With 130 categories the probability of change agreement is .008. However, the probability of chance agreement wouldn’t actually be .008 if many of the 130 categories are “throw away” categories that you or the other raters wouldn’t really pick. My suggestions is to only use as many categories as a theoretically justifiable . . . all of your 130 cats might be theoretically justifiable and you’d just have to make that argument. You might report the fixed marginal kappa in this case (or both).

With that said, I’ve never tried the OKC with 130 cats. Let me know what happens.

February 27, 2012 at 8:44 pm

Markku Paanalahti

Thank you Justus!

I continue to study the problem I have to confront in the near future,

Best Regards

Markku

July 13, 2020 at 9:51 am