Hi,

I want to compare two classifiers ( C1 and C2). I read Salzberg paper and I undestood that he suggests that we divided the data in k fold, use crossvalidation and for each fold, compute the McNemar test. In a course notes that I picked up in the internet it was said that the McNemar test can only be used if n10 + n01 > 10 ( n10 - c1 is wrong and c2 is right and n10 - c1 is right and c2 is wrong), is this true?

The other approach would be using a k-Fold crossvalidation Paired t - Test.  The avaible data is 90 cases, so if I use 10-fold crossvalidation , each fold has only nine cases. Is it better to use five fold then? If so the Student distribution that I have to use if for k-1= 4 degrees of freedom?

Thanks,

Jorge

If 90 cases are classified by two classifiers in a k-fold crossvalidation, and if these crossvalidations are aligned (same training and test objects for the two classifiers in every fold) then the tests can be applied on all 90 classification results.

Bob Duin