perClass

Leave one subject out

2011-10-28T22:41:10Z

Hi dear forum members,
I am trying to use loso function in the PRTools but I think I have been doing something wrong. I have 7 classes and I wrote that code;

prior = [1/7 1/7 1/7 1/7 1/7 1/7 1/7];
A = dataset(training_features, class_labels, 'prior', prior);
W = svc([], 'p', 2);

[E C D] = loso(A, W, class_nos);
confmat(D)

Is that something wrong here?
ANy response will be appreciated.
Thanks a lot.

training SVDD with negative examples (incsvdd)

2011-03-24T21:05:56Z

Hello!

I have a question about training SVDD with negative examples in “incsvdd”; the negatives examples appears in the objective function of the optimizacion problem?, or just appears like restriction.

I have this doubt because when I utilize oversampling on the negative examples, I get the same performance. Therefore, I think that the negatives examples just appears like restriction,, but really I’m not sure.

Could somebody help me?,
Claudio

Combining featrures

2011-05-14T20:11:27Z

Hi All
I have a radar signal classification problem, where the extracted features are both images (time-frequency distributions) and vectors (statistical features). There is one image feature and three vector features for each object. I want to combine these features for using them in a single classifier. My intuition tells me that I just have to vectorize the images and then concatenate the resulting vector with the rest vector features (for the same object), thus creating a large vector feature for each object. However, I am afraid that combining the features there might be problems in the classification since the feature sets were generated using totally different procedures. Is there any idea for a better way of doing this? Thanks

Question about SVM formulation in ‘Classification, Parameter Estimation and State Estimation’

2011-03-18T22:22:39Z

Hello,

I am currently working on my methods section for a paper and I want to show the development of the SVM classifier. I tried finding the dual form of Equation 5.54 by hand and I came up with sign disagreements when compared to 5.56. My relationships derived from the partial derivatives of L came out to be the same as 5.55. This derivation starts on page 169 for reference.

L = 1/2*||w||^2 + SUM an (cn [w^T*zn + b] - 1) (Eq 5.54)
w = SUM an*cn*zn
SUM cn*an = 0 (Eq 5.55)

Combining the three relationships,

L = 1/2*(SUM an*cn*zn)*(SUM am*cm*zm) - SUM an

In the book, L seems to be multiplied through by a negative sign, but I am not sure why. Did I do something wrong?

Also, I tried to find the dual formulation with the slack variables included and was wondering if it was correct (exercise 6 of chapter 5).

L = 1/2*w^2 + C*SUM En + SUM an (cn [w^T*zn + b] - 1 + En) + SUM vn*En (5.61)
Partial derivative relationships
dL/dw: w = SUM an*cn*zn
dL/db: SUM cn*an = 0
dL/dC: SUM En = 0

L = 1/2*(SUM an*cn*zn)*(SUM am*cm*zm) - SUM an + SUM vn*En

The reason I ask is I am not sure how this would be solved as L must be maximized with respect to an but I don’t think En or vn is known yet. Am I missing something?

Thank you in advance,

Thomas

fracrej and C in SVDD in dd_Tools

2010-11-04T16:19:05Z

Hello forum,

I am using the dd_tools and especially the SVDD for classifying hyperspectral images. I am a user coming from the application side (without a deeper understanding in pattern recognition) and have some question regarding the theoretical background of the SVDD and the implementation in dd_tools.
If I understood right, the parameter ‘fracrej’ is mainly influential in the determination of the parameter C while sigma for the flexibility of the hypersphere. I would like to understand how fracrej and C are related but also reading the dd_tools manual and the dissertation of Tax did not help. Does anybode have a simple explanation? I also wonder how the second regularization parameter C2 is optimized when I train the classifier with targets and outliers because I only define one fracrej for the target class.

A nice explanation for the relation between C and fracrej would be highly acnowledged (-:

Thanx in advance,
ben

Is it correct to calculate the average confusion matrix in kfolds?

2010-03-15T13:19:54Z

Hi,

I want to compare the performance of two classifiers. Is it correct to divide the data in k-folds and calculate the confusion matrix for each fold and then average the K confusion matrixs? Is this approach valid for the area under the roc curve?

If I am calculating the sensitivity and specificity, Is this approach is also valid?

Thanks,

Jorge

Another question about performance estimation in an umbalanced classification problem

2010-06-02T20:18:58Z

Anybody could tell me the correct way of evaluating the performance in a multi-class classification problem with umbalanced or imbalanced class presence ?

For example in this 3 class problem, I am used to have the following performance measures derived from the confusion matrix:

(this example is attached in results.txt, I dont know why the formatting is so ugly)

True | Estimated Labels
Labels | Normal Suprav Ventri| Totals
-----------------|---------------------|-------
Normal |282094 1534 3991 |287619
Supraventricular| 102 854 378 | 1334
Ventricular | 521 1309 742 | 2572
-----------------|---------------------|-------
Totals |282717 3697 5111 |291525

Balanced Results
-----------------
| Normal || Supravent || Ventricul || TOTALS |
| Se +P || Se +P || Se +P || Acc | Se | +P |
| 98% 78% || 64% 55% || 29% 49% || 64% | 64% | 61% |

Unbalanced Results
-------------------
| Normal || Supravent || Ventricul || TOTALS |
| Se +P || Se +P || Se +P || Acc | Se | +P |
| 98% 100% || 64% 23% || 29% 15% || 97% | 64% | 46% |

(this example is attached in results.txt, I dont know why the formatting is so ugly)

As you can see, the presence of the 3 classes is VERY umbalanced (287619, 1334, 2572), and consequently I noticed that the positive predictive value (+P) is biased by this umbalance. Trying to avoid this, I calculated the “balanced” results, which means scaling all rows in the confusion matrix to sum the same (or balance the class presence), and then calculate the performance. As can be noted, the class sensitivity (Se) is not affected by this row-operations. The totals means the average Se and +P of all classes, and Acc is the trace of the confusion matrix divided by the total amount of examples (291525).

Thanks in advance for any comment.
Mariano.

Classifier performance estimation and data division.

2010-04-28T10:27:58Z

I have the following doubt regarding to some pattern recognition experiments which I am doing. First of all, I am interested in doing a feature selection step, to obtain a classification model optimum in some sense, and finally estimating its performance. The final idea is porting one trained model to the “real world” to perform some classification task. Suppose I have a decent amount of data to perform a three-way partition (train-validation-test) or any other approach. My question is, which is the most correct approach when data is not a limitation ?

One idea could be dividing the dataset in three (train-validation-test). Then perform feature selection in train, estimating the criterion to maximize via crossvalidation. Then adjusting the operating point in the validation set, and finally estimating performance in the test set. Both last steps also crosssvalidating. The question of this approach is, when you need to port a trained model to the real world, where are you supposed to train this model ? In the whole data ?

If you do the same, but avoiding crossvalidation. I mean, always train in the training set, evaluate in the validation set for feature selection and operating point determination, and te selected model, trained in the train set, and evaluated in the test set gives the final performance estimation. With this approach, it is clear that the model to port to reality is the only one I obtained from the train-validation sets ... BUT should I try different train-validation partitions previous to the final performance estimation ?

Thanks in advance for any help or comments,
Mariano.

Final training for a neural netwrok after cross validation

2010-05-03T20:10:30Z

Hi,

I know that is possible to use cross validation to determine the number of hidden neurons. My question is: after I decided how many hidden neurons I use , what is the training and tuning set that i have to use. In other types of classifiers such as nearest neighbors, I just use the whole dataset, but I cannot do this with neural networks because it would cause overfitting. Should I use the best network in the crossvalidation? That is the one with the smallest validation error?

Thanks,

Jorge

Interpreting figures

2010-04-11T01:08:34Z

Hi,
I will be straight to the point. I am training a mixture of gaussians to estimate a persons age from their speech. I have 168 objects with 13 features in each, 4 classes of data. I trained the gaussians using gaussm and mogc and I used plotm. I know that gaussm trains one gaussian per calss of data object while mogc computes one over all gaussian density.
I provided prior probabilities when I created my dataset object and set the labtype to ‘soft’ in order to use the expectation maximisation algorithm. The following is my code:
A = dataset(inputs,labels’);
A = setlabtype(A,’soft’);
A = setprior(A, [0.16666666666667 0.22619047619048 0.33333333333333 0.27380952380952]);
W1 = mogc(A,4,0,0.8);
figure(2);
scatterd(A,[10,5]);
plotm(W1,6,10);

and attached is the resulting plot using plotm as shown.
Can anyone please explain to me how to interpret the plot?
What does “feature 1” and “feature 2” mean?

Kind regards,
Faiyo.