Hi Bernhard,
in PRSD Studio, prior probabilities are part of a classifier or (indirectly) of an operating point, not of the data set itself. Philosophically, we believe that that’s where these assumptions belong. I may have the same data set but different problems/labels and thus different assumptions on priors.
When you are training a classifier that makes use of priors, such as the normal-based models (sdlinear, sdquadratic, sdmixture), you may specify your assumed priors using the ‘priors’ option:
>> a
'Fruit set' 260 by 2 sddata, 3 classes: 'apple'(100) 'banana'(100) 'stone'(60)
>> p=sdlinear(a,'priors',[0.8 0.1 0.1])
sequential pipeline 2x3 'Linear discriminant'
1 Gauss eq.cov. 2x3 3 classes, 3 components (sdp_normal)
2 Output normalization 3x3 (sdp_norm)
In my opinion, ROC analysis relieves you from assuming the priors. When making the decisions, we weight the model soft outputs with some class weights. Using ROC analysis, we identify the weights needed in order to reach the desirable performance on the test set, irrespective of the prior assumption we put in the model. If our assumptions change, the ROC-estimated class weights will of course change as well to provide desired performance. The beauty is we don’t need to care. We only need to assure that the test set is realistic for estimating reliable class weights. The training set may be skewed in any direction or, more practically, be balanced so we estimate as good parameters of our model as possible.
Does it help with your problem?
Pavel
