SDCROSSVAL Perform cross-validation S=SDCROSSVAL(ALG,DATA,options) [S,RES,E]=SDCROSSVAL(ALG,DATA,options) INPUTS ALG untrained algorithm or pipeline returning decisions DATA dataset to perform cross-validation on OPTIONS 'method' cross-val method: rotation (def),randomization,leave-one-out 'folds' number of folds to perform (default: 10) 'seed' random seed (default: no seed set) 'ops' set of operating points to estimate ROC with variances OUTPUT S String summarizing the results (mean/std for each perf.measure) RES Structure with estimated performances per fold E Evaluation object storing per-fold trained algorithms DESCRIPTION SDCROSSVAL performs N-fold cross-validation of untrained algorithm, pipeline or mapping ALG. Rotation, randomization and leave-one-out schemes are supported. In the default rotation mode, the DATA is split into N folds (splitting each class separately). Repeatedly, N-1 subsets are used to train ALG and the performance is estimated on the Nth subset. Randomization splits DATA using RANDSUBSET method. By default 50% of samples are used for training, the rest for testing. The numerical value following the 'random' option is passed to RANDSUBSET. The leave-one-out is run over samples by default but may be executed on all unique categories of a sample property using the 'over' option. This allows us to cross-validate algorithm over patients or objects. If a set of operating point is provided using the 'ops' option, SDCROSSVAL estimates ROC with variances at these op.points. ALG must return soft output in this case. EXAMPLES Rotation over 20 folds S=SDCROSSVAL(ALG,DATA,'folds',20) Randomization, use 80% of DATA for training S=SDCROSSVAL(ALG,DATA,'method','rand',0.8) Randomization, use 100 samples per class for training S=SDCROSSVAL(ALG,DATA,'method','rand',100) Leave-one-out S=SDCROSSVAL(ALG,DATA,'method','loo') Leave-one-out over patients labels S=SDCROSSVAL(ALG,DATA,'method','loo','over','patient') READ MORE http://perclass.com/doc/guide/evaluation.html#intro
sdcrossval is referenced in examples:
