Username Remember Me?
Password   forgot password?
   
   
More detailed explanation of sdbands() used with ‘LDA’ (LDA spectral feature extractor)
Posted: 12 February 2016 10:34 AM   [ Ignore ]  
Novice
Rank
Total Posts:  6
Joined  2014-04-16

Hi Pavel,

It is still not very clear to me how the function sdbands() with ‘LDA’ prop works. Also, I don’t see what the ‘cluster’ value should be.

I used this and compared the results with the sdlda() trained on the same dataset, and it gives something really different. By using sdbands() with ‘LDA’ and 4 clusters, I go from 19 features (reflectance data) to 16 features. Could you please explain what those new features represent?

NB: Just for the illustration, you can see images attached (Center lines are median & average by class of “spectra” values with the new features, and top and bottom of the polygons are percentiles 75 & 25).

Edit: Attachment doesn’t work. Here are some temporary links:
http://139.165.125.214/~PBarnabe/Plot3D_sdbands_LDA_Cluster.png
http://139.165.125.214/~PBarnabe/Plot3D_LDA_Values.png

Thanks!

Pierre

[ Edited: 12 February 2016 10:41 AM by Pierre]
Profile
 
 
Posted: 12 February 2016 10:54 AM   [ Ignore ]   [ # 1 ]  
Administrator
Avatar
RankRankRankRank
Total Posts:  365
Joined  2008-04-26

Hi Pierre,

LDA extractor computes LDA projection for each new band. Because you have five classes, the LDA projection results in 4D subspace (num.of classes - 1). As you’ve defined 4 bands by clustering, you’re getting 4x4 = 16 outputs features. LDA maximizes separation of your classes in each band. The resulting values are linear combinations of inputs (wavelengths in each band) and, therefore, may be more difficult to interpret than the standard ‘mean’ band extractor that only averages the per-wavelength values of each band.

Does it help?

Kind Regards,

Pavel

Profile
 
 
Posted: 12 February 2016 10:59 AM   [ Ignore ]   [ # 2 ]  
Administrator
Avatar
RankRankRankRank
Total Posts:  365
Joined  2008-04-26

The attachments did not work because of 800x600 pixel size limit. We have increased that to 1200x1000 pixels.
I’m now attaching the images you linked for future reference.

Image Attachments
Plot3D_sdbands_LDA_Cluster.pngPlot3D_LDA_Values.png
Profile
 
 
Posted: 12 February 2016 11:58 AM   [ Ignore ]   [ # 3 ]  
Novice
Rank
Total Posts:  6
Joined  2014-04-16

Wow, that was quick!

Thanks Pavel, it helps a bit, yes.

So if I understand well, the algorithm will try to separate the 19 bands in 4 (equal or unequal?) new clusters, but this time, only by looking at the LDA obtained for those bands? Also, for each cluster, you get 4 values, which correspond the the LDA of those bands.

- Just to be sure : the clusters only regroup connected (neighbor) bands?
- What is the criteria used to choose those 4 new clusters?
- Can a cluster include much more bands than another?
- Is there a possibility to see to which initial bands a cluster corresponds?

...and finally, as LDA transformation is already a linear combination of all bands, in which case do you think there could be a benefit to use this instead of directly using sdlda() ? When you need to emphasize local strategic variations of the reflectance?

Also, after what I saw in different tests, the last (4th) LDA feature of each cluster doesn’t seem to contain any valuable information. Would it be wiser not to consider it during the classification?

I hope this is clear… Sorry if I ask too many questions :-)

Profile
 
 
Posted: 12 February 2016 12:49 PM   [ Ignore ]   [ # 4 ]  
Administrator
Avatar
RankRankRankRank
Total Posts:  365
Joined  2008-04-26

So if I understand well, the algorithm will try to separate the 19 bands in 4 (equal or unequal?) new clusters, but this time, only by looking at the LDA obtained for those bands? Also, for each cluster, you get 4 values, which correspond the the LDA of those bands.

yes

- Just to be sure : the clusters only regroup connected (neighbor) bands?

yes, sdbands clustering defines groups of adjacent wavelengths.

- What is the criteria used to choose those 4 new clusters?

k-means clustering considering similarity between wavelengths

- Can a cluster include much more bands than another?

yes

- Is there a possibility to see to which initial bands a cluster corresponds?

yes, sdbands(p) - check out the documentation: http://perclass.com/doc/guide/feature_extraction/spectra.html#ex:print

...and finally, as LDA transformation is already a linear combination of all bands, in which case do you think there could be a benefit to use this instead of directly using sdlda() ? When you need to emphasize local strategic variations of the reflectance?

computing LDA projection only for specific set of wavelengths effectively combines (feature) selection with LDA extraction. If you use LDA on complete spectra, each input is a linear combination of all wavelengths. Using LDA only within the bands makes each LDA simpler (i.e. less suffering from over-fitting if you only have a limited number of samples to learn from)

In my opinion, for some problems only certain parts of spectra are informative to separate classes of interest. Others are not and including them is just not necessary.

Also, after what I saw in different tests, the last (4th) LDA feature of each cluster doesn’t seem to contain any valuable information. Would it be wiser not to consider it during the classification?

Depends :-), To answer this, you can compare classifier results when using all LDA-extracted bands or when running a feature selection in the final feature space and using only a subset. Depending on your available sample size, this additional optimization step that requires further data splitting, may be too much to do robustly.

I hope this is clear… Sorry if I ask too many questions :-)

No problem, Good Luck!

[ Edited: 12 February 2016 12:52 PM by pavel]
Profile
 
 
Posted: 12 February 2016 02:54 PM   [ Ignore ]   [ # 5 ]  
Novice
Rank
Total Posts:  6
Joined  2014-04-16

Thank you very much Pavel.

Have a nice week-end!

Profile