perClass Documentation
version 5.1 (31-May-2017)

Chapter 7: Image visualization

Table of contents

7.1. Introduction ↩

perClass provides a set of tools for working with image data. It allows us to visualize gray-level or multi-band images, compute local image features and identify connected components. Starting with perClass 4, this functionality is available as "Imaging option" (see sdversion for available options).

7.2. Visualizing images with sdimage ↩

Let us consider an RGB image of a traffic scene 'roadsign09.bmp', loaded with Matlab imread command:

>> im = imread('roadsign09.bmp');
>> figure; imagesc(im)

Road sign image

Using sdimage command on matrix im opens an interactive viewer:

>> sdimage(im);

Interactive image plot.

The blue layer on top of the image represents the set of labels of the image data set, internally used by sdimage. As any other sddata set, each sample (pixel) has a label, which is set to "unknown" by default. We may toggle this label layer using the space bar key. Additionally, we may also adjust label transparency from very transparent to opaque in the Image menu.

The three image channels are visualized as three separate image bands. We can move between the bands with the 'up' and 'down' cursor keys or the green arrows icons in the menu'. Each pixel is a data sample, the figure title shows the pixel's value and class label ('unknown' by default). sdimage loads the image if provided with the string filename:

>> sdimage('roadsign09.bmp')

7.3. Creating image data sets objects directly ↩

sdimage command allows us to create image data set directly on the Matlab prompt using 'sddata' option:

>> a=sdimage(im,'sddata')
412160 by 3 sddata, class: 'unknown'

sdimage also accepts the image filename, attempting to load the file using imread:

>> b=sdimage('roadsign11.bmp','sddata')
412160 by 3 sddata, class: 'unknown'

The objects a and b are standard sddata sets with one sample for each pixel and three features corresponding to R,G, and B bands respectively. Note, that the pixel values were also converted into double precision.

7.4. Hand-painting class labels ↩

The interactive sdimage figure allows us to paint class labels for image regions. In order to enter the 'paint' mode, use the Paint menu in the Image menu, select the Create new class command:

Painting pixel labels in an image plot

A dialog window will ask for the name of the class. Let's say we are interested in labeling the road, we provide the name of the class and paint in the image region. Via the Image menu, or by clicking the right mouse button we can change brush size or exit from the paint mode.

Hand-painted image labels

7.5. Cropping images ↩

Often, we only want to work with a smaller area of a large image. sdimage offers us a crop function which makes this very quick.

Select the Crop image item in the Image menu.

Cropping multi-band images with perClass

A cross-hair will appear. Choose two corners of a region you wish to crop. The process may be terminated by clicking right mouse button.

Cropped version of the image data set

The new sdimage figure is opened containing the data from the specified region. Cropped data contains all labels and properties of the complete image. The image size for the new data set will be set to the specified region. If you save the cropped image into data set c:

>> Creating data set c in the workspace.
12840 by 3 sddata, class: 'unknown'
>> getiminfo(a)

ans = 

imsize: [560 736]

>> getiminfo(c)

ans = 

imsize: [105 202]

7.6. Saving image data set to workspace ↩

The Create data set in workspace command in Image menu lets us to store the image data together with the painted labels in a new sddata object in the Matlab workspace. We are asked to provide the variable name for this new data set.

Storing our image data set with the labeled road region in data2 variable, we will see the following message in Matlab command window:

>> Creating data set data2 in the workspace.
412160 by 3 sddata, 2 classes: 'unknown'(402557) 'road'(9603) 

Alternatively, we may access the data set in any open open image figure with 'getdata' option, providing the figure handle:

>> sdimage(12,'getdata')
412160 by 3 sddata, 2 classes: 'unknown'(402454) 'road'(9706) 

7.7. Working with image subsets ↩

The image data sets preserve the image information. We may, for example, use only a subset of data, e.g. the pixels labeled as 'road' in the data2 object above.

>> sub=data2(:,:,'road')
9603 by 3 sddata, class: 'road'

The image subset may be still visualized as an image:

>> sdimage(sub)

sdimage showing image subset

7.8. Creating image matrix from a data set ↩

The data set representation of image data is useful for training pattern recognition algorithms. However, often we may need to apply imaging operations, such as filtering, to our image regions. sdimage allows us to create an image matrix with pixel values using the matrix option:

>> sub
9159 by 3 sddata, class: 'road'

>> I=sdimage(sub,'matrix');

>> size(I)

ans =

   560   736     3

Matrix I is created with the size of the original image the sub data was extracted from. The matrix is filled with zeros and only the pixels available in the sub data set are inserted into this matrix.

Note that the matrix I uses double precision:

>> class(I)

ans =

double

We may now perform any image processing operation such as filtering, and bring the resulting data back into a data set format. This can be done using the linear indices stored in the sub data set 'pixel' property:

>> sub2=setdata(sub, I(sub.pixel))
9159 by 1 sddata, class: 'road'

7.9. Storing multiple images in data sets ↩

Image data sets created from multiple images may be joined. This feature allows us to create larger training sets with pixel-level data from multiple images and train robust classifiers.

Each image data set, created using sdimage, contains 'image' property (labels). If the image is loaded by providing the filename, this will be used as its image label.

>> im1=sdimage('roadsign09.bmp','sddata')
412160 by 3 sddata, class: 'unknown'
>> im2=sdimage('roadsign11.bmp','sddata')
412160 by 3 sddata, class: 'unknown'

>> a=[im1; im2]
824320 by 3 sddata, class: 'unknown'

>> a'
824320 by 3 sddata, class: 'unknown'
sample props: 'lab'->'class' 'class'(L) 'pixel'(N) 'image'(L)
feature props: 'featlab'->'featname' 'featname'(L)
data props:  'data'(N)

>> a.image
sdlab with 824320 entries, 2 groups: 'roadsign09.bmp'(412160) 'roadsign11.bmp'(412160) 

If we create an image from a matrix, sdimage creates random image label to avoid name clash with other images.

>> im1=sdimage(imread('roadsign09.bmp'),'sddata')
412160 by 3 sddata, class: 'unknown'
>> im2=~sdimage`(imread('roadsign11.bmp'),'sddata')
412160 by 3 sddata, class: 'unknown'

>> a=[im1; im2]
824320 by 3 sddata, class: 'unknown'
>> a.image
sdlab with 824320 entries, 2 groups: 'image9552'(412160) 'image6571'(412160) 

Note, that the image name is generated randomly and no check for identical names when concatenating image data sets is performed. It is the responsibility of the user to make sure that different images in one data set are labeled differently.

7.10. Connecting sdimage and sdscatter ↩

It is often useful to inspect the connection between image neighborhoods and the scatter plot. In order to visualize this connection the sdscatter and sdimage commands can be used together.

We may simply show data set with image data using sdscatter and then connect the sdimage plot to the scatter figure using the returned figure handle.

>> data2    %  Created data set data2 in the workspace.
412160 by 3 sddata, 2 classes: 'unknown'(399848) 'road'(12312) 

>> h=sdscatter(data2)
h =
     2

>> sdimage(data2,h); 

Connected image and scatter plot.

By moving the mouse pointer over the image, we see where the image pixel appears in the feature space. Similarly, moving over the scatter plot shows us the corresponding pixel in the image.

By painting the in the scatter plot, the linked image plot also updates. This helps us to analyze position of specific feature space clusters in image domain:

Painting sample labels and visualizing the result in an image.

7.11. Clustering image with k-means ↩

One way to quickly group image data is to perform clustering. Using the Cluster with k-means command in Image menu, the data set underlying our image is clustered. The algorithm considers individual pixels as separate data samples and image bands as features.

We are prompted for the desired number of clusters.

Clustering image - selecting number of clusters

We will obtain a new set of image labels called 'cluster' containing classes called 'C1','C2' etc.

Clustering image using k-means

Typical next step is to interpret the clusters. This may be done by assigning meaningful names using the Rename class command.

7.12. Defining connected components ↩

sdimage allows us to define spatially-connected components. This allows us to quickly access individual objects or regions in an image data set. The Connected components menu is available only if the current set of labels contains two or more classes.

Defining connected components

Connected component command processes the current set of labels. For each class, the connected components are found separately.

Connected components with sdimage

Small isolated components are joined together into a special class (called 'small objects'). This helps us to quickly remove the noise. By default, objects smaller than 10 pixels are removed. This can be changed by the first item in the Connected components menu. In order to separate all isolated objects, use the value of 1.

When we save the data set back to the Matlab workspace (pressing 's' key), we can see the 'object' labels. Note that, because we saved the image data when the 'object' label selected, the resulting data set keeps it as a current label set. Therefore, we may address it as data2.lab

>> Creating data set data2 in the workspace.
12210 by 3 sddata, 11 'object' groups: [242   160    66  2643   114  6481    72     4  2346    62    20]
>> data2.lab'
 ind name                size percentage
   1 C1-object1           242 ( 2.1%)
   2 C1-object2            10 ( 0.1%)
   3 C1-object3           298 ( 2.6%)
   4 C1-object4            20 ( 0.2%)
   5 C1-object5            66 ( 0.6%)
   6 C1-object6          3805 (33.3%)
   7 C1-small objects      66 ( 0.6%)
   8 C2-object1          4332 (38.0%)
   9 C2-object2            72 ( 0.6%)
  10 C2-small objects       4 ( 0.0%)
  11 C3-object1          2452 (21.5%)
  12 C3-object2            18 ( 0.2%)
  13 C3-object3            14 ( 0.1%)
  14 C3-small objects      14 ( 0.1%)

We can remove the small objects quickly with a regular expression. We simply select all classes, that do not contain the 'small' substring:

>> data2(:,:,'~/small').lab'
 ind name                size percentage
   1 C1-object1           242 ( 2.1%)
   2 C1-object2            10 ( 0.1%)
   3 C1-object3           298 ( 2.6%)
   4 C1-object4            20 ( 0.2%)
   5 C1-object5            66 ( 0.6%)
   6 C1-object6          3805 (33.6%)
   7 C2-object1          4332 (38.2%)
   8 C2-object2            72 ( 0.6%)
   9 C3-object1          2452 (21.6%)
  10 C3-object2            18 ( 0.2%)
  11 C3-object3            14 ( 0.1%)

To define connected components programatically, use the sdsegment command:

>> s=sdsegment(data2,'minsize',10)