What activation function is used for bpxnc?  Is it a step function, sigmoid, tanh, etc…

It uses ‘logsig’. See ffnc line 110.

Bob Duin

Thanks Bob.  One more question, if I write

neural=bpxnc(training_data,48);

Then I understand that this creates a neural network with 48 hidden nodes, but what does the actual configuration look like assuming I have 24 feature inputs and 60 output classes?

Would it be:
Input Layer | Hidden Layer | Output Layer

(1 Neuron per input) | 48 Hidden | 1 Neuron per output

And is each input connected to each hidden connected to each output?

Yes, exactly as you expected: 24 x 48 x 60.
So there are two layers of weigths 24 x 48 and 48 x 60.

Bob Duin

Plus the input weights?

So overall network would be 24 x 48 x 60, on each input line is the weight (w) and on each neuron is every input permutation with the bias input?

Every linear combination of weights as seen by the hidden layer and the output neuros, has a bias.

Bob Duin

Sorry, probably not being clear in my question.  In the above scenario we have a 3-layer neural network.  An input layer of 24 neurons, a hidden layer of 48 neurons, and an output layer of 60 neurons.  Each input has a path to each of the first layer of neurons (24x24 paths) with an associated weight.  Each neuron on this layer has an additional bias input.

Along the lines of http://www.mathworks.nl/access/helpdesk/help/toolbox/nnet/neuron_3.html#65 - I just want to confirm that this is how bpxnc is implementing it also as I want to show this in a diagram.

No, no. That is a network with two hidden layers.

By

bpxnc(training_data,48);

a network of 24 x 48 x 60 is created. By

bpxnc(training_data,[24 48]);

a network of 24 x 24 x 48 x 60 is created.

You can inspect the architecture in Mathwork terms as follows:

a = gendatb;
w = bpxnc(a,3);

w.data{2}.data{1}

Now the Neural Network object of the Matlab Neural Network toolbox is displayed.

Best regards,

Bob Duin

Hi Bob, I really appreciate your efforts, but I’m getting more and more confused here.  Would you mind running through this piece by piece:

1.  bpxnc(training_data,48); - This produces a 24 x 48 x 60 network.

2.  Input Layer consists of 24 Neurons (in line with the 24 input features from original post).  Each neuron has a bias input and takes a weighted input from each vector.  ie. if input vector is [A B C D… X] then neuron 1 takes [A*wA] [B*wB] [C*wC] [D*wD]… [X*wX] and [BIAS].  Neuron 2 takes the same etc.

3.  Hidden Layer consists of 48 neurons.  Each neuron takes a weighted input from each of the input layer neurons and a bias.

4.  Output layer consists of 60 neurons (in line with 60 classes).  Each neuron takes a weighted input from each of the hidden layer neurons and a bias.

If this isn’t correct, is there a simple way to visualise the generated network using nntool or something?

1. The terminology of a network with one hidden layer means that there is between input (the given features) and outputs (the class assignments) a single layer of neurons.

2. The input ‘neurons’ (not our terminology) directly observe the world. They just see the feature values. These ARE the feature values if you like. Weights are between the neurons (input layer and hidden layer, hidden layer and output layer). As you may observe in w.data, there is an affine transform w.data{1} that takes care of input scaling. This is not trained during the optimisation of the network. It is a scaling directly computed from the training set.

3. yes, this is the first layer of weights.

4. yes, correct.

May be there are visualisation tools somewhere in the Neural Network toolbox. Please realize the PRTools just offers an interface with this toolbox. It is programmed 15 years ago and revised about 10 years ago. After that it is used, but not revised w.r.t. updates offered by Mathwork.

I have to stop the discussion here.

Bob Duin

Thanks Bob, think I have it now.  There’s some terminology differences in our fields but appreciate you taking the time to explain it.