Artificial neural networks in Scilab

Currently, the Scilab’s ANN toolbox doesn’t have much. Only feed-forward networks and only some basic local search algorithms.

So there’s plenty of work for open source enthusiasts.

It would be nice to add other types of networks and global search algorithms.

The help files for the toolbox don’t have any examples.

Searching the Internet, I found an example in one blog post written by this Filipino girl.

Here I give one more example on using multi-layer perceptron for simple character recognition.

This is not a full-featured character recognition application.

Rather it is just an educational toy to get an idea how to use the ANN toolbox in Scilab.

Our neural network will recognize two letters: T and U.

```T = [...
1 1 1 1 1 ...
0 0 1 0 0 ...
0 0 1 0 0 ...
0 0 1 0 0 ...
0 0 1 0 0 ...
0 0 1 0 0 ...
0 0 1 0 0 ...
]';
```

```U = [...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
0 1 1 1 0 ...
]';
```

These pictures are screenshots of *.pgm files opened with ImageJ.

The contents of the *.pgm files are

T.pgm

```P2
5 7
1
1 1 1 1 1
0 0 1 0 0
0 0 1 0 0
0 0 1 0 0
0 0 1 0 0
0 0 1 0 0
0 0 1 0 0
```

U.pgm

```P2
5 7
1
1 0 0 0 1
1 0 0 0 1
1 0 0 0 1
1 0 0 0 1
1 0 0 0 1
1 0 0 0 1
0 1 1 1 0
```

To recognize these letters, we built a perceptron with three layers.

The input layer has 35 neurons – one neuron for each pixel in the picture.

The output layer has two neurons – one neuron per class in our classification task (one for “T”, another one for “U”).

The hidden layer has 10 neurons.

The numbers of neurons in all layers are collected in a row vector

```N = [35 10 2];
```

Then, the weights of the connections between the neurons are initialized

```W = ann_FF_init(N);
```

Our training set consists of the two images above

```x = [T, U];
```

Now let’s specify the desired responses.

If the input layer reads “T” then the first output neuron should give 1 and the second output neuron should give 0

```t_t = [1 0]';
```

If the input layer reads “U” then the first output neuron should give 0 and the second output neuron should give 1

```t_u = [0 1]';
```

To be used in Scilab ANN routines, the responses need to be combined in a matrix that has the same amount of columns as the training set

```t = [t_t, t_u];
```

Other training parameters are the learning rate and the threshold of error

```lp = [0.01, 1e-4];
```

The number of training cycles:

```epochs = 3000;
```

The network is trained with the standard backpropagation in batch mode:

```W = ann_FF_Std_batch(x,t,N,W,lp,epochs);
```

`W` is the matrix of weights after the training.

Let’s test it:

```y = ann_FF_run(x,N,W)
disp(y)
```

It gave me:

```    0.9007601    0.1166854
0.0954540    0.8970468
```

Looks fine.

Now, let’s feed the network with slightly distorted inputs and see if it recognises them

```T_odd = [...
1 1 1 1 1 ...
0 0 1 0 1 ...
0 0 1 0 1 ...
0 0 1 0 0 ...
0 0 1 0 0 ...
0 0 1 0 0 ...
0 0 1 0 0 ...
]';
```

```U_odd = [...
1 1 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
0 1 1 1 1 ...
]';
```

Also, let’s check something that is neither “T” nor “U”

```M = [...
1 0 0 0 1 ...
1 1 0 1 1 ...
1 0 1 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
]';
```

The output of the network is calculated by inserting each of these vectors into `ann_FF_run` instead of the training vector `x` For example,

```y = ann_FF_run(T_odd,N,W);
disp(y)
```

Then I write the code that displays

• “Looks like T” if the network recognizes “T”
• “Looks like U” if the network recognizes “U”
• “I can’t recognise it” if the letter is not recognised by the network
```if (y(1)> 0.8 & y(2) < 0.2)
disp('Looks like T')
elseif (y(2)> 0.8 & y(1) < 0.2)
disp('Looks like U')
else
disp('I can''t recognise it')
end
```

Feeding `T_odd` gave me the output:

```    0.8798083
0.1333073

Looks like T
```

Feeding `U_odd` produces the output:

```    0.1351676
0.8113366

Looks like U
```

The letter `M` was not recognized:

```    0.2242181
0.8501858

I can't recognise it
```
Published in: on 12/03/2014 at 20:37  Comments (11)
Tags: , , ,

Hello. Can you explain why you select 10 neurons in the hidden layer?. Are there some criteria to select it according to the number of inputs or something like that?. Thank you so much.

• It doesn’t depend on the number of inputs or outputs.

Neurons in the hidden layers detect features of the modeled system. If you use your neural network to model some complicated system with complex behavior influenced by many different factors, then you need many hidden neurons (maybe hundreds of thousands) and more than one hidden layer (but most often two layers are enough). If you use less hidden neurons than required, then your solution will show a kind of an averaged behavior of your real system, where some features are averaged out (or not noticed by your neural network). Here, it is just a simple toy example, so one hidden layer with ten neurons seems to be enough.

2. Hello,

what do values 0.8 and 0.2 represent in here?

Thank you very much

• Hi,

0.8 and 0.2 are thresholds.

If y1 = 1 and y2 = 0 then the letter is exactly T. If y1 = 0 and y2 = 1 then the letter is exactly U. If the letters differ somewhat (but not much) from T and U, then y1 0 but close to 0. I chose 0.8 and 02. somewhat arbitrarily, so that 0.8 is close to 1 and 0.2 is close to 0. If 1 < y1 < 0.8 and 0.2 < y2 < 0 then the letter is not far from T. If 1 < y2 < 0.8 and 0.2 < y1 < 0 then the letter is not far from U.

• HI,

Thank you , I understand that now. I modified the code a bit ,so now it is trained for 3 patterns of U and 3 patterns of T.So when i type disp(y) I get a 2*6 matrix with threshold values.

so if i type the conditions now in order to identify a distorted letter ‘T’

if (y(1)> 0.8 & y(2) < 0.2)
disp('Looks like T')
elseif (y(1) 0.8)
disp('Looks like U')

is it still right ? I mean do I have to make changes since I trained the network for 6 inputs. What exactly do I accomplish by training it for 6 inputs? Do i get the average of the threshold values so i can use much accurate values other than 0.8 and 0.2

0.9454486 0.9499079 0.9499079 0.0674465 0.0617696 0.0621104

0.0491076 0.0448377 0.0448377 0.9401655 0.9428039 0.9429812

Thank you very much..

• There’s a misprint in my code. I have corrected it in the post now. The line
elseif (y(1) 0.8)
should be replaced with
elseif (y(2)> 0.8 & y(1) < 0.2)

Everything else should be the same also in your case. Now try giving it some modified letters. It should recognise them if they don't differ much from T or U.

• Thanks a million sir ,this is really helpful.Keep up the good work !!!

3. Thanks a million sir ,this is really helpful.Keep up the good work !!!