Artificial neural networks in Scilab

Currently, the Scilab’s ANN toolbox doesn’t have much. Only feed-forward networks and only some basic local search algorithms.

So there’s plenty of work for open source enthusiasts.

It would be nice to add other types of networks and global search algorithms.

The help files for the toolbox don’t have any examples.

Searching the Internet, I found an example in one blog post written by this Filipino girl.

Here I give one more example on using multi-layer perceptron for simple character recognition.

This is not a full-featured character recognition application.

Rather it is just an educational toy to get an idea how to use the ANN toolbox in Scilab.

Our neural network will recognize two letters: T and U.

T

T = [...
1 1 1 1 1 ...
0 0 1 0 0 ...
0 0 1 0 0 ...
0 0 1 0 0 ...
0 0 1 0 0 ...
0 0 1 0 0 ...
0 0 1 0 0 ...
]';

U

U = [...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
0 1 1 1 0 ...
]';

These pictures are screenshots of *.pgm files opened with ImageJ.

The contents of the *.pgm files are

T.pgm

P2
5 7
1
1 1 1 1 1
0 0 1 0 0
0 0 1 0 0
0 0 1 0 0
0 0 1 0 0
0 0 1 0 0
0 0 1 0 0

U.pgm

P2
5 7
1
1 0 0 0 1
1 0 0 0 1
1 0 0 0 1
1 0 0 0 1
1 0 0 0 1
1 0 0 0 1
0 1 1 1 0

To recognize these letters, we built a perceptron with three layers.

The input layer has 35 neurons – one neuron for each pixel in the picture.

The output layer has two neurons – one neuron per class in our classification task (one for “T”, another one for “U”).

The hidden layer has 10 neurons.

The numbers of neurons in all layers are collected in a row vector

N = [35 10 2];

Then, the weights of the connections between the neurons are initialized

W = ann_FF_init(N);

Our training set consists of the two images above

x = [T, U];

Now let’s specify the desired responses.

If the input layer reads “T” then the first output neuron should give 1 and the second output neuron should give 0

t_t = [1 0]';

If the input layer reads “U” then the first output neuron should give 0 and the second output neuron should give 1

t_u = [0 1]';

To be used in Scilab ANN routines, the responses need to be combined in a matrix that has the same amount of columns as the training set

t = [t_t, t_u];

Other training parameters are the learning rate and the threshold of error

lp = [0.01, 1e-4];

The number of training cycles:

epochs = 3000;

The network is trained with the standard backpropagation in batch mode:

W = ann_FF_Std_batch(x,t,N,W,lp,epochs);

W is the matrix of weights after the training.

Let’s test it:

y = ann_FF_run(x,N,W)
disp(y)

It gave me:

    0.9007601    0.1166854  
    0.0954540    0.8970468  

Looks fine.

Now, let’s feed the network with slightly distorted inputs and see if it recognises them

T_odd

T_odd = [...
1 1 1 1 1 ...
0 0 1 0 1 ...
0 0 1 0 1 ...
0 0 1 0 0 ...
0 0 1 0 0 ...
0 0 1 0 0 ...
0 0 1 0 0 ...
]';

U_odd

U_odd = [...
1 1 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
0 1 1 1 1 ...
]';

Also, let’s check something that is neither “T” nor “U”

M

M = [...
1 0 0 0 1 ...
1 1 0 1 1 ...
1 0 1 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
1 0 0 0 1 ...
]';

The output of the network is calculated by inserting each of these vectors into ann_FF_run instead of the training vector x For example,

y = ann_FF_run(T_odd,N,W);
disp(y)

Then I write the code that displays

  • “Looks like T” if the network recognizes “T”
  • “Looks like U” if the network recognizes “U”
  • “I can’t recognise it” if the letter is not recognised by the network
if (y(1)> 0.8 & y(2) < 0.2)
    disp('Looks like T')
elseif (y(2)> 0.8 & y(1) < 0.2)
    disp('Looks like U')
else
    disp('I can''t recognise it')
end

Feeding T_odd gave me the output:

    0.8798083  
    0.1333073  
 
 Looks like T 

Feeding U_odd produces the output:

    0.1351676  
    0.8113366  
 
 Looks like U   

The letter M was not recognized:

    0.2242181  
    0.8501858  
 
 I can't recognise it   
Advertisements

The URI to TrackBack this entry is: https://burubaxair.wordpress.com/2014/03/12/artificial-neural-networks-in-scilab/trackback/

RSS feed for comments on this post.

11 CommentsLeave a comment

  1. Vlad Gladkikh,

    Hello. Can you explain why you select 10 neurons in the hidden layer?. Are there some criteria to select it according to the number of inputs or something like that?. Thank you so much.

    • It doesn’t depend on the number of inputs or outputs.

      Neurons in the hidden layers detect features of the modeled system. If you use your neural network to model some complicated system with complex behavior influenced by many different factors, then you need many hidden neurons (maybe hundreds of thousands) and more than one hidden layer (but most often two layers are enough). If you use less hidden neurons than required, then your solution will show a kind of an averaged behavior of your real system, where some features are averaged out (or not noticed by your neural network). Here, it is just a simple toy example, so one hidden layer with ten neurons seems to be enough.

  2. Hello,

    what do values 0.8 and 0.2 represent in here?

    Thank you very much

    • Hi,

      0.8 and 0.2 are thresholds.

      If y1 = 1 and y2 = 0 then the letter is exactly T. If y1 = 0 and y2 = 1 then the letter is exactly U. If the letters differ somewhat (but not much) from T and U, then y1 0 but close to 0. I chose 0.8 and 02. somewhat arbitrarily, so that 0.8 is close to 1 and 0.2 is close to 0. If 1 < y1 < 0.8 and 0.2 < y2 < 0 then the letter is not far from T. If 1 < y2 < 0.8 and 0.2 < y1 < 0 then the letter is not far from U.

      • HI,

        Thank you , I understand that now. I modified the code a bit ,so now it is trained for 3 patterns of U and 3 patterns of T.So when i type disp(y) I get a 2*6 matrix with threshold values.

        so if i type the conditions now in order to identify a distorted letter ‘T’

        if (y(1)> 0.8 & y(2) < 0.2)
        disp('Looks like T')
        elseif (y(1) 0.8)
        disp('Looks like U')

        is it still right ? I mean do I have to make changes since I trained the network for 6 inputs. What exactly do I accomplish by training it for 6 inputs? Do i get the average of the threshold values so i can use much accurate values other than 0.8 and 0.2

        0.9454486 0.9499079 0.9499079 0.0674465 0.0617696 0.0621104

        0.0491076 0.0448377 0.0448377 0.9401655 0.9428039 0.9429812

        Thank you very much..

        • There’s a misprint in my code. I have corrected it in the post now. The line
          elseif (y(1) 0.8)
          should be replaced with
          elseif (y(2)> 0.8 & y(1) < 0.2)

          Everything else should be the same also in your case. Now try giving it some modified letters. It should recognise them if they don't differ much from T or U.

          • Thanks a million sir ,this is really helpful.Keep up the good work !!!

  3. Thanks a million sir ,this is really helpful.Keep up the good work !!!

  4. Helo Vlad,

    Let me know: if you don’t have a simple output (0 or 1), but a range of values, like 0.1 to 20, for example; following your example, exactly, my results are number between 0 and 1. Whats is wrong?

    • In that case your output layer should have the same amount of neurons as the number of your output values. When the network recognizes the first value (0.1 in your example), the first neuron outputs 1, while all the rest are zeros. When the network recognizes the last value (20 in your example), the last neuron outputs 1, while all the rest are zeros. Let’s say, your fifth output is 10. When the network recognizes it, the fifth neuron outputs 1, while all the rest are zeros.

  5. Hello sir.,
    My ann_FF_Std_batch function is not working. It’s giving the error submatrix incorrectly defined in scilab
    Thanks in advance


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: