Basic Artificial Neural Network in C#

• Download source code - 75.4 KB

Background

The human brain can be viewed as a complex assembly of millions of neurons interconnected with each other. The information received by the brain is processed by these tiny neurons. Signals received by a neuron via dendrites and are further relayed by synapse. The synapses either pass a signal to adjoining neuron or not depending on the strength of the signal. As the human brain acquires information, new connections are developed between neurons and existing connections are modified.

Artificial Neural Network attempts to simulate (to a very basic level) the functioning of a human brain. A set of interconnected neurons are created. These interconnected neurons are organized in layers for easier handling. The signal transfer between neurons happen via dendrites. The potential of a neuron to "fire" a signal to the adjoining connected neuron is decided by an activation potential. Usually, this firing tendancy (also known as action potential) is modeled by a sigmoidal function. The sigmoidal function looks like:

The sigmoidal function returns a value between 0 and 1 for any input of x. It is highly recommended that the reader studies through the properties of sigmoid function in order to appreciate its use as activation function. This article does not attempt to discuss the fundamentals of Artificial Neural Network. Instead, the readers are recommended to refer to more authorative texts on the subject.

Using the Code

The Artificial Neural Network contains the following major classes:

Dendrite

This is the fundamental unit which transfers signal from one neuron to another. The signal transmitted through the dendrite is weighted to simulate the intensity of the signal being passed. It should be noted that this class exposes only one property - Weight.

public class Dendrite
    {
        private double wt;
        public double Weight
        {
            get { return wt; }
            set { wt = value; }
        }

        //Provide a constructor for the class.
        //It is always better to provide a constructor instead of using
        //the compiler provided constructor
        public Dendrite()
        {
            wt = getRandom(0.00000001, 1.0);
        }

        private double getRandom(double MinValue, double MaxValue)
        {
            return Util.GetRandomD() * (MaxValue - MinValue) + MinValue;
        }
    }

Neuron

The neurons are the basic building blocks of the Artificial Neural Network. The neurons expose the value in it along with a bias. The neurons also contain properties representing the errors in the expected solutions along with the most important parameter, the dendrites. Each neuron will have as many dendrites as the number of neurons in the previous layer.

class Neuron
    {
        private List<Dendrite> dendrites;
        private double bias, delta, _value;

        public double Bias
        {
            get
            { return bias; }
            set
            { bias = value; }
        }
        public double Delta
        {
            get
            { return delta; }
            set
            { delta = value; }
        }
        public double Value
        {
            get
            { return _value; }
            set
            { _value = value; }
        }

        public void AddDendrites(int nDendrites)
        {
            int i;
            //Dendrite d;
            for(i=0;i<nDendrites;i++)
            {
                //d = new Dendrite();
                dendrites.Add(new Dendrite());
            }
        }

        public int nDendrites()
        {
            return dendrites.Count;
        }

        public Dendrite getDendrite(int index)
        {
            return dendrites[index];
        }

        public Neuron()
        {
            bias = Util.GetRandomD();
            dendrites = new List<Dendrite>();
        }

    }

Layer

The Layer class contains a list of neurons in the layer.

class Layer
   {
       private List<Neuron> neurons;

       public void Clear()
       {
           neurons.Clear();
       }
       public void Initialize(int nNeurons)
       {
           int i;
           for(i=0;i<nNeurons;i++)
           {
               neurons.Add(new Neuron());
           }
       }

       public Neuron getNeuron(int index)
       {
           return neurons[index];
       }
       public void setNeuron(int index, ref Neuron neuron)
       {
           neurons[index] = neuron;
       }
       public void setNeuron(int index, Double value)
       {
           Neuron n = new Neuron();
           n.Value = value;
           neurons[index] = n;
       }

       public void AddDendritesToEachNeuron(int nDendrites)
       {
           int i;
           for(i=0;i<neurons.Count;i++)
           {
               neurons[i].AddDendrites(nDendrites);
           }
       }

       public int nNeurons()
       {
           return neurons.Count;
       }

       /// <summary>
       /// Constructor of the class
       /// </summary>
       public Layer()
       {
           neurons = new List<Neuron>();
       }
   }

Network

The network class contains a list of layers. Thus, the class hierarchy can be shown as:

Network -> Layer -> Neuron -> Dendrite

Once the network is initialized, the dendrites are assigned random weights. As these weights are random, the generated network is hardly of any use. The weights associated with each dendrite must be fine tuned in order to obtain meaningful results. This is known as training. A neural network requires to be trained for a given set of input and corresponding output data. The set of available data is known as Training Set. The training set usually forms approximately 80 - 85% of the total available data. The remaining available data is used to confirm the training. The training primarily involves determining the error at output node and distributing it in the form of change in dendrite weights. This method is known as Back Propagation. Back propagation is mathematically performed as gradient descent method. Readers are recommended to refer to a descriptive treatment of gradient descent method and is considered to be out of scope for this article.

The training of a network is performed in the following way:

1. Run the input
2. Calculate the error on the output neurons
3. From the error computed on output neurons, adjust the weights in dendrites. 
4. Do this for all layers
5. Repeat

The training may be terminated when the error norm reduces below threshold or may be terminated after a definite number of cycles. This code runs for a definite number of cycles.

Once the network is trained (and correctness confirmed), inputs with unknown outputs may be entered to obtain the output. Note that Artificial Neural Networks may NOT return 100% accurate results. The results are usually within acceptable accuracy limits.

Points of Interest

Various heuristic parameters such as learning rate requires significant experience before the the network may successfully be trained at a good speed. This code allows experimenting with number of hidden layers and number of neurons in each hidden layer. The code has been tested with sinusoidal function (between 0 and 1) and other functions such as addition, substraction, etc. and has been found to work with significant accuracy.

History

• 7^th August, 2017: First version of the code