## Issue

I built a numpy only neural network originally based on an online tutorial and have come to realise that I should have some kind of bias neuron. However I have really been struggling to figure out how to implement it into my code and would greatly appreciate some guidance.

```
import numpy as np
class NN():
def __init__(self, layers, type):
"""
layers: a list of layers, eg:
2 input neurons
1 hidden layer of 3 neurons
2 output neurons
will look like [2,3,2]
type: initialisation type, "random" or "uniform" distribution
"""
self.p = 0.1
self.layers = len(layers) - 1
self.inputSize = layers[0]
self.outputSize = layers[self.layers]
self.layerSizes = layers[:-1] #input layer, hiddens, discard output layer
self.inputs = np.zeros(self.inputSize, dtype=float)
self.outputs = np.zeros(self.outputSize, dtype=float)
self.L = {}
if type == "random":
for i in range(1,self.layers+1):
if i < self.layers:
self.L[i] = (np.random.ranf(( self.layerSizes[i-1] , self.layerSizes[i] )).astype(np.float) - 0.5) * 2
else:
self.L[i] = (np.random.ranf(( self.layerSizes[i-1] , self.outputSize )).astype(np.float) - 0.5)*2
elif type == "uniform":
for i in range(1,self.layers+1):
if i < self.layers:
self.L[i] = np.random.uniform( -1 , 1 , (self.layerSizes[i-1],self.layerSizes[i]) )
else:
self.L[i] = np.random.uniform( -1 , 1 , (self.layerSizes[i-1],self.outputSize) )
else:
print("unknown initialization type")
def updateS(self): #forward propogation Sigmoid
for i in range(1,self.layers+1):
if 1 == self.layers: #dodgy no hidden layers fix
self.z = np.dot(self.inputs, self.L[i])
self.outputs = ( self.sigmoid(self.z) - 0.5)*2
elif i == 1: #input layer
self.z = np.dot(self.inputs, self.L[i])
self.temp = self.sigmoid(self.z)
elif i < self.layers: #hidden layers
self.z = np.dot(self.temp, self.L[i])
self.temp = self.sigmoid(self.z)
else: #output layer
self.z = np.dot(self.temp, self.L[i])
self.outputs = ( self.sigmoid(self.z) - 0.5)*2
def sigmoid(self, s):
#activation funtion
return 1/(1+np.exp(-s/self.p))
```

## Solution

Bias is just a variable you add to each neuron as you progress through the neural network feed forward process. Thus the feed forward process from one neuron layer to the next will be the sum of all the weights multiplied by the previous neuron that feed into the next neuron, then the bias of that neuron will be added, or:

output = sum(weights * inputs) + **bias**

To put this into context, look at the picture below:

where:

```
X1: Input value 1.
X2: Input value 2.
B1n: Layer 1, neuron n bias.
H1: Hidden layer neuron 1.
H2: Hidden layer neuron 2.
a(…): activation function.
B2n: Layer 2, neuron n bias.
Y1: network output neuron 1.
Y2: network output neuron 2.
Y1out: network output 1.
Y2out: network output 2.
T1: Training output 1.
T2: Training output 2.
```

when working out H1, you would need to use the following formula:

```
H1 = (X1 * W1) + (X2 * W2) + B11
```

Note that is is before the value for the neuron has been completely calculated via the activation function.

Therefore, im pretty sure that the bias would be entered within the feed forward function:

```
def updateS(self): #forward propogation Sigmoid
for i in range(1,self.layers+1):
if 1 == self.layers: #dodgy no hidden layers fix
self.z = np.dot(self.inputs, self.L[i])
self.outputs = ( self.sigmoid(self.z) - 0.5)*2
elif i == 1: #input layer
self.z = np.dot(self.inputs, self.L[i])
self.temp = self.sigmoid(self.z)
elif i < self.layers: #hidden layers
self.z = np.dot(self.temp, self.L[i])
self.temp = self.sigmoid(self.z)
else: #output layer
self.z = np.dot(self.temp, self.L[i])
self.outputs = ( self.sigmoid(self.z) - 0.5)*2
```

by adding a value to the end of the self.z values. I think these values can be anything you want as a bias simply moves the intercept of the linear equation

Answered By – Qualia91

Answer Checked By – Pedro (BugsFixing Volunteer)