Implement Logistic Regression In neural Nets (From Basics)

Here I will show you how to implement a neural network for logistic regression from basics.

What is logistic regression ?
The two main types of problems in the machine learning are are classification and regression. Both techniques are used for prediction of labels for a given data set. In classification, data can be fitted into a different classes, hence the labels will give discrete values. For regression, the data set is mapped into continuous values. Different types of regression include linear regression, logistic regression etc. 

Here for one training example, the neural network using logistic regression is explained. If two features of a image is considered for training purpose , the linear combination of the features with weights and bias can be shown as below :
Z = w1x1+w2x2+b
The activation function selected for logistic regression here is sigmoid function.
This shows the forward propagation step in neural net. Now the output of this neural net should be close to the actual output for the data. Hence first calculate the loss function for the training example as :
L(a,y)
where 'a' is current output and 'y' is actual label.
For all training examples m, consider the cost function of neural net as J(w,b)
Hence for a good model, optimization is applied for this cost function.
Gradient Descent can be used for optimization. After this weight 'w' can be updated as :
w = w- (learning rate) * dw
b = b- (learning rate) * db

The above figure gives an overall idea about the procees in logistic regression implemented on neural network.

The main equations for our neural network is shown in the above image.
In the next image we will see how to derive the
* w = w- (learning rate) * dw
* b = b- (learning rate) * db
 using calculus.

Hence a simple neural network is explained for logistic regression.