Hello, everyone! I'm the happy old boy from MPIGYangpu district of Shanghai , today I introduce you to logistic regression.
This will be an exciting chapter as we will be exposed to optimization algorithms for the first time. If you think about it, you will find that we have actually encountered many optimization problems in our daily lives, such as how to get from point A to point B in the shortest possible time? How do you put in the least amount of work but get the most out of it? How do you design an engine to make the least amount of fuel consumption and the most power? As you can see, optimization is very powerful. Next, we present several optimization algorithms and use them to train a nonlinear function for classification.
Suppose now we have some data points and we fit these points with a straight line (the line is called a best-fit line), this fitting process is called regression. The main idea of classification using logistic regression is to create a regression formula for the classification boundary line based on the available data as a way to classify. The term "regression" here is derived from best-fit, which means finding the best-fitting set of parameters, and the mathematical analysis behind it will be described in the next section. The practice when training the classifier is to find the best-fit parameters, using an optimization algorithm.
The function we want should be able to both accept all the inputs that are available and then predict the category. For example, in the case of two classes, the above function outputs 0 or 1. We therefore introduce the Sigmoid Function.