INTRODUCTION TO LINEAR REGRESSION

Linear Regression in Machine Learning

Linear regression is one of the easiest and most popular Machine Learning algorithms. It is a statistical method that is used for predictive analysis. Linear regression makes predictions for continuous/real or numeric variables such as sales, salary, age, product price, etc.

Linear regression algorithm shows a linear relationship between a dependent (y) and one or more independent (y) variables, hence called as linear regression. Since linear regression shows the linear relationship, which means it finds how the value of the dependent variable is changing according to the value of the independent variable.

The linear regression model provides a sloped straight line representing the relationship between the variables. Consider the below image:

y= a0+a1x+ ε

Here,

Y= Dependent Variable (Target Variable)
X= Independent Variable (predictor Variable)
a0= intercept of the line (Gives an additional degree of freedom)
a1 = Linear regression coefficient (scale factor to each input value).
ε = random error

The values for x and y variables are training datasets for Linear Regression model representation.

Types of Linear Regression

Linear regression can be further divided into two types of the algorithm:

• Simple Linear Regression:
  If a single independent variable is used to predict the value of a numerical dependent variable, then such a Linear Regression algorithm is called Simple Linear Regression.
• Multiple Linear regression:
  If more than one independent variable is used to predict the value of a numerical dependent variable, then such a Linear Regression algorithm is called Multiple Linear Regression.

Linear Regression Line

A linear line showing the relationship between the dependent and independent variables is called a regression line. A regression line can show two types of relationship:

• Positive Linear Relationship:
  If the dependent variable increases on the Y-axis and independent variable increases on X-axis, then such a relationship is termed as a Positive linear relationship.

• Negative Linear Relationship:
  If the dependent variable decreases on the Y-axis and independent variable increases on the X-axis, then such a relationship is called a negative linear relationship.

Finding the best fit line:

When working with linear regression, our main goal is to find the best fit line that means the error between predicted values and actual values should be minimized. The best fit line will have the least error.

The different values for weights or the coefficient of lines (a0, a1) gives a different line of regression, so we need to calculate the best values for a0 and a1 to find the best fit line, so to calculate this we use cost function.

Cost function-

• The different values for weights or coefficient of lines (a0, a1) gives the different line of regression, and the cost function is used to estimate the values of the coefficient for the best fit line.
• Cost function optimizes the regression coefficients or weights. It measures how a linear regression model is performing.
• We can use the cost function to find the accuracy of the mapping function, which maps the input variable to the output variable. This mapping function is also known as Hypothesis function.

For Linear Regression, we use the Mean Squared Error (MSE) cost function, which is the average of squared error occurred between the predicted values and actual values. It can be written as:

For the above linear equation, MSE can be calculated as:

Where,

N=Total number of observation
Yi = Actual value
(a1xi+a0)= Predicted value.

Residuals: The distance between the actual value and predicted values is called residual. If the observed points are far from the regression line, then the residual will be high, and so cost function will high. If the scatter points are close to the regression line, then the residual will be small and hence the cost function.

Gradient Descent:

• Gradient descent is used to minimize the MSE by calculating the gradient of the cost function.
• A regression model uses gradient descent to update the coefficients of the line by reducing the cost function.
• It is done by a random selection of values of coefficient and then iteratively update the values to reach the minimum cost function.

The top benefits of linear regression in machine learning is as follows.

A top advantage of using a linear regression model in machine learning is the ability to forecast trends and make predictions that are feasible. Data scientists can use these predictions and make further deductions based on machine learning. It is quick, efficient, and accurate. This is predominantly since machines process large volumes of data and there is minimum human intervention. Once the algorithm is established, the process of learning becomes simplified.

Beneficial to small businesses
By altering one or two variables, machines can understand the impact on sales. Since deploying linear regression is cost-effective, it is greatly advantageous to small businesses since short- and long-term forecasts can be made when it comes to sales. This means that small businesses can plan their resources well and create a growth trajectory for themselves. They will also be to understand the market and its preferences and learn about supply and demand.

Preparing Strategies
Since machine learning enables prediction, one of the biggest advantages of a linear regression model in it is the ability to prepare a strategy for a given situation, well in advance, and analyze various outcomes. Meaningful information can be derived from the regression model of forecasting thereby helping companies plan strategically and make executive decisions.