Day 9: Multiple Linear Regression

If is linearly dependent only on , then we can use the ordinary least square regression line, . However, if shows linear dependency on variables , , , , then we need to find the values of and other constants (). We can then write the regression equation as:

Matrix Form of the Regression Equation

Let's consider that depends on two variables, and . We write the regression relation as . Consider the following matrix operation:

We define two matrices,

and

Now, we rewrite the regression relation as . This transforms the regression relation into matrix form.

Generalized Matrix Form

We will consider that shows a linear relationship with variables, , , , . Let's say that we made observations on different tuples :

Now, we can find the matrices:

Finding the Matrix B

We know that

Note: is the transpose matrix of , is the inverse matrix of , and is the identity matrix.

Finding the Value of

Suppose we want to find the value of for some tuple , then,

Example

Consider shows a linear relationship with and :

Now, we can define the matrices:

Now, find the value of :

So, , which means , , and .

Let's find the value of at

Multiple Regression in R

x1 = c(5, 6, 7, 8, 9) x2 = c(7, 6, 4, 5, 6) y = c(10, 20, 60, 40, 50) m = lm(y ~ x1 + x2) show(m)

Running the above code produces the following output:

Call: lm(formula = y ~ x1 + x2) Coefficients: (Intercept) x1 x2 51.953 6.651 -11.163

Multiple Regression in Python

from sklearn import linear_model x = [[5, 7], [6, 6], [7, 4], [8, 5], [9, 6]] y = [10, 20, 60, 40, 50] lm = linear_model.LinearRegression() lm.fit(x, y) a = lm.intercept_ b = lm.coef_ print a, b[0], b[1]