Statistics Toolbox

Example: Multiple Linear Regression

The example comes from Chatterjee and Hadi (1986) in a paper on regression diagnostics. The data set (originally from Moore (1975)) has five predictor variables and one response.

• load moore
  X = [ones(size(moore,1),1) moore(:,1:5)];
  

Matrix X has a column of ones, and then one column of values for each of the five predictor variables. The column of ones is necessary for estimating the y-intercept of the linear model.

• y = moore(:,6);
  [b,bint,r,rint,stats] = regress(y,X);
  

The y-intercept is b(1), which corresponds to the column index of the column of ones.

• stats
  stats =
      0.8107   11.9886    0.0001
  

The elements of the vector stats are the regression R² statistic, the F statistic (for the hypothesis test that all the regression coefficients are zero), and the p-value associated with this F statistic.

R² is 0.8107 indicating the model accounts for over 80% of the variability in the observations. The F statistic of about 12 and its p-value of 0.0001 indicate that it is highly unlikely that all of the regression coefficients are zero.

• rcoplot(r,rint)
  
  
  

The plot shows the residuals plotted in case order (by row). The 95% confidence intervals about these residuals are plotted as error bars. The first observation is an outlier since its error bar does not cross the zero reference line.

In problems with just a single predictor, it is simpler to use the polytool function (see The polytool Demo). This function can form an X matrix with predictor values, their squares, their cubes, and so on.

Mathematical Foundations of Multiple Linear Regression

Quadratic Response Surface Models