Linear Models

Linear Model Classification

It is useful to describe various classifications of linear models to better understand how each type of linear model (e.g. Response Surface Methodology) fits into the "big picture".

A very helpful guide, given by Matlab, that describes and illustrates various regression analysis techniques: http://www.mathworks.com/help/toolbox/stats/bq_676m-2.html

Generalized Linear Models

"Generalized linear models" are linear models that can account for arbitrary numbers of inputs and outputs. These models assume errors are Gaussian, use statistics and create statistical models for data analysis.

General form:

$\mathbf {Y} =\mathbf {BX} +\mathbf {U}$

where:

$Y$ is the response of the function of interest

$\mathbf {B}$ is the coefficient vector/matrix for the linear model coefficients

$\mathbf {X}$ are the inputs to the function

$\mathbf {U}$ is the error

General linear model information:

• http://en.wikipedia.org/wiki/Multivariate_regression_model
• Nelder and Wedderburn (1972)

Matlab functions:

• glmfit
  • http://www.mathworks.com/help/toolbox/stats/glmfit.html
  • Carries out the regression

• glmval
  • http://www.mathworks.com/help/toolbox/stats/glmval.html
  • Evaluates the value of a generalized linear model

Multiple Linear Regression

"Multiple linear regression" is a model for one response variable ("y"), and multiple predictor variables ("X").

Linear regression information:

• http://en.wikipedia.org/wiki/Linear_regression
• http://www.mathworks.com/help/toolbox/stats/mvregress.html
• Mason Ch. 15

Generic multiple linear regression model looks like:

$y_{i}=\beta _{0}+\beta _{1}x_{i1}+\beta _{2}x_{i2}+\dots +\beta _{k}x_{ik}+e_{i}\quad i=1\dots n$

for $n$ responses (and all responses treated independently). Here, $y_{i}$ is the $i^{th}$ response from the "real" (i.e. unknown) process, which is perfectly represented using the polynomial above.

It is desirable to create a surrogate model that creates as good an approximation as possible of the above polynomial.

This "real", unknown polynomial can also be written (similar to ANOVA model):

$y_{i}=\mu _{i}+e_{i}$

where

$\mu _{i}=\beta _{0}+\beta _{1}x_{i1}+\beta _{2}x_{i2}+\dots +\beta _{k}x_{ik}$

If each of the coefficients $\beta _{j}$ are approximated (using linear algebra, e.g. least squares), the mean can be approximated:

${\hat {\mu }}_{i}=\beta _{0}+\beta _{1}x_{i1}+\beta _{2}x_{i2}+\dots +\beta _{k}x_{ik}$

and using that, the "real" polynomial responses $y_{i}$ can be approximated with surrogate (or predicted) polynomial responses ${\hat {y}}_{i}$:

${\hat {y}}_{i}={\hat {\mu }}_{i}$

Multivariate Linear Regression

"Multivariate linear regression" broadens multiple linear regression to account for more than one response variable ("Y").

Multivariate regression/analysis information:

• http://en.wikipedia.org/wiki/Multivariate_analysis

Polynomial Models (Univariate)

Polynomial models can be used to fit a univariate function of a single input paramter, e.g. $y(x)$

This can be done using the following Matlab functions:

• polyfit
  • http://www.mathworks.com/help/techdoc/ref/polyfit.html
  • Fits a polynomial of a given degree to a set of inputs x and outputs y

• polyval
  • http://www.mathworks.com/help/techdoc/ref/polyval.html
  • Evaluates the value of a given polynomial model at given input variable value or values

• polyconf
  • http://www.mathworks.com/help/toolbox/stats/polyconf.html
  • Can be used to construct confidence intervals for polynomial models

Response Surface Models (Multivariate)

Main article: Response Surface Methodology

Response surfaces may be created in a number of different ways:

• regress

• regstats

• SurfaceFit (Statistics toolbox)

• others???

See Also

Pages for different types of linear models:

• Response Surface Methodology