Multivariate Statistical Modelling Based on Generalized Linear Models

Chapter 1: Book Outline and Notes

Chapter 2: review of univariate generalized linear models and extensions

Chapter 3: models for multicategorical responses (i.e. multiple, unordered responses)

Chapter 4: selecting variables for models, variable reduction procedures, checking models, goodness-of-fit, residual analysis (outliers or consistent trend?)

Chapter 5: Semi- and non-parametric approaches

Chapter 6: Fixed-parameter models for time series (extends Ch. 2 and Ch. 3)

Chapter 7: Random effects models for non-normal data

Chapter 8: State space models for analyzing non-normal time series; relate time series observations y_t to unobserved states, like trend and seasonal components

Chapter 9: Survival models; determination of factors that determine survival/transition

Chapter 2: Univariate Generalized Linear Models

Cross-sectional regression analysis: univariate variable of primary interest (response variable) $y$

Explained by a vector $x=(x_{1},x_{2},\dots x_{m})$

Data consist of observations on $(y,x)$:

$(y_{i},x_{i}),i=1,\dots ,n$

Definition of Univariate Generalized Linear Models

classical linear model for ungrouped normal responses and deterministic covariates is:

$y_{i}=z_{i}^{\prime }\beta +\epsilon _{i}$

where:

$z_{i}$ = design vector, function of covariate vector $x_{i}$

$\beta$ = vector of unknown parameters

$\epsilon _{i}$ = errors, normally distributed and independent, $\epsilon _{i}\sim N(0,\sigma ^{2})$

The observations $y_{i}$ are independent and normally distributed,

$y_{i}\sim N(\mu _{i},\sigma ^{2}),i=1,\dots ,n$

A specific generalized linear model is fully characterized by three components:

• type of exponential family
• response or link function
• design vector

Example:

Exponential family

• important members: normal, binomial, poisson, gamma, inverse Gaussian distributions

Models for Continuous Responses

Normal distribution

Gamma distribution

Inverse Gaussian distribution

Models for Binary and Binomial Responses

Linear probability model

Probit model

Logit model

Complementary log-log model

Models for Counted Data

Log-linear Poisson model

Linear Poisson model

Likelihood Inference

Regression analysis with generalized linear models is based on likelihoods

This section contains inferential tools for:

• parameter estimation
• hypothesis testing
• good-ness-of-fit tests
• more detailed material on model choice/checking: see Chapter 4

Assumes that model is completely and correctly specified

Maximum likelihood estimator (MLE): MLE of unknown parameter vector obtained by maximizing the likelihood

Goodness of fit Statistics

two measures of adequacy of model (goodness of fit) are:

Pearson statistic $\chi ^{2}=\sum _{i=1}^{g}{\frac {\left(y_{i}-{\hat {\mu }}_{i}\right)^{2}}{v({\hat {\mu }}_{i})}}$

Deviance $D=-2\phi \sum _{i=1}^{g}\left[l_{i}({\hat {\mu }}_{i})-l_{i}(y_{i})\right]$

where

${\hat {\mu }}_{i}$ = estimated mean function

$v({\hat {\mu }}_{i})$ = estimated variance function

$l_{i}(y_{i})$ = individual log-likelihood

References

Generalized linear models:

• McCullagh and Nelder 1989 - standard source of information about generalized linear models
• Santner and Duffy 1989 - consider cross-classified data and univariate discrete data