S675 class notes

remarks on 10/26/09

1. Fisher's best linear discriminator constructs a linear combination of the variables that is usually good for discrimination. This linear comination is not necessarily the linear combination found by PCA
2. The rule htat assigns an unlabelled u to the class label i for which (u-xbarsubi)^tS^-1(u - xbarsubi) is minimal bas an obvious extension from the case of Z classes (i in {1,2}) to the case of g classes (i in {1,2,...,g}). The general rule is called linear discirminant analysis (LDA).
3. LDA relies on a pooled sample covariance matrix S.

• • Let Sigma = ES. where E is the expectation.
  • It may or maynot be the case that eash class has population covariance matrix Sigma.
  • What if the classes have different covariance matrices, Sigmasub1, ... , Sigmasubg?
  • One possibility: estimate reach Sigmasubi by Ssubi, then assign to u the label i for which (u-xbarsubi)^tSsubi^-1(u-xbarsubi) is minimal.
  • This rule is called quadratic discriminant analysis (QDA)