S675 class notes

Revision as of 14:36, 26 October 2009

Fisher's best linear discriminator constructs a linear combination of the variables that is usually good for discrimination. This linear comination is not necessarily the linear combination found by PCA
The rule htat assigns an unlabelled u to the class label i for which (u-xbarsubi)^tS^-1(u - xbarsubi) is minimal bas an obvious extension from the case of Z classes (i in {1,2}) to the case of g classes (i in {1,2,...,g}). The general rule is called linear discirminant analysis (LDA).
LDA relies on a pooled sample covariance matrix S.

- Let Sigma = ES. where E is the expectation.
- It may or maynot be the case that eash class has population covariance matrix Sigma.
- What if the classes have different covariance matrices, Sigmasub1, ... , Sigmasubg?
- One possibility: estimate reach Sigmasubi by Ssubi, then assign to u the label i for which (u-xbarsubi)^tSsubi^-1(u-xbarsubi) is minimal.
- This rule is called quadratic discriminant analysis (QDA)

@@ Line 1: / Line 1: @@
 ===remarks on 10/26/09===
 #Fisher's best linear discriminator constructs a linear combination of the variables that is usually good for discrimination.  This linear comination is not necessarily the linear combination found by PCA
-#The rule htat assigns an unlabelled u to the class label i for which   (u-xbarsubi)^tS^-1(u - xbarsubi)
+#The rule htat assigns an unlabelled u to the class label i for which   (u-xbarsubi)^tS^-1(u - xbarsubi) is minimal bas an obvious extension from the case of Z classes (i in {1,2}) to the case of g classes (i in {1,2,...,g}).  The general rule is called linear discirminant analysis (LDA).
-is minimal bas an obvious extension from the case of Z classes (i in {1,2}) to the case of g classes (i in {1,2,...,g}).  The general rule is called linear discirminant analysis (LDA).
 #LDA relies on a pooled sample covariance matrix S.
 **Let Sigma = ES. where E is the expectation.