Notes on Factor Analysis MBA 510 at the UNCP

The University of North Carolina at Pembroke
MBA 510--Quantitiative Methods
Notes on Factor Analysis and Principal Components

check this page frequently for updates

Link to MBA 510 Main Page.
Link to Dr. Frederick's page.

Performing Factor Analysis/Principal Components on computers
JMP has a nice interactive spinning plot, only does VARIMAX rotation
SPSS.

Glossary
Example
Uses of factor analysis
Fundamental concepts
Assumptions
Types of structures
How many factors?
Rotation

Example
A market researcher has collected spending information from 100 households. The variables measured are

spending at fast-food restaurants,
spending on vacations
spending on automobiles
spending on videos
spending on clothing

He want to reduce the number of variables from five to two. He suspects that some of the variables depend on current income, or other variables depend on expectations of future income. The correlation coefficients for these five variables were calculated and are shown in the table below.

		Y1	Y2	Y3	Y4	Y5
Automobiles	Y1	1.0	0.7	0.2	0.3	0.4
Vacations	Y2	0.7	1.0	0.1	0.2	0.4
Fast Food	Y3	0.2	0.1	1.0	0.8	0.6
Videos	Y4	0.3	0.2	0.8	1.0	0.7
Clothing	Y5	0.4	0.4	0.6	0.7	1.0

These correlation coefficients make it appear that there are indeed two underlying variables: spending on automobiles and spending on vacations appear to be correlated with each other and probably represent of the effect of expectations of future income. Spending on fast food, videos, and clothing up to be correlated also and probably represents the effect of current income. Factor analysis can be used to estimate these two unobserved underlying factors.

Uses of Factor Analysis

To search for unknown factors (data mining),
To confirm that suspected underlying factors exist,
To distill a large number of redundant variables down to a few nonredundant variables (data reduction). The new variables may be used in other analyses, instead of the many variables, to avoid losing degrees of freedom.

Fundamental Concepts
The observable variables, the Y_ij, are thought of as being functions of common factors, F_ij, and unique factors, U_ij. The common factors and the unique factors are unobservable. They may be unobservable because they really are difficult to quantify accurately or because they were not measured in the study in question. These relationships can be expressed as

Y_1j, = a₁₁F_1j + a₁₂F_2j + U_1j,
Y_2j, = a₂₁F_1j + a₂₂F_2j + U_2j,
Y_3j, = a₃₁F_1j + a₃₂F_2j + U_3j,
Y_4j, = a₄₁F_1j + a₄₂F_2j + U_4j,
Y_5j, = a₅₁F_1j + a₅₂F_2j + U_5j.

These equations form the factor structure. The coefficients in the equations, a_ij, are factor structure coefficients. (If the Y_ij had been represented in their standardized forms Z_ij = (Y_ij, - µ_i)÷s_i then these coefficients would be called the factor loadings.)

Assumptions of Factor Analysis

A factor structure exists. This implies that factors determine the values of the observed variables, Y. There are no effects of one observed variable on other observed variables. This assumption can be tested by Bartlett's test for sphericity. The null hypothesis is that there is no factor structure, so we want to be able to reject this hypothesis. (We want to see a small p value.) Another test is the Measure of Sampling Adequacy, MSA. 0 < MSA < 1 and an MSA over 0.5 is considered good.
Observed values, Y, are linear functions of the factors.
Homogeneity. The factor structure applies equally well to all observations.
Ockham's razor (or Occam's razor): When there are two equally good competing explanations of events, the simpler explanation is probably the right one.
Normality: the unique factors, U, like the error terms in linear regression, come from a normal distribution. This assumption is only important in the rare cases when we want to conduct a test of a hypothesis.
Homoskedasticity: The unique factors, U, have the same variance for all obsrevations, though the unique factors for the different variables do not all have to have the same variance. This assumption also is only important when we conduct hypothesis tests.

Factor Rotation
The purpose of factor rotation is to find a factor loadings pattern that is easier to interpret. Recall the factor indeterminance problem, which says that the factor structure is not unique. There is an infinite range of possible factor structures that will fit the observed data equally well. Factor indeterminance gives us the freedom to impose an additional criterion for the factor structure. We not only want it to provide a good fit, but we want it to be easy to interpret. Several additional criteria are available, each having its own factor rotation method.

Orthagonal rotation methods are methods that will preserve the zero correlation between the factors.

VARIMAX: One of the most commonly used methods is the VARIMAX rotation. This seeks to maximize the separation of the factor loadings in a column of the factor pattern; that is, for each factor, it tries to say that the factor is mostly correlated with a group of observed Y variables and uncorrelated witht the other Y variables. JMP uses VARIMAX.

QUARTIMAX: This method does the opposite of VARIMAX--it seeks to maximize the separation between the rows of the loadings pattern. It tries to say that a given observed variable Y is mostly correlated with a small group of factors and uncorrelated with the rest.

EQUIMAX: This method is a compromise between VARIMAX and QUARTIMAX.

There are several variants on each of these methods.

Oblique rotation methods will allow the factors to be correlated in the process of increasing the separation of the factor loadings. Two common methods are

OBLIMAX and

PROMAX

Glossary

characteristic root: See eigenvalue

common factor

communality:

eigenvalue: (Also called characteristic root or latent root) A measure of the strength of a correlation among a group of variables. The number of eigenvalues for a dataset equals the number of variables in the data set though there is no way to relate an eigenvalue to a single variable. The sum of all eigenvalues must equal the number of variables in the dataset.

factor

factor loading

factor structure

latent root: See eigenvalue

rotation

scree plot: A graph of the eigenvalues (on the vertical axis) against the rank of the eigenvalue (on the horizontal axis). Used for determining the number of factors to use. (The term is sometimes erroneously capitalized. The graph takes its name not from a person, but from geology. Scree is the rocky debris at the base of a cliff.)

Type P, S, or T factor analsys: Don't worry about them.

Type R factor analysis: A factor analysis which seeks to find similar variables. The most common use of factor analysis.

Type Q factor analysis: A factor analysis which seeks to find similar observations. Similar to cluster analysis.

unique factor: