Inverting a
matrix
The matrix analog of division involves an operation called inverting
a matrix. Only square matrices can be inverted. Inversion is a
tedious numerical procedure and it is best performed by computers.
There are many ways to invert a matrix, but ultimately whichever
method is selected by a program is immaterial. If you wish to try one
method by hand, a very popular numerical method is the
Gauss-Jordan method.
Identity matrix
To augment the notion of the inverse of a matrix, A
-1
(A inverse) we
notice the following relation
A
-1
A = A A
-1
= I
I is a matrix of form
I is called the identity matrix and is a special case of a diagonal
matrix. Any matrix that has zeros in all of the off-diagonal positions
is a diagonal matrix.
6.5.3.1. Numerical Examples
(3 of 3) [5/1/2006 10:35:35 AM]
D is the sample variance-covariance matrix for observations of
a multivariate vector of p elements. The determinant of D, in
this case, is sometimes called the generalized variance.
Characteristic
equation
In addition to a determinant and possibly an inverse, every

square matrix has associated with it a characteristic equation.
The characteristic equation of a matrix is formed by subtracting
some particular value, usually denoted by the greek letter
(lambda), from each diagonal element of the matrix, such that
the determinant of the resulting matrix is equal to zero. For
example, the characteristic equation of a second order (2 x 2)
matrix A may be written as
Definition of the
characteristic
equation for 2x2
matrix
Eigenvalues of a
matrix
For a matrix of order p, there may be as many as p different
values for
that will satisfy the equation. These different values
are called the eigenvalues of the matrix.
Eigenvectors of a
matrix
Associated with each eigenvalue is a vector, v, called the
eigenvector. The eigenvector satisfies the equation
Av =
v
Eigenstructure of a
matrix
If the complete set of eigenvalues is arranged in the diagonal
positions of a diagonal matrix V, the following relationship
holds
AV = VL
This equation specifies the complete eigenstructure of A.

Eigenstructures and the associated theory figure heavily in
multivariate procedures and the numerical evaluation of L and V
is a central computing problem.
6.5.3.2. Determinant and Eigenstructure
(2 of 2) [5/1/2006 10:35:36 AM]
By
convention,
rows
typically
represent
observations
and columns
represent
variables
In this case the number of rows, (n = 5), is the number of observations,
and the number of columns, (p = 3), is the number of variables that are
measured. The rectangular array is an assembly of n row vectors of
length p. This array is called a matrix, or, more specifically, a n by p
matrix. Its name is X. The names of matrices are usually written in
bold, uppercase letters, as in Section 6.5.3. We could just as well have
written X as a p (variables) by n (measurements) matrix as follows:
Definition of
Transpose
A matrix with rows and columns exchanged in this manner is called the
transpose of the original matrix.
6.5.4. Elements of Multivariate Analysis
(2 of 2) [5/1/2006 10:35:36 AM]
Mean vector
and
variance-

covariance
matrix for
sample data
matrix
The results are:

where the mean vector contains the arithmetic averages of the three
variables and the (unbiased) variance-covariance matrix S is calculated
by
where n = 5 for this example.
Thus, 0.025 is the variance of the length variable, 0.0075 is the
covariance between the length and the width variables, 0.00175 is the
covariance between the length and the height variables, 0.007 is the
variance of the width variable, 0.00135 is the covariance between the
width and height variables and .00043 is the variance of the height
variable.
Centroid,
dispersion
matix
The mean vector is often referred to as the centroid and the
variance-covariance matrix as the dispersion or dispersion matrix. Also,
the terms variance-covariance matrix and covariance matrix are used
interchangeably.

6.5.4.1. Mean Vector and Covariance Matrix
(2 of 2) [5/1/2006 10:35:37 AM]
Bivariate
normal
distribution
When p = 2, X = (X

1
,X
2
) has the bivariate normal distribution with a
two-dimensional vector of means, m = (m
1
,m
2
) and covariance matrix
The correlation between the two random variables is given by
6.5.4.2. The Multivariate Normal Distribution
(2 of 2) [5/1/2006 10:35:37 AM]
diagonal elements of S are the variances and the off-diagonal elements
are the covariances for the p variables. This is discussed further in
Section 6.5.4.3.1.)
Distribution
of T
2
It is well known that when =
0
with F
(p,n-p)
representing the F distribution with p degrees of freedom
for the numerator and n - p for the denominator. Thus, if
were
specified to be
0
, this could be tested by taking a single p-variate
sample of size n, then computing T
2

and comparing it with
for a suitably chosen .
Result does
not apply
directly to
multivariate
Shewhart-type
charts
Although this result applies to hypothesis testing, it does not apply
directly to multivariate Shewhart-type charts (for which there is no
0
), although the result might be used as an approximation when a
large sample is used and data are in subgroups, with the upper control
limit (UCL) of a chart based on the approximation.
Three-sigma
limits from
univariate
control chart
When a univariate control chart is used for Phase I (analysis of
historical data), and subsequently for Phase II (real-time process
monitoring), the general form of the control limits is the same for each
phase, although this need not be the case. Specifically, three-sigma
limits are used in the univariate case, which skirts the relevant
distribution theory for each Phase.
Selection of
different
control limit
forms for
each Phase
Three-sigma units are generally not used with multivariate charts,

however, which makes the selection of different control limit forms for
each Phase (based on the relevant distribution theory), a natural
choice.
6.5.4.3. Hotelling's T squared
(2 of 2) [5/1/2006 10:35:38 AM]
Estimating
the variances
and
covariances
The variances and covariances are similarly averaged over the
subgroups. Specifically, the s
ij
elements of the variance-covariance
matrix S are obtained as
with s
ijl
for i j denoting the sample covariance between variables X
i
and X
j
for the lth subgroup, and s
ij
for i = j denotes the sample variance
of X
i
. The variances (= s
iil
) for subgroup l and for variables i = 1, 2,
, p are computed as
.

Similarly, the covariances s
ijl
between variables X
i
and X
j
for subgroup
l are computed as
.
Compare T
2
against
control
values
As with an
chart (or any other chart), the k subgroups would be
tested for control by computing k values of T
2
and comparing each
against the UCL. If any value falls above the UCL (there is no lower
control limit), the corresponding subgroup would be investigated.
Formula for
plotted T
2
values
Thus, one would plot
for the jth subgroup (j = 1, 2, , k), with denoting a vector with p
elements that contains the subgroup averages for each of the p
characteristics for the jth subgroup. (
is the inverse matrix of the

"pooled" variance-covariance matrix,
, which is obtained by
averaging the subgroup variance-covariance matrices over the k
subgroups.)
Formula for
the upper
control limit
Each of the k values of
given in the equation above would be
compared with
6.5.4.3.1. T2 Chart for Subgroup Averages Phase I
(2 of 3) [5/1/2006 10:35:39 AM]
Lower
control limits
A lower control limit is generally not used in multivariate control chart
applications, although some control chart methods do utilize a LCL.
Although a small value for
might seem desirable, a value that is
very small would likely indicate a problem of some type as we would
not expect every element of
to be virtually equal to every element
in .
Delete
out-of-control
points once
cause
discovered
and corrected
As with any Phase I control chart procedure, if there are any points that
plot above the UCL and can be identified as corresponding to

out-of-control conditions that have been corrected, the point(s) should
be deleted and the UCL recomputed. The remaining points would then
be compared with the new UCL and the process continued as long as
necessary, remembering that points should be deleted only if their
correspondence with out-of-control conditions can be identified and the
cause(s) of the condition(s) were removed.
6.5.4.3.1. T2 Chart for Subgroup Averages Phase I
(3 of 3) [5/1/2006 10:35:39 AM]
Phase II
control
limits
with a denoting the number of the original subgroups that are deleted before
computing
and . Notice that the equation for the control limits for Phase II
given here does not reduce to the equation for the control limits for Phase I when a
= 0, nor should we expect it to since the Phase I UCL is used when testing for
control of the entire set of subgroups that is used in computing
and .
6.5.4.3.2. T2 Chart for Subgroup Averages Phase II
(2 of 2) [5/1/2006 10:35:42 AM]
Delete
points if
special
cause(s) are
identified
and
corrected
As in the case when subgroups are used, if any points plot outside these
control limits and special cause(s) that were subsequently removed can
be identified, the point(s) would be deleted and the control limits

recomputed, making the appropriate adjustments on the degrees of
freedom, and re-testing the remaining points against the new limits.
6.5.4.3.3. Chart for Individual Observations Phase I
(2 of 2) [5/1/2006 10:35:43 AM]
6. Process or Product Monitoring and Control
6.5. Tutorials
6.5.4. Elements of Multivariate Analysis
6.5.4.3. Hotelling's T squared
6.5.4.3.5.Charts for Controlling Multivariate
Variability
No
satisfactory
charts for
multivariate
variability
Unfortunately, there are no charts for controlling multivariate
variability, with either subgroups or individual observations, that are
simple, easy-to-understand and implement, and statistically defensible.
Methods based on the generalized variance have been proposed for
subgroup data, but such methods have been criticized by Ryan (2000,
Section 9.4) and some references cited therein. For individual
observations, the multivariate analogue of a univariate moving range
chart might be considered as an estimator of the variance-covariance
matrix for Phase I, although the distribution of the estimator is
unknown.
6.5.4.3.5. Charts for Controlling Multivariate Variability
[5/1/2006 10:35:43 AM]