Invariant coordinate selection (ICS) is a multivariate statistical technique used to identify interesting structures in high-dimensional data. It is commonly applied in outlier detection, independent component analysis (ICA), and robust dimension reduction. ICS generalizes principal component analysis (PCA) by simultaneously diagonalizing two scatter matrices, typically the sample covariance matrix and a higher-order or robust scatter matrix.
Overview Unlike PCA, which depends on the scale of the variables and captures only second-order structure, ICS is invariant under full-rank affine transformations of the data and can reveal non-Gaussian features of the underlying distribution. The core idea is to find a linear transformation that simultaneously diagonalizes two scatter matrices, yielding coordinates whose ordering reflects the disagreement between the two scatter measures. The method draws heavily on the robust statistics literature for its repertoire of scatter functionals.
History The simultaneous use of two scatter matrices for data transformation predates ICS and originated in the clustering literature. Art, Gnanadesikan and Kettenring (1982) introduced a data-based metric for cluster analysis in which an estimate of the within-cluster covariance matrix is computed iteratively without knowledge of the cluster labels, and the data are then rescaled by its inverse square root. This procedure, implemented as PROC ACECLUS in SAS, is generally regarded as the earliest instance of the simultaneous-diagonalization idea later generalized by ICS. Caussinus and Ruiz-Gazen (1990) developed a closely related generalized principal component analysis for projection pursuit and cluster identification. In parallel, the same algebraic structure was being investigated in signal processing under the headings of independent component analysis and blind source separation. Cardoso's Fourth-Order Blind Identification (FOBI) jointly diagonalizes the covariance matrix and a fourth-moment scatter matrix to recover independent sources. Oja and collaborators subsequently showed that any pair of affine-equivariant scatter matrices possessing the so-called independence property can be used to recover an independent component model, placing FOBI and related procedures in a common statistical framework. A third influence comes from the robust statistics community, where most of the alternative scatter functionals later adopted by ICS were originally introduced—including the M-estimators of multivariate scatter studied by Maronna, the distribution-free M-estimator of Tyler, the symmetrized M-estimator of Dümbgen, and the minimum covariance determinant (MCD) estimator of Rousseeuw. These threads were unified by Tyler, Critchley, Dümbgen and Oja (2009), who introduced the name invariant coordinate selection, characterized its affine-invariance properties, and established it as a general-purpose exploratory method that subsumes the earlier proposals as special cases.
Definition Let X = ( x 1 , … , x n ) T ∈ R n × p {\displaystyle X=(x_{1},\dots ,x_{n})^{T}\in \mathbb {R} ^{n\times p}} be a sample of n {\displaystyle n} observations in R p {\displaystyle \mathbb {R} ^{p}} , with sample mean x ¯ {\displaystyle {\bar {x}}} . A scatter matrix is a p × p {\displaystyle p\times p} symmetric positive-definite matrix-valued functional S ( X ) {\displaystyle S(X)} of the data that is affine equivariant, meaning that for any non-singular matrix A {\displaystyle A} and vector b {\displaystyle b} ,
S ( X A + 1 n b T ) = A T S ( X ) A . {\displaystyle S(XA+\mathbf {1} _{n}b^{T})=A^{T}\,S(X)\,A.}
The sample covariance matrix is the canonical example, but many alternatives drawn from the robust statistics literature are available (see below). Given two scatter matrices S 1 ( X ) {\displaystyle S_{1}(X)} and S 2 ( X ) {\displaystyle S_{2}(X)} , ICS solves the generalized eigenvalue problem
S 2 ( X ) v = λ S 1 ( X ) v , {\displaystyle S_{2}(X)\,\mathbf {v} =\lambda \,S_{1}(X)\,\mathbf {v} ,}
… excerpt ends here. Continue reading the full article.
