Soft Independent Modeling of Class Analogy (SIMCA): An Overview

Soft Independent Modeling of Class Analogy (SIMCA) is a multivariate classification technique based on Principal Component Analysis (PCA). Developed in the 1970s by Swedish chemometrician Svante Wold, SIMCA has since become one of the most well-established methods for pattern recognition, classification, and data modeling in various fields, including chemistry, biology, and industrial quality control. Its key appeal lies in its ability to classify complex, high-dimensional data, especially in situations where linear relationships between variables are difficult to discern.

In essence, SIMCA builds individual models for each class in a dataset by performing PCA on each class separately. These PCA models then represent the geometry of each class in the high-dimensional space of the data. When an unknown sample needs to be classified, SIMCA compares it to the PCA models of all the classes, assigning the sample to the class to which it best fits. The “soft” aspect of SIMCA refers to the fact that it allows for overlap between class boundaries, meaning that an observation can be classified into more than one class, or potentially none at all, if it does not fit within any model.

The Key Principles of SIMCA

SIMCA revolves around two fundamental concepts: PCA and class modeling. Here’s a breakdown of how these principles work together:

1. Principal Component Analysis (PCA):

At the core of SIMCA is PCA, a dimensionality-reduction technique that transforms a large set of variables into a smaller one that still contains most of the variance in the data. In SIMCA, PCA is applied separately to each class of data to capture the main sources of variation within that class.

For each class, PCA generates a model that describes the structure of the data using a few principal components. These principal components are orthogonal to each other and are linear combinations of the original variables. The goal of PCA is to explain as much of the variance within each class as possible with a minimal number of components.

2. Class Modeling:

Once PCA is applied to a class, SIMCA builds a model for that class in the reduced space defined by the principal components. Each class model has its own set of components and is designed to capture the unique variance of the data within that class.

When a new sample is introduced for classification, SIMCA projects it onto each class model to determine how well it fits within that class. The fit is determined by measuring the sample’s distance from the model in the principal component space. SIMCA uses two types of distances to make this decision:

• Residual distance: This measures how far the sample is from the PCA model in terms of the components not captured by the model (i.e., the variation left unexplained by the PCA model).
• Score distance: This measures the distance of the sample from the center of the class in the PCA space, using the retained principal components.

How SIMCA Works

The SIMCA algorithm can be broken down into several key steps:

Step 1: Data Preparation

The data consists of several classes, with each class corresponding to a different category or group (e.g., different types of wines, chemical compounds, or industrial batches). Before applying SIMCA, the data is centered and scaled, typically using mean centering and unit variance scaling, so that all variables contribute equally to the model.

Step 2: Building Class Models

For each class in the dataset, SIMCA builds a PCA model. This involves:

• Performing PCA on the data of the given class to reduce its dimensionality.
• Retaining a number of principal components that explain most of the variance within that class. The number of components is often determined by cross-validation.
• Constructing a model that describes the geometry of the class in the reduced space of the retained principal components.

Step 3: Classifying New Samples

When a new sample needs to be classified, SIMCA evaluates how well the sample fits within each class model:

• The sample is projected onto each class model to obtain its position in the principal component space (its scores) and its residual distance from the model.
• The residual and score distances are then compared to pre-defined thresholds. These thresholds are typically set using confidence limits derived from the training data.

Step 4: Making a Decision

Based on the sample’s distances from each class model, SIMCA determines:

• Whether the sample belongs to a given class: If the sample’s residual and score distances are both within the threshold for a class, the sample is classified as belonging to that class.
• If the sample is an outlier: If the sample’s residual or score distances exceed the thresholds for all classes, the sample is classified as an outlier.
• If the sample belongs to more than one class: If the sample fits within the thresholds of multiple class models, SIMCA may classify it as belonging to more than one class, reflecting the “soft” nature of the classification.

SIMCA and Multiclass Classification

Unlike other classification methods such as Linear Discriminant Analysis (LDA) or Support Vector Machines (SVM), SIMCA models each class independently. This “one-class-at-a-time” approach makes SIMCA particularly well-suited for situations where the classes are not linearly separable or where they overlap in feature space.

Because SIMCA allows for soft boundaries, it is more tolerant of overlapping classes than methods like LDA, which require that classes be linearly separable. SIMCA’s soft boundaries also mean that it can identify samples that do not belong to any class (outliers), making it useful for anomaly detection.

Mathematical Details of SIMCA

SIMCA relies on the geometric representation of each class in terms of its principal components. Mathematically, this is achieved as follows:

1. PCA for Each Class

Given a dataset $X$ for a class, SIMCA applies PCA to decompose the dataset as:

X=TPT+EX = TP^T + EX=TPT+E

Where:

• $T$ is the score matrix, representing the projection of $X$ onto the principal components.
• $P$ is the loading matrix, representing the directions of the principal components in the original space.
• $E$ is the residual matrix, representing the part of the data that is not explained by the principal components.

2. Score and Residual Distances

For each new sample xnewx, SIMCA computes two distances:

• The score distance measures how far the sample’s projection tnew​ is from the center of the class in the principal component space. This distance is used to determine how typical the sample is relative to the class.
• The residual distance measures how far the sample is from the PCA model in the original space. This distance captures how much of the sample’s variation is not explained by the principal components of the class.

3. Thresholds and Classification

Thresholds for both the score and residual distances are set using the training data for each class. These thresholds correspond to confidence limits (typically 95%) and are used to classify new samples.

If both the score and residual distances for a new sample are within the thresholds for a class, the sample is classified as belonging to that class. If the sample exceeds the thresholds for all classes, it is classified as an outlier.

Applications of SIMCA

SIMCA is widely used in various fields where multivariate data is prevalent:

1. Chemometrics: In chemistry and chemical engineering, SIMCA is used to classify chemical compounds, quality control in manufacturing, and spectroscopic data analysis.
2. Food Industry: SIMCA helps classify food products based on their chemical composition, ensuring product authenticity and quality.
3. Pharmaceutical Industry: SIMCA is used for drug classification and quality control in pharmaceutical manufacturing processes.
4. Biotechnology: SIMCA finds applications in genomics and proteomics for classifying biological samples, such as different cell types or disease states, based on gene expression or protein data.

Advantages and Limitations

Advantages:

• Handles Multivariate and High-Dimensional Data: SIMCA excels at handling datasets with many variables, where traditional classification methods may struggle.
• Independent Class Models: Each class is modeled separately, making SIMCA flexible and adaptable to various types of data.
• Soft Classification: SIMCA allows for samples to belong to multiple classes or none at all, making it useful for identifying outliers and anomalies.

Limitations:

• Complexity: SIMCA’s reliance on PCA models for each class can make it computationally intensive, particularly for large datasets.
• Interpretability: While PCA helps reduce dimensionality, interpreting the principal components and understanding the classification boundaries can be challenging.

Soft Independent Modeling of Class Analogy (SIMCA) is a powerful and versatile classification technique, particularly well-suited for handling complex, multivariate datasets. By building independent PCA models for each class, SIMCA allows for flexible classification, accommodating overlapping classes and identifying outliers. Its applications in chemometrics, pharmaceuticals, food analysis, and biotechnology demonstrate its broad utility, making it a valuable tool in fields that require robust multivariate classification methods.