Cluster Evaluation and Validation

Preface

A clustering algorithm always returns a grouping when its settings require one. The result does not tell us whether three groups are preferable to four, whether the same members would stay together after a small change to the data, or whether another algorithm found the same partition. Those are evaluation questions, and each requires different evidence.

This module separates three questions from the beginning:

1. Separation and resolution. How many compact, separated groups are supported under the chosen representation and distance? Silhouette analysis and the Gap statistic address this question. Consensus clustering and PAC can also compare candidate resolutions through resampling.
2. Stability. Do similar memberships recur when observations are resampled? Consensus values and bootstrap Jaccard similarity address this question.
3. Agreement. Do two completed partitions of the same observations agree? Adjusted Rand index and adjusted mutual information address this question. Normalized mutual information is included because it is common in the literature.

Start with Understanding cluster evaluation.