Applicability and Interpretability of Ward’s Hierarchical Agglomerative Clustering With or Without Contiguity Constraints

Abstract

Hierarchical agglomerative clustering (HAC) with Ward’s linkage has been widely used since its introduction by Ward (Journal of the American Statistical Association, 58(301), 236–244, 1963). This article reviews extensions of HAC to various input data and contiguity-constrained HAC, and provides applicability conditions. In addition, different versions of the graphical representation of the results as a dendrogram are also presented and their properties are clarified. We clarify and complete the results already available in an heterogeneous literature using a uniform background. In particular, this study reveals an important distinction between a consistency property of the dendrogram and the absence of crossover within it. Finally, a simulation study shows that the constrained version of HAC can sometimes provide more relevant results than its unconstrained version despite the fact that the constraint leads to optimize the objective criterion on a reduced set of solutions at each step. Overall, this article provides comprehensive recommendations, both for the use of HAC and constrained HAC depending on the input data, and for the representation of the results.

Appendices

Appendix 1. Proof of Proposition 2

Proof of Proposition 2

We begin by noting that by Proposition 1, the only reversals that may occur are crossovers. With the notation of Proposition 2, a crossover at step t + 1 corresponds to the situation where:

$$ \begin{array}{@{}rcl@{}} \delta(G_{l} , G_{r})\geq \delta(G_{l} \cup G_{r}, G_{\bar{r}}) \textrm{ or } \delta(G_{l} , G_{r})\geq \delta(G_{l} \cup G_{r}, G_{\bar{l}}). \end{array} $$

By symmetry, we focus on the first case. With the notation of Proposition 2, and using the Lance-Willams formula (4), the first condition is equivalent to:

$$ \begin{array}{@{}rcl@{}} \delta(G_{l}, G_{r}) \geq \frac{g_{lr'} \delta(G_{l}, G_{\bar{r}}) + g_{rr^{\prime}} \delta (G_{r}, G_{\bar{r}})}{g_{lr^{\prime}} + g_{rr^{\prime}}} \end{array} $$

while the second one is equivalent to:

$$ \begin{array}{@{}rcl@{}} \delta(G_{l}, G_{r}) \geq \frac{g_{\bar{l}l} \delta(G_{\bar{l}}, G_{l}) + g_{\bar{l}r} \delta (G_{\bar{l}}, G_{r})}{g_{\bar{l}l} + g_{\bar{l}r}} \end{array} $$

hence the result. □

Appendix 2. Step-by-step Description of the Counter-Examples

In the following tables, Bold values are used to signal reversals. Italic values in Table 3 are used to highlight the value of the objective function (ESS_t) for the clustering with 3 clusters.

Table 2 Details of Fig. 1

Table 3 Details of Fig. 2

Table 4 Details of Fig. 3

Table 5 Details of Fig. 4

Table 6 Details of Fig. 11

Appendix 3. Counter-Example of the Monotonicity of $\bar {I}_{t}$ for Standard HAC in the Euclidean Case

Fig. 11

A reversal for Euclidean standard HAC with height defined as $\bar {I}_{t}$. Top left: Configuration of the objects in $\mathbb {R}^{2}$. Top right: Coordinates of the objects and Euclidean distance matrix corresponding to this configuration. Bottom left: Representation of the values of the dissimilarity (dark colors correspond to larger values, so distant objects). Bottom right: dendrogram obtained from standard HAC. Only the first 3 merges of the dendrogram are represented to ensure a comprehensive view of the sequence of heights