LRA: Local Rigid Averaging of Stretchable Non-rigid Shapes

Abstract

We present a novel algorithm for generating the mean structure of non-rigid stretchable shapes. Following an alignment process, which supports local affine deformations, we translate the search of the mean shape into a diagonalization problem where the structure is hidden within the kernel of a matrix. This is the first step required in many practical applications, where one needs to model bendable and stretchable shapes from multiple observations.

Appendices

Appendix 1: Proof of Lemma 2

A useful Lemma for matrix derivation states that

$\nabla _A {\mathrm {Tr}} ABA^TC = CAB+C^TAB^T$.

Proof

We denote $f(A)=AB$, from which it follows that

$$\begin{aligned} \nabla _A {{\mathrm {Tr}}} ABA^TC= & {} \nabla _A {\mathrm {Tr}} f(A)A^TC \nonumber \\= & {} \nabla _{\tilde{A}} {\mathrm {Tr}} f(\tilde{A}) A^TC + \nabla _{\tilde{A}} {\mathrm {Tr}} f(A)\tilde{A} ^T C \nonumber \\= & {} (A^TC)^Tf'(\tilde{A}) + ( \nabla _{\tilde{A}} {\mathrm {Tr}} f(A)\tilde{A} ^T C )^T \nonumber \\= & {} C^T A B^T + (\nabla _ {\tilde{A} ^T} {\mathrm {Tr}} \tilde{A} ^T C f(A))^T \nonumber \\= & {} C^TAB^T + (Cf(A)^T)^T \nonumber \\= & {} C^TAB^T + CAB. \end{aligned}$$

(19)

$\square$

Appendix 2: Proof of Corollary 1

$$\begin{aligned} || P^T \varLambda P ||_F^2= & {} {\mathrm {Tr}} (P^T \varLambda P)(P^T \varLambda P)^T \nonumber \\= & {} {\mathrm {Tr}} ( P^T \varLambda P P^T \varLambda P ) \nonumber \\= & {} {\mathrm {Tr}} ( PP^T \varLambda (P^T P )^T \varLambda ). \end{aligned}$$

(20)

Using Lemma 2 and marking $PP^T=A$ and $B=C=\varLambda$ we see that

$$\begin{aligned} \nabla _{PP^T} || P^T \varLambda ^T P ||_F^2 = 2\varLambda PP^T \varLambda . \end{aligned}$$

(21)

Because $\nabla _ P PP^T = 2P$, we use the chain rule and infer that

$$\begin{aligned} \nabla _{P} || P^T \varLambda ^T P ||_F^2= & {} 2\varLambda PP^T \varLambda \nabla _P PP^T \nonumber \\= & {} 4\varLambda PP^T \varLambda P. \end{aligned}$$

(22)

Appendix 3: Proof of Corollary 2

Since

$$\begin{aligned} V_{ij}PP^T V_{ij}^T= & {} {\mathrm {Tr}} (V_{ij}PP^T V_{ij}^T) \\= & {} {\mathrm {Tr}} (PP^T V_{ij}^T V_{ij}), \end{aligned}$$

we can once again use Lemma 2. We denote $P=A$, B = ${\mathcal {I}}$, and C = $V_{ij}^T V_{ij}$, and infer that

$$\begin{aligned} \nabla _P V_{ij}PP^T V_{ij}^T = 2V_{ij}V_{ij}^T P. \end{aligned}$$

(23)

From the chain rule we readily have that

$$\begin{aligned}&\nabla _P \left( V_{ij}PP^T V_{ij}^T - d_{ij} \right) ^2 \nonumber \\&\quad =2 \left( V_{ij}PP^T V_{ij}^T - d_{ij}^2 \right) \nabla _P V_{ij}PP^T V_{ij}^T \nonumber \\&\quad = 4 \left( V_{ij}PP^T V_{ij}^T - d_{ij}^2 \right) V_{ij}V_{ij}^T P. \end{aligned}$$

(24)

Appendix 4: Aligning Stretchable Structures

We present a new alignment algorithm which is a merger of a traditional non-rigid alignment approach and a global intrinsic regularization which compensates for stretching. During the iterations we search for neighbors only for points which share similar intrinsic (long) distances from known anchor points. Since those distances were generated using a local metric which is invariant for stretching, this regularization is also invariant to such deformations, without the need to know a priori their location or strength. One should think about the non-rigid ICP procedure as a projection operator from space into the structure, while the intrinsic constraints are responsible for achieving an intrinsic alignment. In what follows we show how to generate long geodesics which are locally invariant to stretching. We first summarize known results on diffusion geometry, where given the Laplace Beltrami operator one can define long distances. Next we show how to build a local metric which is invariant to affine deformations and is used to empower the Laplace Beltrami operator with stretching invariant properties. This appendix can not replace the cited papers, but provides a descent summary to understand the steps required for alignment.

Spectral geometry: from local metric to global distances:

We model a shape as a complete two dimensional Riemannian manifold X with a metric tensor g, noted by (X, g). As we assume X is embedded into ${\mathbb {R}}^3$ by a regular map $\mathbf {x} : U \subseteq {\mathbb {R}}^2 \rightarrow {\mathbb {R}}^3$, the metric tensor can be expressed in coordinates as the coefficients of the first fundamental form $g_{ij} = \langle \frac{\partial \mathbf {x}}{\partial u_i} , \frac{\partial \mathbf {x} }{\partial u_j}\rangle$, where $u_i$ are the coordinates of U. Hence, the arc-length becomes $dp^2 = g_{11}{du_1}^2 + 2g_{12}du_1 du_2 + g_{22}{du_2}^2$.

Spectral geometry is based on the partial differential equation

$$\begin{aligned} \left( \frac{\partial }{\partial t} + \Delta _g \right) f(t,x) = 0, \end{aligned}$$

(25)

which is called the heat equation. It describes heat propagation, where f(t, x) is the heat distribution at a point x in time t , and $\Delta _g$ is the Laplace-Beltrami operator (LBO) which is evaluated from the local metric g. The fundamental solution of the heat equation is called the heat kernel and can be expressed using the spectral decomposition of $\varDelta _g$,

$$\begin{aligned} h_t(x,x') = \sum _{i\ge 0} e^{-\lambda _i t} \phi _i(x) \phi _i(x'), \end{aligned}$$

(26)

where $\phi _i$ and $\lambda _i$ are, respectively, the eigenfunctions and eigenvalues the $\varDelta _g$. Out of the heat kernel we can construct a family of intrinsic metrics known as diffusion metrics,

$$\begin{aligned} d^2_t(x,x')= & {} \int \left( h_t(x,\cdot ) - h_t(x',\cdot ) \right) ^2 da \nonumber \\= & {} \sum _{i> 0} e^{-2\lambda _i t} (\phi _i(x) - \phi _i(x'))^2, \end{aligned}$$

(27)

which measure the diffusion distance of the two points for a given time t. The parameter t can be considered as scale, and the family $\{ d_t \}$ can be thought of as a scale-space of metrics. Further details for numerically constructing a metric dependent Laplace-Beltrami operator can be found in Raviv et al. (2014) and Raviv and Kimmel (2015).

Locally affine metric invariants:

The seminal work of Blaschke (1923) and Su (1983) showed that if scaling is known, an equi-affine metric, also known as special-affine, can be constructed on a two dimensional (surface) manifold, where the equi-affine re-parametrization invariant metric (Su 1983; Blaschke 1923) reads $q_{ij} = |r|^{-\frac{1}{4}}r_{ij}$, where $r_{ij} = \det \left( \frac{\partial \mathbf {x}}{\partial u_1} , \frac{\partial \mathbf {x}}{\partial u_2} , \frac{\partial ^2 \mathbf {x}}{\partial u_1 \partial u_2} \right)$, and $r = \det (r_{ij}) = r_{11}r_{22}-r_{12}^2$. This scheme and its applications was recently examined in Raviv et al. (2014), where we turned this local pseudo metric into a practical tool.

In a parallel effort we studied scale invariant metrics using curvature for regularization (Aflalo et al. 2013). Specifically, consider the surface (X, g), then the scale invariant metric takes the form $\left| K \right| g_{ij}$, where K is the Gaussian curvature. We further showed how to normalize higher order non-rigid structures (Raviv and Raskar 2015), and showed its usefulness in medical volumetric data (e.g., Computed Tomography).

Surprisingly, it is possible to combine the two results, equi-affine and scaling, in one framework and removing the annoying equi limitation. First shown in Raviv and Kimmel (2015), one needs to replace the Gaussian curvature in the scale invariant metric definition with an equi-affine invariant curvature $K^q$, leading to $a_{ij} = \left| K^q \right| q_{ij}$, a locally affine invariant metric. Going back to diffusion distances, we need to replace the metric $g_{ij}$ with the metric $a_{ij}$ in order to evaluate affine invariant distances. Notice that the invariance is a local property leading to a global invariance in distance measures. One does not need to know a priori where and by how much stretching occurred. More details on the numeric construction can be found in Raviv and Kimmel (2015).