A Numerical Framework for Elastic Surface Matching, Comparison, and Interpolation

Abstract

Surface comparison and matching is a challenging problem in computer vision. While elastic Riemannian metrics provide meaningful shape distances and point correspondences via the geodesic boundary value problem, solving this problem numerically tends to be difficult. Square root normal fields considerably simplify the computation of certain distances between parametrized surfaces. Yet they leave open the issue of finding optimal reparametrizations, which induce corresponding distances between unparametrized surfaces. This issue has concentrated much effort in recent years and led to the development of several numerical frameworks. In this paper, we take an alternative approach which bypasses the direct estimation of reparametrizations: we relax the geodesic boundary constraint using an auxiliary parametrization-blind varifold fidelity metric. This reformulation has several notable benefits. By avoiding altogether the need for reparametrizations, it provides the flexibility to deal with simplicial meshes of arbitrary topologies and sampling patterns. Moreover, the problem lends itself to a coarse-to-fine multi-resolution implementation, which makes the algorithm scalable to large meshes. Furthermore, this approach extends readily to higher-order feature maps such as square root curvature fields and is also able to include surface textures in the matching problem. We demonstrate these advantages on several examples, synthetic and real.

Appendices

A Notation

Throughout this section, $h \in T_q\mathcal {I}=C^\infty (M,{\mathbb {R}}^3)$ denotes a tangent vector to $q\in \mathcal {I}$, and X, Y are vector fields on M. Traces are denoted by ${{\,\mathrm{Tr}\,}}$ or a dot, T is the tangent functor, and D stands for directional derivatives. For instance, the derivative at q in the direction of h is denoted by $D_{(q,h)}$. We write ${\overline{g}}=\langle \cdot ,\cdot \rangle$ for the Euclidean inner product on ${\mathbb {R}}^3$, $|\cdot |$ for the Euclidean norm on ${\mathbb {R}}^3$, $g_q=q^*{\overline{g}}$ for the pull-back metric on TM, $g_q^{-1}$ for the cometric on $T^*M$, and $g_q^{-1}\otimes {\overline{g}}$ for the product metric on $T^*M\otimes {\mathbb {R}}^3$. The metric $g_q$ corresponds to a fiber-linear map $\flat$ from TM to $T^*M$, and the cometric $g_q^{-1}$ corresponds to a fiber-linear map $\sharp$ from $T^*M$ to M. The Riemannian surface measure of $g_q$ is denoted by $A_q$, and the corresponding half-density by $A_q^{1/2}$. The surface measure is a section of the volume bundle $\mathrm {Vol}$, and the half-density of the half-density bundle $\mathrm {Vol}^{1/2}$. The normal projection $\bot :M\rightarrow L({\mathbb {R}}^3,{\mathbb {R}}^3)$ is defined as $\bot =\langle \cdot ,n_q\rangle n_q$, the tangential projection $\top :M\rightarrow L({\mathbb {R}}^3,TM)$ is defined as $\top =(Tq)^{-1}({\text {Id}}_{{\mathbb {R}}^3}-\bot )$, and one has the identity $\bot +Tq\circ \top ={{\,\mathrm{Id}\,}}_{{\mathbb {R}}^3}$. Depending on the context, $\nabla$ is the covariant derivative on ${\mathbb {R}}^3$, which coincides with the usual coordinate derivative, or the Levi-Civita covariant derivative of $g_q$. For instance, in the definition $\nabla ^2_{X,Y}h:=\nabla _X\nabla _Yh-\nabla _{\nabla _XY}h$, only $\nabla _XY$ is the Levi-Civita covariant derivative, and all other derivatives are coordinate derivatives.

B Formula for SRNF Metrics

In this section we establish the explicit formula (14) for the SRNF metric. We need some variational formulas from e.g. Bauer et al. (2012):

$$\begin{aligned} D_{(q,h)}A_q =&{{\,\mathrm{Tr}\,}}\big (g_q^{-1}.\langle \nabla h,Tq\rangle \big )A_q = {{\,\mathrm{Tr}\,}}\big ((\nabla h)^\top \big )\;, \\ D_{(q,h)}A_q^{1/2} =&\tfrac{1}{2}{{\,\mathrm{Tr}\,}}\big ((\nabla h)^\top \big )A_q^{1/2}\;, \\ D_{(q,h)}n_q =&-Tq\circ \langle n_q,\nabla h\rangle ^\sharp \;, \\ D_{(q,h)}N_q {=}&\Big (-Tq\circ \langle n_q,\nabla h\rangle ^\sharp +n_q \tfrac{1}{2}{{\,\mathrm{Tr}\,}}\big ((\nabla h)^\top \big )\Big )A_q^{1/2}\;. \end{aligned}$$

Putting these together, one obtains the following expression for the SRNF metric (13):

$$\begin{aligned} G_q(h,h) :=&\int _M |D_{(q,h)}N_q|^2 \\=&\int _M|Tq\circ \langle n_q,\nabla h\rangle ^\sharp |^2A_q \\ {}&+ \int _M|n_q \tfrac{1}{2}{{\,\mathrm{Tr}\,}}\big ((\nabla h)^\top \big )|^2A_q \\=&\int _M|\langle n_q,\nabla h\rangle |_{g_q^{-1}}^2A_q + \frac{1}{4}\int _M{{\,\mathrm{Tr}\,}}\big ((\nabla h)^\top \big )^2A_q \\=&\int _M|(\nabla h)^\bot |_{g_q^{-1}\otimes \overline{g}}^2A_q + \frac{1}{4}\int _M{{\,\mathrm{Tr}\,}}\big ((\nabla h)^\top \big )^2A_q\;. \end{aligned}$$

C Approximation Properties of SRNF Distances

This section makes precise in what sense the SRNF distance approximates the geodesic distance of the SRNF metric. The geodesic distance of the SRNF metric is lower-bounded by the SRNF distance because the latter is a chordal distance:

$$\begin{aligned} \mathrm {dist}_{\mathcal {I}}(q_0,q_1)^2 :=&\inf _q \int _0^1 G_q(\partial _t q,\partial _t q) dt \\=&\inf _q \int _0^1 |D_{(q,\partial _t q)}N_q|^2 dt {\ge } \Vert N_{q_1}-N_{q_0}\Vert _{L^2}^2\;, \end{aligned}$$

where the infimum is over all paths q as in (11). Conversely, the geodesic distance of the SRNF metric is upper-bounded by the length of the linear interpolation between the immersions $q_0$ and $q_1$, provided they are sufficiently close to each other for this to make sense, leading to the upper bound

$$\begin{aligned} \mathrm {dist}_{\mathcal {I}}(q_0,q_1)^2&\le G_{q_0}(q_1-q_0,q_1-q_0) \\ {}&= \Vert D_{(q_0,q_1-q_0)}N_{q_0}\Vert _{L^2}^2 \approx \Vert N_{q_1}-N_{q_0}\Vert _{L^2}^2\;, \end{aligned}$$

which is valid in any chart around $q_0$ up to terms of order $o(\Vert N_{q_1}-N_{q_0}\Vert _{L^2}^2)$.

D Formula for SRCF Metrics

Recall that the scalar and vector-valued second fundamental forms are defined as

$$\begin{aligned} s_q(X,Y)&{:=} \langle \nabla _X(Tq\circ Y),n_q\rangle \;,&S_q(X,Y)&{:=} s_q(X,Y)n_q\;, \end{aligned}$$

for any vector fields X and Y on M. Following Bauer et al. (2012), one obtains the variational formulas

$$\begin{aligned}&D_{(q,h)} s_q(X,Y) = \langle D_{(q,h)}\nabla _X(Tq\circ Y),n_q\rangle \\ {}&\qquad +\langle \nabla _X(Tq\circ Y),D_{(q,h)}n_q\rangle \\ {}&\quad = \langle \nabla _X\nabla _Yh,n_q\rangle -\langle \nabla _X(Tq\circ Y),Tq\circ \langle \nabla h,n_q\rangle ^\sharp \rangle \\ {}&\quad = \langle \nabla _X\nabla _Yh,n_q\rangle -g_q( \nabla _XY,\langle \nabla h,n_q\rangle ^\sharp ) \\ {}&\quad = \langle \nabla _X\nabla _Yh,n_q\rangle -\langle \nabla _{\nabla _XY} h,n_q\rangle = \langle \nabla ^2_{X,Y}h,n_q\rangle \;, \\&\quad D_{(q,h)} S_q(X,Y) = (D_{(q,h)} s_q(X,Y))n_q + s_q(X,Y)(D_{(q,h)} n_q) \\ {}&\quad = (\nabla ^2_{X,Y}h)^\bot -s_q(X,Y)Tq\circ \langle \nabla h,n_q\rangle ^\sharp \;. \end{aligned}$$

Using the formula $\Delta _q=-{{\,\mathrm{Tr}\,}}(g_q^{-1}\nabla ^2)$ for the Laplacian, this yields the following variational formula for the vector-valued mean curvature $H_q={{\,\mathrm{Tr}\,}}(g_q^{-1}S_q)$:

$$\begin{aligned} D_{(q,h)}g_q&= D_{(q,h)}\langle Tq,Tq\rangle = \langle \nabla h,Tq\rangle + \langle Tq,\nabla h\rangle \;, \\ D_{(q,h)}g_q^{-1}&= -g_q^{-1}(D_{(q,h)}g_q)g_q^{-1}\;, \\ D_{(q,h)}H_q&= {{\,\mathrm{Tr}\,}}(g_q^{-1}D_{(q,h)}S_q) + {{\,\mathrm{Tr}\,}}(S_qD_{(q,h)}g_q^{-1}) \\ {}&= -(\Delta _q h)^\bot -{{\,\mathrm{Tr}\,}}(g_q^{-1}s_q)Tq\circ \langle \nabla h,n_q\rangle ^\sharp \\ {}&\qquad \quad -2{{\,\mathrm{Tr}\,}}(S_qg_q^{-1}\langle \nabla h,Tq\rangle g_q^{-1})\;. \end{aligned}$$

Letting $C(\nabla h)$ denote the first-order terms in $-D_{(q,h)}H_q$, one obtains the desired formula for the SRCF metric:

$$\begin{aligned} G_q(h,h)&:=\int _M|D_{(q,h)}H_q|^2A_q = \int _M |(\Delta _qh)^\bot +C(\nabla h)|^2A_q\;. \end{aligned}$$