The Gross–Prasad conjecture and local theta correspondence

Abstract

We establish the Fourier–Jacobi case of the local Gross–Prasad conjecture for unitary groups, by using local theta correspondence to relate the Fourier–Jacobi case with the Bessel case established by Beuzart-Plessis. To achieve this, we prove two conjectures of Prasad on the precise description of the local theta correspondence for (almost) equal rank unitary dual pairs in terms of the local Langlands correspondence. The proof uses Arthur’s multiplicity formula and thus is one of the first examples of a concrete application of this “global reciprocity law”.

Appendices

Appendix A: Addendum to [17]

In this appendix, we elaborate on some results of [17, Appendix C] which are used in the proof of Theorem 4.4. In particular,

• we fill in some missing details in the proof of [17, Proposition C.1(ii)] and streamline its proof by exploiting the recently established Howe duality conjecture [20, 21];

• we extend some results of Muić [45, Lemma 4.2 and Theorem 5.1(i)] (used in the proof of [17, Proposition C.1(ii)]), which were written only for symplectic-orthogonal dual pairs, to cover all dual pairs considered in [17], streamlining some of his proofs in the process.

1.1 A.1 The issues

Let us be more precise. We freely use the notation of [17, Sect. C.1].

Let $\pi$ be an irreducible square-integrable representation of G(W) such that

$$\begin{aligned} \sigma _0 := \varTheta _{\tilde{V},W,\varvec{\chi },\psi }(\pi ) \ne 0. \end{aligned}$$

By the bullet point on [17, p. 645], together with the Howe duality, $\sigma _0$ is irreducible and square-integrable. Then we showed that

• (i) any irreducible subquotient of $\varTheta _{V,W,\varvec{\chi },\psi }(\pi )$ is tempered in the first bullet point on [17, p. 646];

• (ii) $\sigma := \theta _{V,W,\varvec{\chi },\psi }(\pi )$ is an irreducible constituent of $I^{H(V)}_{Q(Y_1)}(\chi _W \otimes \sigma _0)$ in the the second bullet point on [17, p. 646],

and claimed that

• (iii) any irreducible subquotient of $\varTheta _{V,W,\varvec{\chi },\psi }(\pi )$ is not square-integrable in the third bullet point on [17, p. 646];

• (iv) any irreducible subquotient of $\varTheta _{V,W,\varvec{\chi },\psi }(\pi )$ is a subrepresentation of $I^{H(V)}_{Q(Y_1)}(\chi _W \otimes \sigma _0')$ for some irreducible smooth representation $\sigma _0'$ of $H(\tilde{V})$ in the fourth bullet point on [17, p. 646].

However, in the third and fourth bullet points on [17, p. 646], we have used results of Muić [45, Lemma 4.2 and Theorem 5.1(i)], which were written only for symplectic-orthogonal dual pairs. Moreover, we have not given the proof of (iv): we have simply asserted that it is true as if it is obvious (which it is not). Thus, we need to give the details of the proof of (iii) and (iv), as well as that of the results of Muić for all dual pairs considered in [17].

1.2 A.2 Proof of (iii)

First, we address (iii). Our original argument in [17] used [45, Lemma 4.2 and Theorem 5.1(i)], which we state and prove in Lemma A.1 and Corollary A.5 below. Here, we give a more streamlined argument using the recently established Howe duality conjecture [20, 21].

Let $\sigma '$ be an irreducible subquotient of $\varTheta _{V,W,\varvec{\chi },\psi }(\pi )$. Suppose that $\sigma '$ is square-integrable. Since $\varTheta _{V,W,\varvec{\chi },\psi }(\pi )$ is of finite length and tempered by (i), it follows by [60, Corollaire III.7.2] that $\sigma '$ is in fact a quotient of $\varTheta _{V,W,\varvec{\chi },\psi }(\pi )$. Hence we must have $\sigma ' \cong \sigma$ by the Howe duality. But $\sigma$ is not square-integrable by (ii), which is a contradiction. This completes the proof of (iii).

1.3 A.3 Proof of [45, Lemma 4.2]

For the proof of (iv), we will need the following result of Muić [45, Lemma 4.2].

Lemma A.1

(Muić) Let $G(W) \times H(V)$ be an arbitrary reductive dual pair as in [17, Sect. 3]. Let $\pi$ be an irreducible smooth representation of G(W). Then all irreducible subquotients of $\varTheta _{V,W,\varvec{\chi },\psi }(\pi )$ have the same supercuspidal support.

Proof

We may assume that $\varTheta _{V,W,\varvec{\chi },\psi }(\pi ) \ne 0$. Since $\varTheta _{V,W,\varvec{\chi },\psi }(\pi )$ is of finite length, it follows by the theory of the Bernstein center [3] that

$$\begin{aligned} \varTheta _{V,W,\varvec{\chi },\psi }(\pi ) = \sigma _1 \oplus \dots \oplus \sigma _r \end{aligned}$$

for some smooth representations $\sigma _i$ of H(V) of finite length such that

• for each i, all irreducible subquotients of $\sigma _i$ have the same supercuspidal support, say, ${\text {supp}}\sigma _i$;

• if $i \ne j$, then ${\text {supp}}\sigma _i \ne {\text {supp}}\sigma _j$.

Of course, if we were willing to appeal to the Howe duality, then it would follow immediately that $r=1$, so that the lemma is proved. However, we may appeal to an older result of Kudla. Namely, Kudla’s supercuspidal support theorem [36] (see also [17, Proposition 5.2] and the references therein) says that the supercuspidal support of $\theta _{V,W,\varvec{\chi },\psi }(\pi )$ is determined by that of $\pi$. Hence we must have $r=1$. $\square$

1.4 A.4 Plancherel measures

To prove (iv), we will also need the following property of Plancherel measures. We freely use the convention of [17, Appendix B].

Lemma A.2

Let G(W) be an arbitrary classical group as in [17, Sect. 2]. Let $\pi$ be an irreducible tempered representation of G(W) such that

$$\begin{aligned} \pi \subset I^{G(W)}_P(\tau _1 \otimes \dots \otimes \tau _r \otimes \pi _0), \end{aligned}$$

where P is a parabolic subgroup of G(W) with Levi component $\mathrm {GL}_{k_1}(E) \times \dots \times \mathrm {GL}_{k_r}(E) \times G(W_0)$, $\tau _i$ is an irreducible (unitary) square-integrable representation of $\mathrm {GL}_{k_i}(E)$, and $\pi _0$ is an irreducible square-integrable representation of $G(W_0)$. Let $\tau$ be an irreducible (unitary) square-integrable representation of $\mathrm {GL}_k(E)$ and put

$$\begin{aligned} \mathcal {I}(\tau ) = \{ i \, | \, \tau _i \cong \tau \}. \end{aligned}$$

Then we have

$$\begin{aligned} \mathop {\mathrm{ord}}\limits _{s=0} \mu (\tau _s \otimes \pi ) = 2 \cdot \# \mathcal {I}(\tau ) + 2 \cdot \# \mathcal {I}((\tau ^c)^\vee ) + \mathop {\mathrm{ord}}\limits _{s=0} \mu (\tau _s \otimes \pi _0). \end{aligned}$$

Moreover, we have

$$\begin{aligned} \mathop {\mathrm{ord}}\limits _{s=0} \mu (\tau _s \otimes \pi _0) = {\left\{ \begin{array}{ll} 0\text { or } 2 &{}\quad {\text { if } (\tau ^c)^\vee \cong \tau ;} \\ 0 &{} \quad {\text { if } (\tau ^c)^\vee \ncong \tau .} \end{array}\right. } \end{aligned}$$

Proof

By the multiplicativity of Plancherel measures (see [17, Sect. B.5]), we have

$$\begin{aligned} \mu (\tau _s \otimes \pi ) = \left( \prod _{i=1}^r \mu (\tau _s \otimes \tau _i) \cdot \mu (\tau _s \otimes (\tau _i^c)^\vee ) \right) \cdot \mu (\tau _s \otimes \pi _0). \end{aligned}$$

For any irreducible (unitary) square-integrable representation $\tau '$ of $\mathrm {GL}_{k'}(E)$, we have

$$\begin{aligned} \mu (\tau _s \otimes \tau ') = \gamma (s, \tau \times (\tau ')^\vee , \psi _E) \cdot \gamma (-s, \tau ^\vee \times \tau ', \bar{\psi }_E) \end{aligned}$$

and hence

$$\begin{aligned} \mathop {\mathrm{ord}}\limits _{s=0} \mu (\tau _s \otimes \tau ') = {\left\{ \begin{array}{ll} 2 &{} {\text { if } \tau \cong \tau ';} \\ 0 &{} {\text { if } \tau \ncong \tau ',} \end{array}\right. } \end{aligned}$$

which reflects the triviality of R-groups for general linear groups. This proves the first assertion. The second assertion follows from [60, Corollaire IV.1.2] if $(\tau ^c)^\vee \cong \tau$ and [60, Proposition IV.2.2] if $(\tau ^c)^\vee \ncong \tau$. $\square$

1.5 A.5 Proof of (iv)

Now we prove (iv). Let $\sigma '$ be an irreducible subquotient of $\varTheta _{V,W,\varvec{\chi },\psi }(\pi )$. By (i) and (iii), we have

$$\begin{aligned} \sigma ' \subset I^{H(V)}_Q(\tau _1 \otimes \dots \otimes \tau _r \otimes \sigma _0') \end{aligned}$$

for some $r \ge 1$ and irreducible square-integrable representations $\tau _i$ and $\sigma _0'$ of $\mathrm {GL}_{k_i}(E)$ and $H(V_0)$ respectively, where Q is a parabolic subgroup of H(V) with Levi component $\mathrm {GL}_{k_1}(E) \times \dots \times \mathrm {GL}_{k_r}(E) \times H(V_0)$. We need to show that $\tau _i = \chi _W$ for some i.

By Lemma A.1 and the multiplicativity of Plancherel measures, we have

$$\begin{aligned} \mu ((\chi _W)_s \otimes \sigma ') = \mu ((\chi _W)_s \otimes \sigma ). \end{aligned}$$

By (ii) and Lemma A.2, the right-hand side has a zero at $s=0$ of order at least 4. Hence, by Lemma A.2again, we must have $\tau _i = \chi _W$ for some i. This completes the proof of (iv).

Remark A.1

In the proof of (iii) and (iv), we have used some results of Waldspurger [60], which were written only for connected reductive linear algebraic groups. However, it is straightforward to extend them to the cases of (disconnected) orthogonal groups and (nonlinear) metaplectic groups.

1.6 A.6 Proof of [45, Theorem 5.1(i)]

As we noted above, we have used [45, Theorem 5.1(i)] besides [45, Lemma 4.2] in our original argument in [17]. Although it is not necessary for the proof of (iii) and (iv) (because of the use of the Howe duality), we shall give a proof here. In fact, we prove the following more general result by refining the argument in the proof of (iv).

Lemma A.4

Let G(W) be an arbitrary classical group as in [17, Sect. 2]. Let $\pi$ be an irreducible tempered representation of G(W) such that

$$\begin{aligned} \pi \subset I^{G(W)}_P(\tau _1 \otimes \dots \otimes \tau _r \otimes \pi _0), \end{aligned}$$

where P is a parabolic subgroup of G(W) with Levi component $\mathrm {GL}_{k_1}(E) \times \dots \times \mathrm {GL}_{k_r}(E) \times G(W_0)$, $\tau _i$ is an irreducible (unitary) square-integrable representation of $\mathrm {GL}_{k_i}(E)$, and $\pi _0$ is an irreducible square-integrable representation of $G(W_0)$. Likewise, let $\pi '$ be an irreducible tempered representation of G(W) such that

$$\begin{aligned} \pi ' \subset I^{G(W)}_{P'}(\tau '_1 \otimes \dots \otimes \tau '_{r'} \otimes \pi _0') \end{aligned}$$

with analogous data $P'$, $r'$, $\tau '_i$, $\pi '_0$. Assume that

$$\begin{aligned} \mu (\tau _s \otimes \pi ) = \mu (\tau _s \otimes \pi ') \end{aligned}$$

for all irreducible (unitary) square-integrable representations $\tau$ of $\mathrm {GL}_k(E)$ for all $k \ge 1$. Then we have $r=r'$ and

$$\begin{aligned} \{ \tau _1, \dots , \tau _r, (\tau _1^c)^\vee , \dots , (\tau _r^c)^\vee \} = \{ \tau _1', \dots , \tau _r', ((\tau _1')^c)^\vee , \dots , ((\tau _r')^c)^\vee \} \end{aligned}$$

as multi-sets. Moreover, we have

$$\begin{aligned} \mu (\tau _s \otimes \pi _0) = \mu (\tau _s \otimes \pi _0') \end{aligned}$$

for all irreducible (unitary) square-integrable representations $\tau$ of $\mathrm {GL}_k(E)$ for all $k \ge 1$.

Proof

Note that the second assertion is an immediate consequence of the first assertion and the multiplicativity of Plancherel measures. To prove the first assertion, it suffices to show that

$$\begin{aligned} \# \mathcal {I}(\tau ) + \# \mathcal {I}((\tau ^c)^\vee ) = \# \mathcal {I}'(\tau ) + \# \mathcal {I}'((\tau ^c)^\vee ) \end{aligned}$$

(A.1)

for any irreducible (unitary) square-integrable representation $\tau$ of $\mathrm {GL}_k(E)$, where $\mathcal {I}(\tau ) = \{ i \, | \, \tau _i \cong \tau \}$ and $\mathcal {I}'(\tau ) = \{ i \, | \, \tau _i' \cong \tau \}$. If $(\tau ^c)^\vee \cong \tau$, then by Lemma A.2, we have

$$\begin{aligned} 4 \cdot \# \mathcal {I}(\tau ) + \alpha = 4 \cdot \# \mathcal {I}'(\tau ) + \alpha ' \end{aligned}$$

for some $0 \le \alpha , \alpha ' \le 2$. This forces $\# \mathcal {I}(\tau ) = \# \mathcal {I}'(\tau )$, so that (A.1) holds. If $(\tau ^c)^\vee \ncong \tau$, then (A.1) is a direct consequence of Lemma A.2. This completes the proof. $\square$

The following corollary (which is [45, Theorem 5.1(i)]) is now immediate:

Corollary A.5

(Muić) Suppose that $\pi$ and $\pi '$ are irreducible tempered representations of G(W) which have the same supercuspidal support. If $\pi$ is square-integrable, then so is $\pi '$.

Proof

If $\pi$ and $\pi '$ have the same supercuspidal support, then the multiplicativity of Plancherel measures implies that

$$\begin{aligned} \mu (\tau _s \otimes \pi ) = \mu (\tau _s \otimes \pi ') \end{aligned}$$

for all irreducible (unitary) square-integrable representations $\tau$ of $\mathrm {GL}_k(E)$ for all $k \ge 1$. The assertion then follows from Lemma A.4. $\square$

1.7 A.7 Some variant

Finally, admitting the local Langlands correspondence, we shall state a variant of Lemma A.4 in terms of L-parameters.

Let G(W) be an arbitrary classical group as in [17, Sect. 2]. To each irreducible tempered representation $\pi$ of G(W), the local Langlands correspondence assigns an L-parameter $\phi$, which we regard as a semisimple representation of $WD _E$ as described in [15, Sect. 8]. Moreover, for any irreducible tempered representation $\tau$ of $\mathrm {GL}_k(E)$ with associated L-parameter $\phi _\tau$, Langlands’ conjecture on Plancherel measures [38, Appendix II] says that

$$\begin{aligned} \begin{aligned} \mu (\tau _s \otimes \pi )&= \gamma (s, \phi _\tau \otimes \phi ^\vee , \psi _E) \cdot \gamma (-s, \phi _\tau ^\vee \otimes \phi , \bar{\psi }_E)\\&\quad \times \gamma (2s, R \circ \phi _\tau , \psi ) \cdot \gamma (-2s, R \circ \phi _\tau ^\vee , \bar{\psi }), \end{aligned} \end{aligned}$$

(A.2)

where

$$\begin{aligned} R = {\left\{ \begin{array}{ll} \mathrm {Sym}^2 &{} {\text { if } G(W) \text { is odd orthogonal or metaplectic;}} \\ \wedge ^2 &{} {\text { if } G(W) \text { is even orthogonal or symplectic;}} \\ \mathrm {As}^+ &{} {\text { if } G(W) \text { is even unitary;}} \\ \mathrm {As}^- &{} {\text { if } G(W) \text { is odd unitary.}} \end{array}\right. } \end{aligned}$$

In fact, (A.2) immediately follows from [2, Proposition 2.3.1], [44, Proposition 3.3.1], [33, Lemma 2.2.3] (together with induction in stages) for classical groups considered there. (See also §7.3 in the case of unitary groups.) In other words, recalling the definitions of of Plancherel measures and normalized intertwining operators, we see that (A.2) is a consequence of a property of normalized intertwining operators. Also, in the case of metaplectic groups, (A.2) follows from the case of odd orthogonal groups combined with [19, Proposition 10.1].

Lemma A.6

Let $\pi$ and $\pi '$ be irreducible tempered representations of G(W) with associated L-parameters $\phi$ and $\phi '$ respectively. Assume that

$$\begin{aligned} \mu (\tau _s \otimes \pi ) = \mu (\tau _s \otimes \pi ') \end{aligned}$$

for all irreducible (unitary) square-integrable representations $\tau$ of $\mathrm {GL}_k(E)$ for all $k \ge 1$. Then we have

$$\begin{aligned} \phi = \phi '. \end{aligned}$$

Proof

For any irreducible (unitary) square-integrable representation $\tau$ of $\mathrm {GL}_k(E)$ with associated L-parameter $\phi _\tau$, we have

$$\begin{aligned}&\gamma (s, \phi _\tau \otimes \phi ^\vee , \psi _E) \cdot \gamma (-s, \phi _\tau ^\vee \otimes \phi , \bar{\psi }_E)\\&\quad = \gamma (s, \phi _\tau \otimes (\phi ')^\vee , \psi _E) \cdot \gamma (-s, \phi _\tau ^\vee \otimes \phi ', \bar{\psi }_E) \end{aligned}$$

by assumption and (A.2). Comparing the orders of zero at $s=0$, we see that the multiplicities of $\phi _\tau$ in $\phi$ and $\phi '$ are equal (see also [19, Lemma 12.3]). This completes the proof. $\square$

1.8 A.8 Erratum to [17]

On this occasion, we also correct some typos in [17].

• Lemma C.2: ${\text {Isom}}(Y_a', X_a)$ should be read as the set of invertible conjugate linear maps from $Y_a'$ to $X_a$.

• Bottom of p. 650: $\mathrm {Asai}$ should be read as $\mathrm {As}^+$ (resp. $\mathrm {As}^-$) if $G(W^\bullet )$ is even unitary (resp. odd unitary).

Appendix B: Generic L-packets and adjoint L-factors

In this appendix, we prove a conjecture of Gross–Prasad and Rallis [23, Conjecture 2.6] under a certain working hypothesis.

1.1 B.1 Notation

Let G be a connected reductive algebraic group defined and quasi-split over F. Fix a Borel subgroup B of G over F and a maximal torus T in B over F. Let N be the unipotent radical of B, so that $B = T N$. If P is a parabolic subgroup of G over F, we say that P is standard (relative to B) if $P \supset B$. If P is a standard parabolic subgroup of G over F, then we have a Levi decomposition $P = MU$, where M is the unique Levi component of P such that $M \supset T$ and U is the unipotent radical of P. We call M a standard Levi subgroup of G. Let $W^M = {\text {Norm}}_M(T)/T$ be the Weyl group of M and $w^M_0$ the longest element in $W^M$. Put

$$\begin{aligned} \mathfrak {a}_M^* = {\text {Rat}}(M) \otimes _\mathbb {Z}\mathbb {R}, \quad \mathfrak {a}_M = {\text {Hom}}_\mathbb {Z}({\text {Rat}}(M), \mathbb {R}), \end{aligned}$$

where ${\text {Rat}}(M)$ is the group of algebraic characters of M defined over F. We write $\langle \cdot , \cdot \rangle : \mathfrak {a}_M^* \times \mathfrak {a}_M \rightarrow \mathbb {R}$ for the natural pairing. Let $\mathfrak {a}^*_{M,\mathbb {C}} = \mathfrak {a}_M^* \otimes _\mathbb {R}\mathbb {C}$ be the complexification of $\mathfrak {a}_M^*$. Let $A_M$ be the split component of the center of M and $\varSigma (P)$ the set of reduced roots of $A_M$ in P. We may regard $\varSigma (P)$ as a subset of $\mathfrak {a}_M^* \cong {\text {Rat}}(A_M) \otimes _\mathbb {Z}\mathbb {R}$. For $\alpha \in \varSigma (P)$, let $\alpha ^\vee \in \mathfrak {a}_M$ denote its corresponding coroot. Put

$$\begin{aligned} (\mathfrak {a}_M^*)^+ = \{ \lambda \in \mathfrak {a}_M^{*} \, | \, \langle \lambda , \alpha ^\vee \rangle > 0 \quad \text { for all} \ \alpha \in \varSigma (P) \}. \end{aligned}$$

We define a homomorphism $H_M : M \rightarrow \mathfrak {a}_M$ by requiring that

$$\begin{aligned} |\chi (m)|_F = q^{-\langle \chi , H_M(m) \rangle } \end{aligned}$$

for all $\chi \in {\text {Rat}}(M)$ and $m \in M$, where q is the cardinality of the residue field of F.

Let $\pi$ be an irreducible smooth representation of M. For $\lambda \in \mathfrak {a}^*_{M,\mathbb {C}}$, we define a representation $\pi _\lambda$ of M by $\pi _\lambda (m) = q^{- \langle \lambda , H_M(m) \rangle } \pi (m)$. We write

$$\begin{aligned} I^G_P(\pi _\lambda ) := {\text {Ind}}^G_P(\pi _\lambda ) \end{aligned}$$

for the induced representation of G. If $\pi$ is tempered and ${\text {Re}}(\lambda ) \in (\mathfrak {a}^*_M)^+$, then $I^G_P(\pi _\lambda )$ has a unique irreducible quotient $J^G_P(\pi _\lambda )$.

Let $\widehat{M}$ be the dual group of M and ${}^L M = \widehat{M} \rtimes W_F$ the L-group of M. Let $Z(\widehat{M})$ be the center of $\widehat{M}$. We write $\iota _M : {}^L M \hookrightarrow {}^L G$ for the natural embedding. If $\phi : WD _F \rightarrow {}^L M$ is an L-parameter, we say that $\phi$ is tempered if the projection of $\phi (W_F)$ to $\widehat{M}$ is bounded. For $\lambda \in \mathfrak {a}^*_{M,\mathbb {C}}$, we define an L-parameter $\phi _\lambda : WD _F \rightarrow {}^L M$ by $\phi _\lambda = a_\lambda \cdot \phi$, where $a_\lambda \in Z^1(W_F, Z(\widehat{M}))$ is a 1-cocycle which determines the character $m \mapsto q^{- \langle \lambda , H_M(m) \rangle }$ of M.

1.2 B.2 Hypothesis

In this appendix, we admit the local Langlands correspondence for any standard Levi subgroup M of G:

$$\begin{aligned} {\text {Irr}}(M) = \bigsqcup _{\phi } \varPi _{\phi }, \end{aligned}$$

where the disjoint union on the right-hand side runs over all equivalence classes of L-parameters $\phi$ for M and $\varPi _{\phi }$ is a finite set of representations of M, the so-called L-packet. More precisely, we will use the following properties of the local Langlands correspondence:

• (i) $\pi \in \varPi _\phi$ is tempered if and only if $\phi$ is tempered.

• (ii) $\varPi _{\phi _\lambda } = \{ \pi _\lambda \, | \, \pi \in \varPi _\phi \}$ for $\lambda \in \mathfrak {a}_{M,\mathbb {C}}^*$.

• (iii) If $\phi$ is an L-parameter for G, then replacing $\phi$ by its $\widehat{G}$-conjugate if necessary, we can write

  $$\begin{aligned} \phi = \iota _M \circ (\phi _M)_{\lambda _0}, \end{aligned}$$

  where

  • M is a standard Levi subgroup of G,

  • $\phi _M$ is a tempered L-parameter for M,

  • $\lambda _0 \in (\mathfrak {a}_M^*)^+$.

  Then we have

  $$\begin{aligned} \varPi _{\phi } = \{ J^G_P(\pi _{\lambda _0}) \, | \, \pi \in \varPi _{\phi _M} \}, \end{aligned}$$

  where P is the standard parabolic subgroup of G with Levi component M. Note that $\pi \in \varPi _{\phi _M}$ is tempered by (i) and $\pi _{\lambda _0}$ has L-parameter $(\phi _M)_{\lambda _0}$ by (ii).

• (iv) If $\phi$ is a tempered L-parameter for M, then for any generic character $\psi _{N_M}$ of $N_M := N \cap M$, $\varPi _\phi$ contains a $(N_M, \psi _{N_M})$-generic representation $\pi$ of M (see [53, Conjecture 9.4]). Moreover, we have

  $$\begin{aligned} \gamma ^{\mathrm {Sh}}(s, \pi _\lambda , r_M, \psi ) = \gamma (s, r_M \circ \phi _\lambda , \psi ), \end{aligned}$$

  where the left-hand side is Shahidi’s $\gamma$-factor [53] and $r_M$ is the adjoint representation of ${}^L M$ on ${\text {Lie}}({}^L U)$. In fact, we only need the equality up to an invertible function.

The above hypothesis is known to hold for general linear groups by [26, 29, 51] and for classical groups by [2, 44].

1.3 B.3 A conjecture of Gross–Prasad and Rallis

If $\phi$ is an L-parameter for G, we say that $\phi$ is generic if its associated L-packet $\varPi _\phi$ contains a $(N, \psi _N)$-generic representation of G for some generic character $\psi _N$ of N.

Proposition B.1

Let $\phi$ be an L-parameter for G. Then, under the hypothesis in Sect. B.2, $\phi$ is generic if and only if $L(s, \mathrm {Ad}\circ \phi )$ is holomorphic at $s=1$. Here, $\mathrm {Ad}$ is the adjoint representation of ${}^L G$ on its Lie algebra ${\text {Lie}}({}^L G)$.

1.4 B.4 Proof of Proposition B.1

Fix an L-parameter $\phi$ for G and write $\phi = \iota _M \circ (\phi _M)_{\lambda _0}$ as in (iii). Then by (iii), $\phi$ is generic if and only if $J^G_P(\pi _{\lambda _0})$ is $(N, \psi _N)$-generic for some $\pi \in \varPi _{\phi _M}$ and some generic character $\psi _N$ of N, in which case $\pi$ is necessarily $(N_M, \psi _N|_{N_M})$-generic by a result of Rodier [50], [7, Corollary1.7]. Here, we have also used the fact that for any element w in $W^G$, there exists a representative $\tilde{w}$ of w (depending on $\psi _N$) such that $\psi _N$ is compatible with $\tilde{w}$ (see [54, Sect. 2], [12, Sect. 1.2]). Now we invoke the following result of Heiermann–Muić [28, Proposition 1.3].

Lemma B.2

Let $\psi _N$ be a generic character of N and $\pi$ an irreducible tempered $(N_M, \psi _N|_{N_M})$-generic representation of M. Then $J^G_P(\pi _{\lambda _0})$ is $(N, \psi _N)$-generic if and only if $\gamma ^{\mathrm {Sh}}(0, \pi _\lambda , r_M, \psi )$ is holomorphic at $\lambda =\lambda _0$.

Proof

Since the assertion in [28, Proposition 1.3] is slightly different, we include a proof for the convenience of the reader. We realize the representation $I^G_P(\pi _\lambda )$ by using the unique (up to a scalar) Whittaker functional on $\pi$ with respect to $(N_M, \psi _N|_{N_M})$. Then we can define a Whittaker functional

$$\begin{aligned} \varLambda (\pi _\lambda ) : I^G_P(\pi _\lambda ) \longrightarrow \mathbb {C}\end{aligned}$$

with respect to $(N,\psi _N)$ by (holomorphic continuation of) the Jacquet integral (see [52, Proposition 3.1]). By [50], [7, Corollary 1.7], $\varLambda (\pi _\lambda )$ is a basis of ${\text {Hom}}_N(I^G_P(\pi _\lambda ), \psi _N)$ for all $\lambda \in \mathfrak {a}_{M,\mathbb {C}}^*$. Put $w = w^G_0 w^M_0$ and choose its representative $\tilde{w}$ so that $\psi _N$ is compatible with $\tilde{w}$. As in Sect. 7.3, we can define an unnormalized intertwining operator

$$\begin{aligned} \mathcal {M}(\tilde{w}, \pi _\lambda ) : I^G_P(\pi _\lambda ) \longrightarrow I^G_{w(P)}(w(\pi _\lambda )) \end{aligned}$$

by (meromorphic continuation of) an integral which is absolutely convergent for ${\text {Re}}(\lambda ) \in (\mathfrak {a}^*_M)^+$ (see [60, Proposition IV.2.1]), where w(P) is the standard parabolic subgroup of G with Levi component $w M w^{-1}$. Then we have

$$\begin{aligned} \varLambda (\pi _\lambda ) = C(\tilde{w}, \pi _\lambda ) \cdot \varLambda (w(\pi _\lambda )) \circ \mathcal {M}(\tilde{w}, \pi _\lambda ) \end{aligned}$$

(B.1)

for some meromorphic function $C(\tilde{w}, \pi _\lambda )$, the so-called local coefficient. Here, $C(\tilde{w}, \pi _\lambda )$ depends on the choice of Haar measures in the definitions of $\varLambda (\pi _\lambda )$, $\varLambda (w(\pi _\lambda ))$, $\mathcal {M}(\tilde{w}, \pi _\lambda )$, but we ignore the normalization of Haar measures since it does not affect the proof. Since $J^G_P(\pi _{\lambda _0})$ is isomorphic to the image of $\mathcal {M}(\tilde{w}, \pi _{\lambda _0})$ and the functor ${\text {Hom}}_N(\, \cdot \, , \psi _N)$ is exact, $J^G_P(\pi _{\lambda _0})$ is $(N, \psi _N)$-generic if and only if the restriction of $\varLambda (w(\pi _{\lambda _0}))$ to the image of $\mathcal {M}(\tilde{w}, \pi _{\lambda _0})$ is nonzero. By (B.1), this condition is equivalent to the holomorphy of $C(\tilde{w}, \pi _\lambda )$ at $\lambda = \lambda _0$. On the other hand, by the definition of Shahidi’s $\gamma$-factor, we have

$$\begin{aligned} C(\tilde{w}, \pi _\lambda ) = \gamma ^{\mathrm {Sh}}(0, \pi _\lambda , r_M, \psi ) \end{aligned}$$

up to an invertible function. (Note that the convention in [53] is different from ours: the homomorphism $H_M$ is normalized so that $|\chi (m)|_F = q^{\langle \chi , H_M(m) \rangle }$ in [53]. This is why we have $\gamma ^{\mathrm {Sh}}(0, \pi _\lambda , r_M, \psi )$ on the right-hand side rather than $\gamma ^{\mathrm {Sh}}(0, \pi _\lambda , r_M^\vee , \bar{\psi })$.) This completes the proof. $\square$

Now it follows by Lemma B.2 combined with (iv) that $\phi$ is generic if and only if

$$\begin{aligned} \frac{L(1, r_M^\vee \circ (\phi _M)_\lambda )}{L(0, r_M \circ (\phi _M)_\lambda )} \end{aligned}$$

(B.2)

is holomorphic at $\lambda = \lambda _0$. We consider the analytic property of (B.2). For $\alpha \in \varSigma (P)$, let $A_\alpha$ be the identity component of ${\text {Ker}}(\alpha )$, $M_\alpha$ the centralizer of $A_\alpha$ in G, and $U_\alpha$ the root subgroup associated to $\alpha$. Then $M_\alpha$ is a Levi subgroup of G (but not necessarily a Levi component of a standard parabolic subgroup of G) and $M U_\alpha$ is a maximal parabolic subgroup of $M_\alpha$. We may regard $\mathfrak {a}_{M_\alpha }$ as a subspace of $\mathfrak {a}_M$. Put

$$\begin{aligned} (\mathfrak {a}_{M}^{M_\alpha })^* = \{ \lambda \in \mathfrak {a}_M^* \, | \, \langle \lambda , H \rangle = 0 \quad \text { for all} \ H \in \mathfrak {a}_{M_\alpha } \}. \end{aligned}$$

For $\lambda \in \mathfrak {a}_{M,\mathbb {C}}^*$, let $\lambda ^{M_\alpha }$ denote its orthogonal projection to $(\mathfrak {a}_M^{M_\alpha })^* \otimes _\mathbb {R}\mathbb {C}$. We can write

$$\begin{aligned} \lambda ^{M_\alpha } = s_\alpha (\lambda ) \cdot \varpi _\alpha \end{aligned}$$

for some $s_\alpha (\lambda ) \in \mathbb {C}$, where $\varpi _\alpha \in (\mathfrak {a}_{M}^{M_\alpha })^*$ is the unique element such that $\langle \varpi _\alpha , \alpha ^\vee \rangle = 1$. Then we have

$$\begin{aligned} (\mathrm{B}.2) = \prod _{\alpha \in \varSigma (P)} \frac{L(1 - s_\alpha (\lambda ), r_\alpha ^\vee \circ \phi _M)}{L(s_\alpha (\lambda ), r_\alpha \circ \phi _M)}, \end{aligned}$$

where $r_\alpha$ is the adjoint representation of ${}^L M$ on ${\text {Lie}}({}^L U_\alpha )$. Note that $L(s, r_\alpha \circ \phi _M)$ is holomorphic and nonzero for ${\text {Re}}(s) > 0$ since $\phi _M$ is tempered. Since $\lambda _0 \in (\alpha ^*_M)^+, s_{\alpha }(\lambda _0)$ is a positive real number for all $\alpha \in \varSigma (P)$. Hence (B.2) is holomorphic at $\lambda = \lambda _0$ if and only if

$$\begin{aligned} \prod _{\alpha \in \varSigma (P)} L(1 - s_\alpha (\lambda ) - s_{\alpha }(\lambda _{0}), r_\alpha ^\vee \circ \phi _M) \end{aligned}$$

is holomorphic at $\lambda = 0$. Since the L-factors have no zeros, this condition is equivalent to the holomorphy of $L(s - s_{\alpha }(\lambda _{0}), r_\alpha ^\vee \circ \phi _M)$ at $s=1$ for all $\alpha \in \varSigma (P)$, which in turn is equivalent to the holomorphy of

$$\begin{aligned} L(s, r_M^\vee \circ (\phi _M)_{\lambda _{0}}) = \prod _{\alpha \in \varSigma (P)} L(s - s_{\alpha }(\lambda _{0}), r_\alpha ^\vee \circ \phi _M) \end{aligned}$$

at $s=1$. Thus, we have shown that $\phi$ is generic if and only if $L(s, r_M^\vee \circ (\phi _M)_{\lambda _0})$ is holomorphic at $s=1$.

On the other hand, we have

$$\begin{aligned} L(s, \mathrm {Ad}\circ \phi ) = L(s, r_M \circ (\phi _M)_{\lambda _0}) \cdot L(s, \mathrm {Ad}_M \circ (\phi _M)_{\lambda _0}) \cdot L(s, r_M^\vee \circ (\phi _M)_{\lambda _0}), \end{aligned}$$

where $\mathrm {Ad}_M$ is the adjoint representation of ${}^L M$ on ${\text {Lie}}({}^L M)$. Since $\phi _M$ is tempered and $s_{\alpha }(\lambda _{0}) > 0$ for all $\alpha \in \varSigma (P)$,

$$\begin{aligned} L(s, r_M \circ (\phi _M)_{\lambda _0}) = \prod _{\alpha \in \varSigma (P)} L(s + s_{\alpha }(\lambda _{0}), r_\alpha \circ \phi _M) \end{aligned}$$

and $L(s, \mathrm {Ad}_M \circ (\phi _M)_{\lambda _0}) = L(s, \mathrm {Ad}_M \circ \phi _M)$ are holomorphic and nonzero for ${\text {Re}}(s) > 0$. Hence $L(s, \mathrm {Ad}\circ \phi )$ is holomorphic at $s=1$ if and only if $L(s, r_M^\vee \circ (\phi _M)_{\lambda _0})$ is holomorphic at $s=1$. This completes the proof of Proposition B.1.

Remark B.3

If G is a classical group, then one has the following variant of Proposition B.1 which does not rely on the local Langlands correspondence. Fix a generic character $\psi _N$ of N. If $\pi$ is an irreducible $(N,\psi _N)$-generic representation of G, let $\varPi$ be its functorial lift to the general linear group established in [10, 11, 13, 34, 35] (see [11, Definition 7.1] for the precise definition in the case when G is split over F). Put

$$\begin{aligned} L^{\mathrm {Sh}}(s, \pi , {\mathrm {Ad}}) := L^{\mathrm {Sh}}(s, \varPi , R), \end{aligned}$$

where the right-hand side is Shahidi’s L-factor [53] and

$$\begin{aligned} R = {\left\{ \begin{array}{ll} \mathrm {Sym}^2 &{} {\text { if } G \text { is odd special orthogonal;}} \\ \wedge ^2 &{} {\text { if } G \text { is even special orthogonal or symplectic;}} \\ \mathrm {As}^+ &{} {\text { if } G \text { is even unitary;}} \\ \mathrm {As}^- &{} {\text { if } G \text { is odd unitary.}} \end{array}\right. } \end{aligned}$$

If $\pi$ is tempered, then so is $\varPi$ (see [11, Proposition 7.4] when G is split over F and [35, Proposition 8.6] when G is even unitary) and hence $L^{\mathrm {Sh}}(s, \pi , {\mathrm {Ad}})$ is holomorphic and nonzero for ${\text {Re}}(s)>0$ (see [53, Proposition 7.2]). If we admit the local Langlands correspondence, then by [30], we have $L^{\mathrm {Sh}}(s, \pi , {\mathrm {Ad}}) = L(s, {\mathrm {Ad}} \circ \phi )$, where $\phi$ is the L-parameter of $\pi$.

Now let P be a standard parabolic subgroup of G with Levi component M and $\pi$ an irreducible tempered $(N_M, \psi _N|_{N_M})$-generic representation of M. For any $\lambda \in \mathfrak {a}_{M,\mathbb {C}}^*$, one has the L-factor $L^{\mathrm {Sh}}(s, I^G_P(\pi _\lambda ), {\mathrm {Ad}})$ as above since the set of $\lambda$ such that $I^G_P(\pi _\lambda )$ is irreducible and $(N,\psi _N)$-generic is Zariski dense in $\mathfrak {a}_{M,\mathbb {C}}^*$. Then by the above argument (together with the multiplicativity), one can show that for $\lambda _0 \in (\mathfrak {a}_M^*)^+$, $J^G_P(\pi _{\lambda _0})$ is $(N, \psi _N)$-generic if and only if $L^{\mathrm {Sh}}(s, I^G_P(\pi _{\lambda _0}), {\mathrm {Ad}})$ is holomorphic at $s=1$.