THE SPACE OF UNIPOTENTLY SUPPORTED CLASS FUNCTIONS ON A FINITE REDUCTIVE GROUP

F. DIGNE, G. LEHRER, AND J. MICHEL

to Robert Steinberg

1. Introduction

Let G be a connected reductive algebraic group over an algebraic closure 𝔽 of the finite field 𝔽_{q} of q elements; assume G has an 𝔽_{q}-structure with associated Frobenius endomorphism F and let ℓ be a prime distinct from the characteristic of 𝔽_{q}. In [DLM1, §7.1] and [DLM2] we outlined a program for the determination of the irreducible \overline{ℚ}_{ℓ}-characters of the finite group G^{F}, which showed that the problem may be largely reduced (by induction) to an explicit determination of the Lusztig restrictions ^{∗}R_{M}^{G}(χ) of all the irreducible characters χ of G^{F}, for all rational Levi subgroups M of G. Here, and throughout this paper, the word “rational” means “stable under the action of F ”. As shown in [DLM2], this problem may be addressed through the determination of the Lusztig restrictions ^{∗}R_{M}^{G}(Γ_{u}), where Γ_{u} is the generalized Gelfand-Graev character corresponding to the G^{F} -conjugacy class of the rational unipotent element u ∈ G^{F}.

Now the characters Γ_{u} are examples of class functions on G^{F} which vanish outside the unipotent set. Such functions form a vector space over \overline{ℚ}_{ℓ}, which we denote by C_{uni}(G^{F}); it is the space of unipotently supported class functions on G. The Γ_{u} form a basis of this space, and our strategy in this work will be to determine the map ^{∗}R_{M}^{G} : C_{uni}(G^{F}) → C_{uni}(M^{F}) explicitly. We shall use Lusztig’s orthogonal decomposition of the space C_{uni}(G^{F}) into summands corresponding to “rational blocks” (see below) and determine ^{∗}R_{M}^{G} on each block generically, i.e. in terms of Weyl group data which is associated with the block. In particular, we obtain a simple expression for the Lusztig restriction of generalized Green functions. We then express the generalized Gelfand-Graev characters in terms of this basis to describe their Lusztig restriction. In [DLM2] we computed ^{∗}R_{M}^{G} of the generalized Gelfand-Graev character which corresponds to a regular unipotent class. In this work, we apply the general method to carry out the corresponding computation explicitly in the subregular case.

Our general result on ^{∗}R_{M}^{G} of generalized Gelfand-Graev characters (6.11) essentially reduces this computation to the two problems of finding the Poincaré polynomials P̃_{ι,κ} of certain intersection cohomology complexes on closures of unipotent classes, and to the computation of induction-restriction tables for twisted characters of Weyl groups. In §8 we also prove a result (8.1 below) which reduces these computations in the case of SL_{n} to the case of GL_{n′}, for various n′. These investigations are part of our strategy of reducing the computation of character values to the case of “high” unipotent classes in the usual partial order.

The first five sections of this paper consist largely of a recasting of the of work of Lusztig, which may be found in [L],[L2],[L3], in a form which permits practical computation. They also contain several orthogonality relations for Green functions and their generalizations, which are proved by relating the inner product in C_{uni}(G^{F}) to the inner product of twisted class functions on a Weyl group. In section 6, we prove orthogonality relations for the generalized Gelfand-Graev characters in the same way, in addition to determining their Lusztig restriction. By and large we maintain the notation of [DLM2]. We shall rely on the context to distinguish between the Frobenius endomorphism F of an 𝔽_{q}-group G and the automorphisms, also denoted F, which are induced by F on reflection groups (such as the Weyl group) which are associated with G. Throughout this work we shall freely use the character theory of cosets of a finite group, for which the reader is referred to [DM2, (0.4)] or [K2]. Characters of cosets are also sometimes known as “twisted class functions”.

Date: 30th August, 2002.

2. Preliminaries

Let ι = (C, ζ) be a pair consisting of a unipotent class of G and an irreducible G-equivariant \overline{ℚ}_{ℓ}-local system ζ on it; then C will be called the support of ι and sometimes denoted C_{ι}. If we fix a non-trivial additive character χ_{0} of the prime field 𝔽_{p} of 𝔽_{q}, as in [DLM2, 1.6] we may define a generalized Gelfand-Graev function Γ_{ι} associated with ι; one of our objectives here is to express Lusztig restrictions of generalized Gelfand-Graev characters in terms of generalized Gelfand-Graev characters.

As in [DLM2], if the pair ι is F -stable, we shall follow Lusztig [L3, 24.1–24.2] in making a specific choice of an isomorphism σ : F^{∗}ζ ⟶^{∼} ζ, and we denote by Y_{ι} the characteristic function of ζ which corresponds to σ, and by X_{ι} the characteristic function of the intersection cohomology complex of ζ (for u ∈ C^{F}, we have X_{ι}(u) = Y_{ι}(u)). The set P of all pairs ι is partitioned into “blocks” I, each of which has an associated cuspidal datum (L, ι_{0} = (C_{0}, ζ_{0})) where L is a Levi subgroup of some parabolic subgroup of G, which is unique up to Gconjugacy. If the block concerned is rational, then as explained in [DLM2, 1.4], both L and the parabolic subgroup may be assumed to be rational. The pairs in the block I are in bijection with the irreducible characters of the group W_{G}(L) = N_{G}(L)/L, which is a Coxeter group. If I is a rational block and φ_{ι} is the character associated in this way to ι = (C, ζ) ∈ I^{F}, then an extension φ̃_{ι} of φ_{ι} to W_{G}(L)⋊<F > determines an isomorphism σ : F^{∗}ζ ⟶^{∼} ζ as above. In this work, we shall always choose φ̃_{ι} to be the “preferred extension” described in [L3, 17.2] (as Lusztig does in [L3, 24.2]).

The functions Y_{ι} form a basis of the space of unipotently supported class functions on G^{F} as ι runs over the set P^{F} of all rational pairs. For a given block I, the functions X_{ι} form another basis of the space spanned by {Y_{ι}}_{ι∈I^{F}}, and if we write X_{ι} = ∑_{κ} P_{κ,ι}Y_{κ} then the P_{κ,ι} are polynomials in q with integer coefficients We have P_{κ,ι} = 0 unless C_{κ} ⊂ \overline{C}_{ι} and if C_{κ} = C_{ι} then P_{κ,ι} = δ_{κ,ι} (see e.g., [L, 6.5]). We will assume from now on that the pairs ι have been totally ordered in such a way that C_{κ} ⊂ \overline{C}_{ι} ⇒ κ ≤ ι. Then the matrix (P_{κ,ι}) is upper unitriangular.

Set X̃_{ι} = q^{c_{ι}}X_{ι} and Ỹ_{ι} = q^{c_{ι}}Y_{ι} where c_{ι} = \frac{1}{2}(codim C_{ι} − dim Z_{L}). Then we have X̃_{ι} = ∑_{κ} P̃_{κ,ι}Ỹ_{κ}, where P̃_{κ,ι} = q^{c_{ι}−c_{κ}}P_{κ,ι}.

Remark 2.1. We shall speak below of “complex conjugation” in the field \overline{ℚ}_{ℓ}, denoted by a ↦ \overline{a}. This is justified by noting that \overline{ℚ}_{ℓ} is abstractly isomorphic to ℂ. In practice, we shall apply this notion almost exclusively to the subfield of \overline{ℚ}_{ℓ} which is generated by all roots of unity, on which conjugation is uniquely defined since it fixes ℚ and inverts roots of unity. We therefore speak of “real” values (meaning fixed by conjugation) and “complex conjugates” in this context. The space C_{uni}(G^{F}) is then an inner product space with Hermitian form defined by

⟨ f, g ⟩_{G^{F}} = |G|^{−1} ∑_{x∈G^{F}} f(x)\overline{g(x)}.

Remark 2.2. The cuspidal datum (L, ι_{0}) defines a unique block I_{M} of any Levi subgroup M of G which contains a G-conjugate of L. Assume M and L rational, and let L′ = Int g(L)(:= gLg^{−1}) be a conjugate of L which is rational and contained in M; let M_{0} ⊃ L be the conjugate Int g^{−1}(M) of M. Define w ∈ W_{G}(L) by ẇ = g^{−1}F (g) ∈ N_{G}(L). Then (L′, F) is conjugate to (L, ẇF) and M_{0} is ẇF -stable; moreover we may identify (via Int g^{−1}) (M, F) with (M_{0}, ẇF) and hence (W_{M}(L′), F) with (W_{M_{0}}(L), wF), C_{uni}(M^{F}) with C_{uni}(M_{0}^{ẇF}) and (I_{M}, F) with (I_{M_{0}}, wF). A particular case of this occurs when M_{0} = L, when we refer to the twisted version of L as L_{w} (for w ∈ W_{G}(L)). The cuspidal pair ι_{0} of L is taken by Int(g) to a cuspidal pair of L_{w}. The corresponding characteristic function on L_{w}^{F} is likewise taken by g^{−1} to a function on L^{ẇF}, which we denote by X_{ι_{0},w}.

We recall that Lusztig induction R_{M}^{G} has an easy description in terms of the functions X_{ι}, which applies with some restrictions on p and q. The results of this paper will depend on this, and hence we shall assume, sometimes without explicit mention, for the whole of our work that (cf. [DLM2, 3.1]) the characteristic p is good for G and that q > q_{0}(G), a constant which depends only on the Dynkin diagram of G.

Proposition 2.3. Assume p good and q sufficiently large, and that M contains a rational conjugate L_{w} of L as in 2.2. Assume (as we may, by the above discussion) that L_{w} is a split Levi subgroup of M. Then for ι ∈ I_{M}^{F}, we have:

(i) R_{M}^{G}(X̃_{ι}) = ∑_{κ∈I^{F}} ⟨ φ̃_{ι}, Res_{W_{M_{0}} (L).wF}^{W_{G}(L).F} φ̃_{κ} ⟩_{W_{M_{0}} (L).wF} X̃_{κ}, where R_{M}^{G} is the Lusztig induc-

tion functor. (ii) ⟨ φ̃_{ι}, Res_{W_{M_{0}} (L).wF}^{W_{G}(L).F} φ̃_{κ} ⟩_{W_{M_{0}} (L).wF} = 0 unless \overline{C_{ι}} ⊂ \overline{C_{κ}} ⊂ \overline{Ind_{M}^{G} C_{ι}}.

Proof. Assertion (i) is in [DLM2, 3.3]. Let us prove (ii). For the rightmost inclusion recall that, from the definition of the induction of perverse sheaves, only pairs κ with support smaller than that of the class induced from the support of ι can have non zero coefficient in R_{M}^{G}(X̃_{ι}). To prove the other inclusion, first notice that if ⟨ φ̃_{ι}, Res_{W_{M_{0}} (L).wF}^{W_{G}(L).F} φ̃_{κ} ⟩_{W_{M_{0}} (L).wF} is non-zero

then so is ⟨ φ_{ι}, Res_{W_{M_{0}} (L)}^{W_{G}(L)} φ_{κ} ⟩_{W_{M_{0}} (L)}. But it follows from formula (II) in [S, 1.2] that the latter inner product is zero unless there exists a representative of C_{κ} in C_{ι}.U where U is the unipotent radical of a parabolic subgroup admitting M as a Levi component. This in turn implies C_{ι} ⊂ \overline{C_{κ}} by [DLM1, 5.8]. □

Remark 2.4. We shall often have a situation where M is a rational Levi subgroup of G which contains a rational conjugate L_{w} of L, as in 2.2. In this situation we shall consistently assume w ∈ W_{G}(L) to have been chosen so that L_{w} is split in M, i.e., is contained in a rational parabolic subgroup of M. In this case w ∈ W_{G}(L) is determined up to F -conjugacy in W_{G}(L) and the function R_{L_{w}}^{G}(X_{ι_{0},w}) is well defined (see [DLM2, 3.2 and 3.3(1)]). This is implicit in the statement and proof of 2.3.

3. Generalized Green functions and Lusztig restriction

In this section we shall interpret Lusztig induction and restriction in terms of ordinary induction and restriction of twisted class functions on cosets of parabolic subgroups of Coxeter groups. This will be done by defining a linear isomorphism between the spaces of twisted class functions on W_{G}(L) and a certain subspace of the space of unipotently supported functions. Under this map, the (normalized) characteristic functions of the F -classes of W_{G}(L) correspond to functions we define as “generalized Green functions”. These are analogues of the ordinary Green functions (the latter corresponding to the “principal block”, which is the unique block for which L = T, a maximal torus of G) which constitute a basis of the space of unipotently supported class functions. In order to compute their Lusztig restriction, we shall relate the generalized Gelfand-Graev characters to these.

For the whole of this section, we fix a rational cuspidal datum (L, ι_{0}), where we may assume that L is split, i.e. is contained in a rational parabolic subgroup of G. Let C(W_{G}(L).F) be the space of W_{G}(L)-invariant functions (i.e. class functions) on W_{G}(L).F and recall that C_{uni}(G^{F}) is the space of unipotently supported class functions on G^{F}. For each w ∈ W_{G}(L), we fix a w-twisted rational conjugate L_{w} of L as in 2.2, 2.4, and X̃_{ι_{0},w} ∈ C_{uni}(L_{w}^{F}) is the class function on L^{F} (see 2.2 and 2.4) associated with ι_{0}.

Definition 3.1. Let C_{I}(G^{F}) be the subspace of C_{uni}(G^{F}) spanned by the functions {Y_{ι} | ι ∈ I^{F} }.

(i) Define the linear isomorphism Q^{G} from C(W_{G}(L).F) to C_{I}(G^{F}) by Q^{G}(φ̃_{ι}) = X̃_{ι}. (ii) For w ∈ W_{G}(L) define γ_{wF} ∈ C(W_{G}(L).F) by

γ_{wF} (vF) = \begin{cases}0, & if vF is not W_{G}(L)-conjugate to wF \\ |C_{W_{G}(L)}(wF)|, & otherwise. \end{cases}

(iii) The generalized Green function Q_{wF}^{G} is defined by Q_{wF}^{G} = Q^{G}(γ_{wF}).

Note that since the (distinct) γ_{wF} form a basis of C(W_{G}(L).F), the generalized Green functions Q_{wF}^{G} form a basis of C_{I}(G^{F}).

We shall omit the superscript in Q^{G} and Q_{wF}^{G} when there is no ambiguity.

Proposition 3.2. We have Q_{wF} = R_{L_{w}}^{G} X̃_{ι_{0},w}.

Proof. Since the φ̃_{ι} form an orthonormal basis of C(W_{G}(L).F), and ⟨ θ, γ_{wF} ⟩_{W_{G}(L).F} = θ(wF) for any θ ∈ C(W_{G}(L).F), we have

γ_{wF} = ∑_{ι∈I^{F}} ⟨ φ̃_{ι}, γ_{wF} ⟩_{W_{G}(L).F} φ̃_{ι} = ∑_{ι∈I^{F}} φ̃_{ι}(wF)φ̃_{ι},

whence by linearity

(3.3)  Q_{wF} = ∑_{ι∈I^{F}} φ̃_{ι}(wF)X̃_{ι}.

But by [DLM2, 3.1] we have

(3.4)  X̃_{ι} = |W_{G}(L)|^{−1} ∑_{v∈W_{G}(L)} φ̃_{ι}(vF)R_{L_{v}}^{G}(X̃_{ι_{0},v}).

Now in 3.4, the summand corresponding to w ∈ W_{G}(L) depends only on the W_{G}(L)-class of wF. To see this, observe that the function X̃_{ι_{0},v} is invariant under conjugation by N_{G}(L_{v})^{F}, so that R_{L_{v}}^{G} X̃_{ι_{0},v} depends only on the G^{F} -class of L_{v}, which is parametrized by the W -class of the coset W_{L}.vF, or by the W_{G}(L)-class of the element vF ∈ W_{G}(L).F.

Since the φ̃_{ι} take real values, the second orthogonality relation for them reads

∑_{ι} φ̃_{ι}(wF)φ̃_{ι}(vF) = \begin{cases}0, & if vF is not W_{G}(L)-conjugate to wF \\ |C_{W_{G}(L)}(wF)|, & otherwise. \end{cases}

Substituting 3.4 into 3.3 and using this relation, the result follows. □

It follows from this proposition that our generalized Green functions are the same as those in [L3, 8.3.1], since q^{c_{ι_{0}}} X_{ι_{0}} is the restriction to the unipotent elements of the characteristic function of the perverse sheaf denoted by IC(\overline{Σ}, E)[dim(Σ)] in [L3, 8.2] and for cuspidal local systems, Lusztig’s induction coincides with the induction of perverse sheaves by [L4].

Both C(W_{G}(L).F) and C_{I}(G^{F}) have natural structures as non-degenerate inner product spaces. Although Q^{G} is not an isometry, its effect on scalar products can be computed.

Definition 3.5. Define the function Z_{L} ∈ C(W_{G}(L).F) by Z_{L}(wF) = |Z_{L}^{0wF} | = |Z_{L_{w}}^{0F}|.

Proposition 3.6. We have, for any two functions θ, ϕ ∈ C(W_{G}(L).F),

⟨ Q^{G}(θ), Q^{G}(ϕ) ⟩_{G^{F}} = ⟨ Z_{L}^{−1}θ, ϕ ⟩_{W_{G}(L).F} .

Proof. First note that [L3, 24.3.6], suitably interpreted to take into account the distinction between our X̃_{ι_{0}} and Lusztig’s X_{ι_{0}}, shows that

(3.7)  ⟨ X̃_{ι}, X̃_{κ} ⟩_{G^{F}} = ⟨ Z_{L}^{−1}φ̃_{ι}, φ̃_{κ} ⟩_{W_{G}(L).F} .

Now in order to prove the proposition, it suffices to do so as θ and ϕ run over a basis of C(W_{G}(L).F) In particular, it suffices to take θ = φ̃_{ι} and ϕ = φ̃_{κ}. But then the statement is precisely the equation 3.7, whence the result. □

It follows easily from the definition 3.1(iii) and 3.6 that the generalized Green functions form an orthogonal basis of C_{I}(G^{F}). More precisely, we have

Corollary 3.8.

⟨ Q_{wF} , Q_{w′F} ⟩_{G^{F}} = \begin{cases}0 & if wF and w′F are not conjugate in W_{G}(L), \\ \frac{|C_{W_{G}(L)}(wF)|}{|Z_{L}^{0wF} |} & otherwise. \end{cases}

The formula 3.8 superficially seems different from [L3, 9.11]. However the two formulae are actually equivalent, although there is a power of q in loc.cit. which is absent here. This is explained by the facts that in [loc.cit., 9.11] the inner product used differs from ours, in that it does not involve conjugation, and that the formula given there is for the inner product of two Green functions corresponding to contragredient local systems, with contragredient Frobenius isomorphisms. In Lusztig’s notation, if the characteristic function of the sheaf F with Frobenius isomorphism φ_{1} is f, then the characteristic function of F^{∨} with Frobenius isomorphism φ_{1}^{∨} is q^{−2c_{ι_{0}}} \overline{f} (see the computation in the proof of [L3, 9.8]); this, in conjunction with the fact that R_{M}^{G} commutes with complex conjugation, shows the formulae are equivalent.

Remark 3.9. The preferred extension ε̃ of the alternating character ε of W_{G}(L) will play a prominent rôle in our work. A fact which we shall use repeatedly, and which results from the description in [L3, 17.2] of the preferred extension, is that ε̃ is trivial on Frobenius, i.e., for w ∈ W_{G}(L), ε̃(w.F) = ε(w). Note also that since the preferred extension is real, if φ̃_{ι} is the preferred extension corresponding to ι ∈ I^{F}, then there is a sign ε_{ι} = ±1 such that φ̃_{ι} ⊗ ε̃ = ε_{ι}φ̃ι̂, where ι̂ is defined by φ_{ι} ⊗ ε = φι̂.

Let H be any linear algebraic group with a Frobenius morphism F : H → H which corresponds to an 𝔽_{q}-structure on G. Let T be a maximally split maximal torus of H and write R_{u}(H) for the unipotent radical of H. Then the Weyl group W = W_{H}(T) acts as a reflection group on Y (T) ⊗ ℝ, and F has an induced action as qϕ on this space, where ϕ is a linear transformation of finite order (cf. [DM1, p.40]). Write {f_{1}, f_{2} …, f_{ℓ}} for a set of basic invariants of W and let d_{i} = deg(f_{i}). It is known (cf. [Sp, 6.1]) that the f_{i} may be chosen to be eigenfunctions for ϕ, i.e. ϕf_{i} = δ_{i}f_{i} for each i, where δ_{i} ∈ ℂ.

Lemma 3.10. With notation as in the previous paragraph, we have (i) The order of H is given by

|H^{F} | = q^{dim R_{u}(H)+∑_{i}(d_{i}−1)} ∏_{i}(q^{d_{i}} − δ_{i}).

(ii) If F is varied by keeping ϕ fixed and allowing q to vary, the order function in (i) is a polynomial in q and

|H^{F} |(q^{−1}) = q^{− dim H}ε_{H}|H^{F} |_{q′},

where, for any linear algebraic group H we write ε_{H} = (−1)^{𝔽_{q}-rank of H} and where we denote by |H^{F} |_{q′} the part prime to q of |H^{F} |.

Proof. The formula in (i) is well known (see, e.g., [Le, 1.8]). Part (ii) is obtained directly from (i), taking into account the following three facts. First, it follows from [Sp, 6.5(i)] that the eigenvalues of ϕ on Y (T) ⊗ ℝ are the δ_{i}^{−1}; secondly, if δ_{i} ≠ δ_{i}^{−1}, both occur as eigenvalues of ϕ in the same degree. The latter fact follows because ϕ is real, and so its eigenvalues come in conjugate pairs. As a consequence, we have ∏_{i}(q^{d_{i}} − δ_{i}) = ∏_{i}(q^{d_{i}} − δ_{i}^{−1}), which is required for the identity (ii). Finally, one needs the fact that ε_{H} = det_{Y (T)⊗ℝ}(−ϕ) which holds because for any automorphism ϕ of finite order of a lattice Y, we have det_{Y ⊗ℝ}(ϕ) = (−1)^{d} where d is the codimension of the fixed point subspace of ϕ in Y ⊗ ℝ. □

Remark 3.11. In this work, we shall encounter several functions, whose definition generally involves the number of F -fixed points of some variety on which F acts, and which are (Laurent) polynomials in q. This means that if ϕ remains fixed but q is allowed to vary as in 3.10, they are Laurent polynomials in q. Examples of such functions include the orders of 𝔽_{q}-groups (as in 3.10), P̃_{ι,κ}, and for a unipotent element u ∈ G^{F} with a fixed parametrization (e.g., in the Bala-Carter classification), Q_{wF} (u), and |C_{G^{F}} (u)|. In the case of functions in C_{uni}(G^{F}), the term polynomial will be used when they are linear combinations of the Y_{ι}, with coefficients which are polynomials in the above sense. For any such function f(q), we use the notation f^{∗} to denote the function defined by f^{∗}(q) = f(q^{−1}). The Y_{ι} are fixed by this operation.

The next result gives some properties of the function Z_{L} ∈ C(W_{G}(L).F).

Lemma 3.12. (i) We have |Z_{L}^{0vF} | = |Z_{G}^{0F} | ∑_{i=0}^{l}(r̃)^{∧i}(vF)q^{l−i}(−1)^{i} where l = dim Z_{L}^{0}− dim Z_{G}^{0} and where r̃ is the restriction to W_{G}(L).F of the character of the representation of W_{G}(L)⋊<F > on Y (Z_{L}^{0}/Z_{G}^{0}) ⊗ ℝ, which is an extension of the reflection character r of W_{G}(L).

(ii) We have Z_{L}(q^{−1}) = ε_{Z_{L}}q^{− dim Z_{L}} ε̃· Z_{L}(q).

Proof. We have |Z_{L}^{0vF} | = |Z_{G}^{0F} | ∑_{i}(−1)^{i} Trace(vF |H_{c}^{i}(Z_{L}^{0}/Z_{G}^{0})). As in [DL, proof of 5.7] or [Le, (1.4)], we have

|Z_{L}^{0vF} | = |Z_{G}^{0F} | ∑_{i}(−1)^{i}q^{l−i} Trace(vF, ∧^{i}Y (Z_{L}^{0}/Z_{G}^{0}))

where l = dim Z_{L}^{0}−dim Z_{G}^{0}. Now the space Y (Z_{L}^{0}/Z_{G}^{0})⊗ℝ realizes the reflection representation of the Coxeter group W_{G}(L), as can be seen from [L2, 9.2] and [H, theorem 6], and part (i) of the lemma follows.

For (ii), let v ∈ W_{G}(L) and consider the torus Z_{L}^{0}, with Frobenius action vF. From 3.10 (ii) applied here, we have |Z_{L}^{0vF} |(q^{−1}) = ε′_{Z_{L}}q^{− dim Z_{L}}|Z_{L}^{0vF} |(q), where

ε′_{Z_{L}} = (−1)^{𝔽_{q}-rank of Z_{L} with Frobenius vF} .

But, since ε̃(vF) = det_{Y (Z_{L}^{0})}(v) (recall that v acts trivially on Z_{G} and that ε̃ is the trivial extension), we have ε′_{Z_{L}} = ε_{Z_{L}} ε̃(vF). □

When G is quasi-simple, W_{G}(L) is irreducible, so that r is irreducible. We then have

Lemma 3.13. When r is irreducible, r̃ is the preferred extension of the reflection character.

Proof. The lemma is a consequence of the definition of the preferred extension in [L3, 17.2], and the fact (which can be checked by tracing through [L2, 9.2]) that if we write F = qϕ on V = Y (Z_{L}^{0}/Z_{G}^{0}) ⊗ ℝ so that r̃ is the extension of r in which F acts via ϕ, the automorphism ϕ stabilizes a set of positive roots of a root system for W_{G}(L) in V. We need just consider the case when ϕ is non-trivial, so that (W_{G}(L), ϕ) is of type ^{2}A_{n}, ^{2}E_{6}, ^{3}D_{4} or ^{2}D_{n}. In the cases 2A_{n}, ^{2}E_{6}, in the language of [L3, 17.2] one has a_{r} = 1 so the preferred extension is the one where F acts by −w_{0}, which agrees with ϕ. In the case ^{3}D_{4}, the preferred extension is the only rational one so again agrees with ϕ. Finally, in the case ^{2}D_{n} one checks from the description in [L3, 17.2] that the preferred extension is the one which realizes the reflection representation of B_{n} ≃ D_{n}⋊<F >, and indeed ϕ acts as a reflection, since it acts by exchanging two of the simple roots and fixing the others. □

If G is not quasi-simple the group W_{G}(L) is a direct product of the irreducible Coxeter groups W_{G_{i}}(L) where G_{i} runs over the quasi-simple components of G. The representation of W_{G}(L) on Y (Z_{L}^{0}/Z_{G}^{0}) ⊗ ℝ decomposes into the sum over i of summands isomorphic to the reflection representation r_{i} of the component W_{G_{i}}(L) on Y (Z_{L}^{0}/Z_{G_{i}}^{0}) ⊗ ℝ tensored with the identity representations of the other components. The action of F permutes the r_{i} in the same way it permutes the G_{i}. Since the preferred extension of the identity is the identity, it follows that if G_{i} is F -stable, the extension of r_{i} which appears in Y (Z_{L}^{0}/Z_{G}^{0}) ⊗ ℝ is the preferred extension of r_{i}.

We now describe Lusztig restriction in terms of the generalized Green functions, which form a basis of the space C_{uni}(G^{F}). Let w ∈ W_{G}(L) and suppose M is a rational Levi subgroup which contains a rational conjugate L_{w} of L. Then we shall use the identifications explained in 2.2, 2.4 to consider Q^{M} as a linear isomorphism between C(W_{M_{0}}(L).wF) and C_{I_{M}}(M^{F}).

Theorem 3.14. Let M be a rational Levi subgroup of some parabolic subgroup of G. Then ∗R_{M}^{G} ∘ Q^{G} = 0 unless M contains some rational G-conjugate L_{w} of L, and if this condition holds, then in the above notation, we have

(i) ^{∗}R_{M}^{G} ∘ Q^{G} = Q^{M} ∘ Res_{W_{M_{0}} (L).wF}^{W_{G}(L).F}.
(ii) R_{M}^{G} ∘ Q^{M} = Q^{G} ∘ Ind_{W_{M_{0}} (L).wF}^{W_{G}(L).F}.

Proof. We need only verify the statements on a basis of the relevant space of functions. We start by proving (ii), for which it suffices to evaluate both sides on X̃_{ι} for ι ∈ I_{M}^{F}. By Frobenius reciprocity, 2.3 (i) can be written as

R_{M}^{G}(X̃_{ι}) = ∑_{κ∈I^{F}} ⟨ Ind_{W_{M_{0}} (L).wF}^{W_{G}(L).F} φ̃_{ι}, φ̃_{κ} ⟩_{W_{M_{0}} (L).wF} Q^{G}(φ̃_{κ}) =

Q^{G}( ∑_{κ∈I^{F}} ⟨ Ind_{W_{M_{0}} (L).wF}^{W_{G}(L).F} φ̃_{ι}, φ̃_{κ} ⟩_{W_{M_{0}} (L).wF} φ̃_{κ}) = Q^{G}(Ind_{W_{M_{0}} (L).wF}^{W_{G}(L).F} φ̃_{ι}),

whence (ii) follows.

Now take θ ∈ C(W_{G}(L).F) and consider ^{∗}R_{M}^{G} ∘ Q^{G}(θ). The space C_{uni}(M^{F}) has a basis ⋃_{I′_{M}}{X̃_{ι} | ι ∈ I′_{M}^{F} } where I′_{M} runs over the F -stable blocks of M. Now

⟨^{∗}R_{M}^{G} ∘ Q^{G}(θ), X̃_{ι} ⟩_{M^{F}} = ⟨ Q^{G}(θ), R_{M}^{G}(X̃_{ι}) ⟩_{G^{F}} ,

and by 2.3 the function R_{M}^{G}(X̃_{ι}) is in C_{I′_{G}}(G^{F}), where I′_{G} is the block of G corresponding to I′_{M}. Thus the scalar product is 0 if I′_{G} is not equal to I. Furthermore, the block I is of the form I′_{G} for some (unique by [DLM2, 1.2]) block I′_{M} of M only if M contains a G-conjugate L_{w} of L, whence the first statement of the theorem.

It follows also, that to prove (i), we need only show that for any θ ∈ C(W_{G}(L).F), if we apply both sides of (i) to θ, the resulting functions have the same inner product with any function in C_{I_{M}}(M^{F}). But C_{I_{M}}(M^{F}) is spanned by the functions Q^{M}(ψ) with ψ ∈ C(W_{M}(L_{w}).F), so that it suffices to consider inner products with these functions. We have

⟨^{∗}R_{M}^{G} ∘ Q^{G}(θ), Q^{M}(ψ) ⟩_{M^{F}} = ⟨ Q^{G}(θ), R_{M}^{G}(Q^{M}(ψ)) ⟩_{G^{F}}
= ⟨ Q^{G}(θ), Q^{G} ∘ Ind_{W_{M_{0}} (L).wF}^{W_{G}(L).F} (ψ) ⟩_{G^{F}}  by (ii)
= ⟨ θZ_{L}^{−1}, Ind_{W_{M_{0}} (L).wF}^{W_{G}(L).F} (ψ) ⟩_{W_{G}(L).F}  by 3.6
= ⟨ Z_{L}^{−1} Res_{W_{M_{0}} (L).wF}^{W_{G}(L).F} (θ), ψ ⟩_{W_{M_{0}} (L).wF}
= ⟨ Q^{M} ∘ Res_{W_{M_{0}} (L).wF}^{W_{G}(L).F} (θ), Q^{M}(ψ) ⟩_{M^{F}} ,

which completes the proof. □

Remark 3.15. The result 3.14 may be expressed as asserting the commutativity of the following diagrams.

C(W_{G}(L).F) ⟶^{Q^{G}} C_{I}(G^{F})
Ind↑  ^{R_{M}^{G}}↑
C(W_{M_{0}}(L).wF) ⟶^{Q^{M_{0}}} C_{I_{M_{0}}} (M_{0}^{ẇF})

and

C(W_{G}(L).F) ⟶^{Q^{G}} C_{I}(G^{F})
Res↓  ^{^{∗}R}_{M}^{G}↓
C(W_{M_{0}}(L).wF) ⟶^{Q^{M_{0}}} C_{I_{M_{0}}} (M_{0}^{ẇF})

As an immediate corollary, we have the following explicit formula for the Lusztig restriction of the generalized Green functions.

Corollary 3.16. With notation as in 3.14, we have

∗R_{M}^{G}Q_{vF}^{G} = |W_{M_{0}}(L)|^{−1} ∑_{{x∈W_{G}(L)|x(vF)x^{−1}∈W_{M_{0}} (L).wF }} Q_{x(vF)x^{−1}}^{M}.

Proof. It is easy to see that

Res_{W_{M_{0}} (L).wF}^{W_{G}(L).F} γ_{vF} = |W_{M_{0}}(L)|^{−1} ∑_{{x∈W_{G}(L)|x(vF)x^{−1}∈W_{M_{0}} (L).wF }} γ_{x(vF)x^{−1}}.

The result now follows immediately by applying 3.14 (i) to the function γ_{vF}. □

The duality involution D_{G} (restricted to C_{I}(G^{F})) has an elegant description in this setting.

Proposition 3.17. (cf. [L]) Let D_{G} be the duality involution; then

(i) We have D_{G}(Q_{wF}) = η_{L}ε̃(wF)Q_{wF}, where, for any reductive group G we write η_{G} = (−1)^{semisimple 𝔽_{q}-rank of G} = ε_{G}ε_{Z_{G}}.

(ii) The duality involution D_{G} : C_{I}(G^{F}) ⟶ C_{I}(G^{F}) corresponds under Q^{G} to multiplication by η_{L}ε̃ in C(W_{G}(L).F). In particular D_{G}(X̃_{ι}) = η_{L}ε_{ι}X̃ι̂, where ι̂ and ε_{ι} are defined in 3.9.

Proof. The statement (i) may be found in [L, §8] whose proof applies to the twisted case without change. The first statement in (ii) follows immediately since Q^{G} is linear, and the second statement follows from the relation ε̃⊗ φ̃_{ι} = ε_{ι}φ̃ι̂ (3.9). □

4. Unipotently supported class functions and twisted class functions on reflection groups

For ι ∈ I^{F} define a function Q̃_{ι} on W_{G}(L).F by

(4.1)  Q̃_{ι}(wF) = \frac{1}{a_{ι}} ∑_{a∈A(u)} q^{−c_{ι}} \overline{Y_{ι}(u_{a})}Q_{wF} (u_{a})

where we fix u ∈ C_{ι}^{F} and set A(u) = C_{G}(u)/C_{G}^{0}(u), a_{ι} = |A(u)| and take u_{a} to be a representative of the G^{F} -orbit in C^{F} which corresponds to the F -class of a ∈ A(u). The function Q̃_{ι} does not actually depend on the choice of u ∈ C_{ι}^{F}. Indeed, using the relation

(4.2)  a_{ι}^{−1} ∑_{a∈A(u)} Y_{ι}(u_{a})\overline{Y_{γ}(u_{a})} = δ_{ι,γ}

(see [DLM2, 1.5]) and 3.3, we obtain

(4.3)  Q̃_{ι}(wF) = ∑_{γ∈I^{F}} φ̃_{γ}(wF)P̃_{ι,γ}.

The relation (4.3) justifies the remark above that Q̃_{ι} is independent of the choice of u ∈ C_{ι}^{F}. Note also that the formula (4.3) makes sense even when ι ∉ I; but then, since P̃_{ι,γ} = 0 when ι and γ are in different blocks, Q̃_{ι} = 0.

Proposition 4.4. For any unipotent element u ∈ G^{F}, we have Q_{wF} (u) = ∑_{ι∈I^{F}} Q̃_{ι}(wF)Ỹ_{ι}(u)

Proof. As remarked above, if ι ∉ I then 4.3 shows that the corresponding summand of the right hand side is 0, since then P̃_{ι,γ} = 0 for all γ ∈ I^{F}. So

∑_{ι∈I^{F}} Q̃_{ι}(wF)Ỹ_{ι}(u) = ∑_{ι∈P^{F}} Q̃_{ι}(wF)Ỹ_{ι}(u)

We now use the second orthogonality formula for the Y_{ι}(u):

(4.5)  ∑_{ι∈P^{F}} Y_{ι}(u)\overline{Y_{ι}(u′)} = \begin{cases}|A(u)^{F} | & if u ∼_{G^{F}} u′ \\ 0 & otherwise \end{cases}

where ∼_{G^{F}} means G^{F} -conjugacy. Thus

∑_{ι∈I^{F}} Q̃_{ι}(wF)Ỹ_{ι}(u) = ∑_{ι∈P^{F} ,a∈A(u)} a_{ι}^{−1}\overline{Y_{ι}(u_{a})}Q_{wF} (u_{a})Y_{ι}(u) =

|A(u)|^{−1}|A(u)^{F} |#{a | u_{a} ∼_{G^{F}} u}Q_{wF} (u) = Q_{wF} (u)

□

Note that the equation 4.5 will often be used when u = u′ = u′′_{a} for some rational unipotent element u′′ and some a ∈ A(u′′), in which case we have |A(u)^{F} | = |C_{A(u′′)}(aF)| = |A(u′′)^{aF} |. The functions Ỹ_{ι} form a basis of C_{I}(G^{F}) as ι runs over I^{F}. The next result relates the Q̃_{ι} to expansions in terms of this basis.

Lemma 4.6. (i) For any function f ∈ C_{I}(G^{F}), the coefficient of f in the basis Ỹ_{ι} is

(4.7)  \frac{1}{a_{ι}} ∑_{a∈A(u)} q^{−c_{ι}} \overline{Y_{ι}(u_{a})}f(u_{a}).

(ii) For any function θ ∈ C(W_{G}(L).F), we have Q^{G}(θ) = ∑_{ι∈I^{F}} ⟨ θ, Q̃_{ι} ⟩_{W_{G}(L).F} Ỹ_{ι}. (iii) The functions (Q^{G})^{−1}(Ỹ_{ι}) form the basis of C(W_{G}(L).F) which is dual to the basis

{Q̃_{ι}}.

Proof. By 4.4 and the definition (4.1) of Q̃_{ι}, (i) holds when f = Q_{wF}, and since the Q_{wF} form a basis of C_{I}(G^{F}) and the formula 4.7 is linear in f, (i) holds in general. Similarly, (ii) holds when θ = γ_{wF}, again by 4.4. By linearity, (ii) holds generally. The statement (iii) follows immediately from (ii). □

5. Lusztig’s algorithm and orthogonality relations for generalized Green functions

We shall require

Lemma 5.1. Let H be a finite group, χ_{1}, χ_{2}, ⋯ the irreducible characters of H (over a field of characteristic zero) and f any class function on H which is non-zero at each element of H. Let f^{−1} be the pointwise inverse of f. Then we have the matrix equation

(5.2)  {⟨ f^{−1}χ_{i}, χ_{j} ⟩_{H}}_{i,j} = {⟨ fχ_{i}, χ_{j} ⟩_{H}}_{i,j}^{−1}.

Proof. Since the χ_{i} form an orthonormal basis of the space of class functions on H, the left side of 5.2 is simply the matrix of the linear transformation induced by multiplication by f^{−1}, and the assertion is no more than the observation that multiplication by f^{−1} is the inverse of multiplication by f. □

The statement 5.1 remains valid when H is a finite coset, the χ_{i} are extensions to H of the irreducible characters of the underlying group, and f is a twisted class function on H.

We now recall the algorithm outlined by Lusztig in [L3, §24] for the computation of the polynomials P_{ι,κ}. In the following, unless otherwise stated, we fix a block I and work in C_{I}(G^{F}). Lusztig’s algorithm is based on the following matrix equation, which is an immediate consequence of the relation X̃_{ι} = ∑_{κ} P̃_{κ,ι}Ỹ_{κ} and 3.7.

tP̃Λ̃P̃ = {⟨ X̃_{ι}, X̃_{κ} ⟩_{G^{F}} }_{ι,κ} = {⟨ Z_{L}^{−1}φ̃_{ι}, φ̃_{κ} ⟩_{W_{G}(L).F} }_{ι,κ}

where P̃ = {P̃_{ι,κ}}_{ι,κ} and Λ̃ = {⟨ Ỹ_{ι}, Ỹ_{κ} ⟩_{G^{F}} }_{ι,κ}. We shall use the inverse of this equation:

P̃^{−1}Λ̃^{−1}(^{t}P̃^{−1}) = Ω̃

where Ω̃ = {ω̃_{ι,κ}}_{ι,κ} and ω̃_{ι,κ} = ⟨ Z_{L}φ̃_{ι}, φ̃_{κ} ⟩_{W_{G}(L).F}, the inverse of the matrix on the right hand side being given by Lemma 5.1. The matrix Ω̃ may be considered known (see 3.5) since it is given in terms of Weyl group data. The rows and columns of Λ̃ and P̃ may be ordered in a way compatible with the order on unipotent classes; they may further be ordered so that pairs with the same support form a connected sequence in the order. Then Λ̃ is block-diagonal and P̃ block-triangular with identity diagonal blocks, the blocks corresponding to unipotent classes. Given Ω̃, there are unique matrices Λ̃ and P̃ of this shape which satisfy the above equation.

We note for future reference that 3.12 immediately gives

(5.3)  ω̃_{ι,κ} = |Z_{G}^{0F} | ∑_{i=0}^{l} q^{l−i}(−1)^{i}⟨ φ̃_{ι} ⊗ φ̃_{κ}, r̃^{∧i} ⟩_{W_{G}(L).F}

where l = dim Z_{L}^{0} − dim Z_{G}^{0} and where r̃ is the restriction to W_{G}(L).F of the character of the representation of W_{G}(L)⋊<F > on Y (Z_{L}^{0}/Z_{G}^{0}) ⊗ ℝ, which is an extension of the reflection character r of W_{G}(L).

The following proposition is a generalization of [K1, 1.1.4].

Corollary 5.4. (second orthogonality formula for Green functions)

⟨ Z_{L}Q̃_{ι}, Q̃_{γ} ⟩_{W_{G}(L).F} = \{⟨ Ỹ_{ι}, Ỹ_{γ} ⟩_{G^{F}} \}_{ι,γ}^{−1} = \begin{cases}a_{ι}^{−1} ∑_{a∈A(u)} \frac{|C_{G}^{0}(u_{a})^{F} |}{q^{2c_{ι}}}Y_{ι}(u_{a})\overline{Y_{γ}(u_{a})} & if C_{ι} = C_{γ} \\ 0 & otherwise \end{cases}

where notation is as in 4.1.

Proof. Using the values given in 4.3 for Q̃_{ι} and Q̃_{γ}, we obtain:

{⟨ Z_{L}Q̃_{ι}, Q̃_{γ} ⟩_{W_{G}(L).F} }_{ι,γ} = P̃{⟨ Z_{L}φ̃_{κ}, φ̃_{κ′} ⟩_{W_{G}(L).F} }_{κ,κ′} ^{t}P̃ = P̃Ω̃^{t}P̃ = {⟨ Ỹ_{ι}, Ỹ_{γ} ⟩_{G^{F}} }_{ι,γ}^{−1}

Now ⟨ Ỹ_{ι}, Ỹ_{γ} ⟩_{G^{F}} is 0 if C_{ι} ≠ C_{γ} and otherwise is equal to

(5.5)  ∑_{a∈H^{1}(F,A(u))} |C_{G^{F}} (u_{a})|^{−1}Ỹ_{ι}(u_{a})\overline{Ỹ_{γ}(u_{a})} = a_{ι}^{−1} ∑_{a∈A(u)} |C_{G}^{0}(u_{a})^{F} |^{−1}Ỹ_{ι}(u_{a})\overline{Ỹ_{γ}(u_{a})}.

To see 5.5, note that (A(u_{a}), F) is isomorphic to (A(u), aF), so that

|C_{G^{F}} (u_{a})| = |C_{A(u)}(aF)||C_{G}^{0}(u_{a})^{F} |.

Finally, it follows from 4.2 and 5.1 that the matrix whose (ι, γ) entry is either side of 5.5 is the inverse of the matrix whose (ι, γ) entry is the expression in the statement. □

Remark 5.6. The matrices P̃ and Ω̃ have been defined block by block, but may be extended in an obvious way to matrices for the whole of C_{uni}(G^{F}), which are block-diagonal for the various blocks I; then the computation at the start of the above proof shows in particular that {⟨ Ỹ_{ι}, Ỹ_{γ} ⟩_{G^{F}} }_{ι,γ}^{−1} = 0 if ι and γ belong to different blocks.

5.4 in turn gives an orthogonality formula for the Q_{wF}, regarded as elements of C(W_{G}(L).F) for a fixed value of the argument:

Corollary 5.7. For u a unipotent element of G^{F}, define the function Q_{−}(u) ∈ C(W_{G}(L).F) by Q_{−}(u)(wF) = Q_{wF} (u) (for wF ∈ W_{G}(L).F). Then

⟨ Q_{−}(u), Z_{L}Q_{−}(u′) ⟩_{W_{G}(L).F} =

|A(u)|^{−1} ∑_{a∈A(u)} |C_{G}^{0}(u_{a})^{F} |( ∑_{ι∈I^{F}} \overline{Y_{ι}(u_{a})}Y_{ι}(u))( ∑_{ι∈I^{F}} Y_{ι}(u_{a})\overline{Y_{ι}(u′)}) if u ∼_{G} u′
0  otherwise.

Proof. Applying 4.4 and then 5.4 to the left-hand side we get

⟨ Q_{−}(u), Z_{L}Q_{−}(u′) ⟩_{W_{G}(L).F} = ⟨ ∑_{ι} Q̃_{ι}Ỹ_{ι}(u), ∑_{γ} Z_{L}Q̃_{γ} Ỹ_{γ}(u′) ⟩_{W_{G}(L).F}

= ∑_{ι,γ} Ỹ_{ι}(u)\overline{Ỹ_{γ}(u′)}\{⟨ Ỹ_{ι}, Ỹ_{γ} ⟩_{G^{F}} \}_{ι,γ}^{−1}.

we then use that the matrix {⟨ Ỹ_{ι}, Ỹ_{γ} ⟩_{G^{F}} }_{ι,γ} is real to write the complex conjugate of the expression in 5.4 and we get the result. □

If we sum formula 5.7 over all blocks, we obtain the simpler expression:

Proposition 5.8.

∑_{I} ⟨ Q_{−}^{I}(u), Z_{L}Q_{−}^{I}(u′) ⟩_{W_{G}(L).F} = \begin{cases}|C_{G^{F}} (u)| & if u ∼_{G^{F}} u′ \\ 0 & otherwise \end{cases}

where I runs over the rational blocks and where the superscript I on the Q_{−} indicates the block from which it comes.

Proof. If u ≁_{G} u′, the left side is clearly zero. If u ∼_{G} u′ then using Remark 5.6, the sum over all blocks of the right-hand side of 5.7 is

|A(u)|^{−1} ∑_{a∈A(u)} |C_{G}^{0}(u_{a})^{F} |( ∑_{ι∈P^{F}} \overline{Y_{ι}(u_{a})}Y_{ι}(u))( ∑_{ι∈P^{F}} Y_{ι}(u_{a})\overline{Y_{ι}(u′)}).

Applying the second orthogonality formula 4.5 for Y_{ι}, this reduces to

|A(u)|^{−1} ∑_{{a∈A(u)|u_{a}∼_{G^{F}} u and u_{a}∼_{G^{F}} u′}} |C_{G}^{0}(u_{a})^{F} ||A(u)^{F} ||A(u′)^{F} |

which is 0 unless u ∼_{G^{F}} u′ and equal to |C_{G^{F}} (u)| otherwise. □

6. Gelfand-Graev characters and their Lusztig restriction

As in [L] and [DLM2], for ι ∈ I^{F} and u ∈ C_{ι}^{F}, we define Γ_{ι} = ∑_{a∈A(u)} Y_{ι}(u_{a})Γ_{u_{a}}, where Γ_{u_{a}} is the generalized Gelfand-Graev character attached to the class of u_{a}, and other notation is as in 4.3. We need here to assume that p is large enough for the generalized Gelfand-Graev characters to be defined, e.g., p > 3(h − 1) where h is the Coxeter number for G.

Proposition 6.1. We have Γ_{ι} = a_{ι}ζ_{I}^{−1}Q^{G}(ε̃Z_{L}Q̃_{ι}^{∗}), where ζ_{I} is a fourth root of unity (the one associated to I in [L, 7.2] when G is split).

Proof. We start from the formula [L, 7.5 (b)] of Lusztig, which must be modified for the case of a non-split group in a way hinted at in [L, 8.7]. We claim that for a possibly non-split group, the equation [L, 7.5(b)] should read

(6.2)  Γ_{ι_{0}} = a_{ι_{0}}ζ_{I}^{−1} ∑_{ι,ι_{1}} |W_{G}(L)|^{−1} ∑_{w∈W_{G}(L)} φ̃ι̂_{1}(wF)φ̃_{ι}(wF)|Z_{L}^{0wF} |P̃_{ι_{0},ι}^{∗}ε_{ι_{1}} X̃_{ι_{1}}.

The only part of the generalization which is not obvious, and which is the source of the coefficient ε_{ι_{1}} in the above formula, is (as indicated in [L, 8.7]) the lemma [L, 7.2] whose statement should be changed for the general situation to read ˆ X̃_{ι} |_{G^{F}} _{uni}= ζ_{I}q^{(dim G−dim Z_{L})/2}ε_{ι}X̃ι̂. The proof given in [L, 7.2] cannot be applied in our more general case, since dim V_{ι} has to be replaced by Trace(F | V_{ι}), which might vanish. Nonetheless the generalization may be proved by considering a Frobenius twisted by various v ∈ W_{G}(L) on the induced sheaf which Lusztig considers in that proof.

We now rewrite 6.2 as

Γ_{ι_{0}} = a_{ι_{0}}ζ_{I}^{−1}|W_{G}(L)|^{−1} ∑_{w∈W_{G}(L)} |Z_{L}^{0wF} | ∑_{ι} φ̃_{ι}(wF)P̃_{ι_{0},ι}^{∗} ∑_{ι_{1}} φ̃ι̂_{1}(wF)ε_{ι_{1}} X̃_{ι_{1}}

= a_{ι_{0}}ζ_{I}^{−1}|W_{G}(L)|^{−1} ∑_{w∈W_{G}(L)} |Z_{L}^{0wF} | ∑_{ι} φ̃_{ι}(wF)P̃_{ι_{0},ι}^{∗} ∑_{ι_{1}} ε̃(wF)φ̃_{ι_{1}}(wF)X̃_{ι_{1}} by 3.9

= a_{ι_{0}}ζ_{I}^{−1}|W_{G}(L)|^{−1} ∑_{w∈W_{G}(L)} |Z_{L}^{0wF} | ∑_{ι} φ̃_{ι}(wF)P̃_{ι_{0},ι}^{∗}ε̃(wF)Q_{wF} by 3.3

= a_{ι_{0}}ζ_{I}^{−1}|W_{G}(L)|^{−1} ∑_{w∈W_{G}(L)} |Z_{L}^{0wF} |ε̃(wF)Q̃_{ι_{0}}^{∗}(wF)Q_{wF} by 4.3.

The proposition now follows by 3.1(iii). □

Let us write ζ̃_{I} for the root of unity denoted by ζ_{I} in [DLM2]. The point of this notation is to distinguish ζ̃_{I} and ζ_{I}, since they turn out to be different generalizations to non-split groups of Lusztig’s constant.

Proposition 6.3. For any reductive group, let σ_{G} := (−1)^{semi-simple rank(G)}. Then ζ̃_{I} = η_{L}σ_{L}ζ_{I}.

Proof. We have

⟨ D_{G}Γ_{ι}, X̃_{κ} ⟩_{G^{F}} = ⟨ Γ_{ι}, D_{G}X̃_{κ} ⟩_{G^{F}} = η_{L}ε_{κ}⟨ Γ_{ι}, X̃κ̂ ⟩_{G^{F}} by 3.17(ii)
= η_{L}ε_{κ}⟨ Z_{L}^{−1}(Q^{G})^{−1}(Γ_{ι}), (Q^{G})^{−1}(X̃κ̂) ⟩_{W_{G}(L).F} by 3.6
= η_{L}ε_{κ}a_{ι}ζ_{I}^{−1}⟨ ε̃Q̃_{ι}^{∗}, φ̃κ̂ ⟩_{W_{G}(L).F}
= η_{L}a_{ι}ζ_{I}^{−1}⟨ Q̃_{ι}^{∗}, φ̃_{κ} ⟩_{W_{G}(L).F} by 3.9
= η_{L}a_{ι}ζ_{I}^{−1}P̃_{ι,κ}^{∗} by the ∗ of 4.3.

The equation [DLM2, 1.7] is transformed into this last relation if σ_{L}ζ̃_{I}^{−1} is replaced by η_{L}ζ_{I}^{−1}, whence the proposition. □

It will be convenient to use the normalization Γ̃_{ι} = a_{ι}^{−1}ζ_{I}Γ_{ι}. We shall now discuss orthogonality relations among the Γ̃_{ι} and among the Γ_{u}, as well as the Lusztig restriction of the Γ̃_{ι}. Note that from 6.1 it follows that if I is a rational block and ι ∈ I^{F}, then Γ̃_{ι} ∈ C_{I}(G^{F}).

Lemma 6.4. For any rational block I define Γ̃_{u}^{I} = ∑_{ι∈I} \overline{Ỹ_{ι}^{∗}(u)}Γ̃_{ι}. If there is a pair ι ∈ I^{F} whose support contains u, the orthogonal projection of Γ_{u} onto C_{I}(G^{F}) is ζ_{I}^{−1}q^{c_{ι}} Γ̃_{u}^{I}; otherwise it is 0.

Proof. Using 4.5, the defining relation for Γ_{ι} can be inverted to give

Γ_{u} = |A(u)|^{−1} ∑_{ι∈P^{F}} \overline{Y_{ι}(u)}Γ_{ι}.

If we restrict the above sum to ι ∈ I^{F} we obtain the orthogonal projection of Γ_{u} onto C_{I}(G^{F}), since the various spaces C_{I}(G^{F}) are mutually orthogonal. The lemma now follows in straightforward fashion from the definitions. □

Proposition 6.5. We have Γ̃_{u}^{I} = Q^{G}(ε̃Z_{L}\overline{Q}_{−}^{∗}(u)).

Proof. Apply (Q^{G})^{−1} to the expression 6.4 for Γ̃_{u}^{I} to get

(Q^{G})^{−1}(Γ̃_{u}^{I}) = ∑_{ι∈I^{F}} \overline{Ỹ_{ι}^{∗}(u)}ε̃Z_{L}Q̃_{ι}^{∗}.

Now take the complex conjugate of the ^{∗} of the relation 4.4 and substitute into this last equation. Taking into account that the functions Q̃_{ι} are real valued (i.e. stable under complex conjugation), which is a consequence of 4.3 since the φ̃_{ι} are real, we obtain the proposition. □

Corollary 6.6. We have ⟨ Γ̃_{ι}, D_{G}Γ̃_{κ} ⟩_{G^{F}} = ε_{G}q^{dim Z_{L}} ({⟨ Ỹ_{ι}, Ỹ_{κ} ⟩_{G^{F}} }_{ι,κ}^{−1})^{∗}, which is zero if C_{ι} ≠ C_{κ}.

Proof. We have

⟨ Γ̃_{ι}, D_{G}Γ̃_{κ} ⟩_{G^{F}} = ⟨ Z_{L}^{−1}(Q^{G})^{−1}(Γ̃_{ι}), η_{L}ε̃(Q^{G})^{−1}(Γ̃_{κ}) ⟩_{W_{G}(L).F} by 3.6 and 3.17(ii)
= η_{L}⟨ Q̃_{ι}^{∗}, Z_{L}ε̃Q̃_{κ}^{∗} ⟩_{W_{G}(L).F}
= η_{L}ε_{Z_{L}}q^{dim Z_{L}}⟨ Q̃_{ι}^{∗}, Z_{L}^{∗}Q̃_{κ}^{∗} ⟩_{W_{G}(L).F} by 3.12
= ε_{L}q^{dim Z_{L}}⟨ Q̃_{ι}^{∗}, Z_{L}^{∗}Q̃_{κ}^{∗} ⟩_{W_{G}(L).F} since η_{L} = ε_{L}ε_{Z_{L}}
= ε_{L}q^{dim Z_{L}} (⟨ Q̃_{ι}, Z_{L}Q̃_{κ} ⟩_{W_{G}(L).F} )^{∗}
= ε_{L}q^{dim Z_{L}} ({⟨ Ỹ_{ι}, Ỹ_{κ} ⟩_{G^{F}} }_{ι,κ}^{−1})^{∗} by 5.4.

The result now follows because ε_{L} = ε_{G} since L is G-split. □

Corollary 6.7. Let u, v ∈ G^{F} be unipotent elements and I a rational block. Then

⟨ Γ̃_{u}^{I}, D_{G}Γ̃_{v}^{I} ⟩_{G^{F}} = ε_{G}q^{dim Z_{L}} (⟨ \overline{Q}_{−}(u), Z_{L}\overline{Q}_{−}(v) ⟩_{W_{G}(L).F} )^{∗} ,

which is non-zero only if u and v are conjugate in G.

Proof. We have, from 6.5, proceeding as in 6.6

⟨ Γ̃_{u}^{I}, D_{G}Γ̃_{v}^{I} ⟩_{G^{F}} = ⟨ ε̃\overline{Q}_{−}^{∗}(u), η_{L}ε̃^{2}Z_{L}\overline{Q}_{−}^{∗}(v) ⟩_{W_{G}(L).F}
= η_{L}⟨ ε̃\overline{Q}_{−}^{∗}(u), Z_{L}\overline{Q}_{−}^{∗}(v) ⟩_{W_{G}(L).F}
= η_{L}ε_{Z_{L}}q^{dim Z_{L}}⟨ ε̃\overline{Q}_{−}^{∗}(u), ε̃Z_{L}^{∗}\overline{Q}_{−}^{∗}(v) ⟩_{W_{G}(L).F}
= ε_{L}q^{dim Z_{L}}⟨ \overline{Q}_{−}^{∗}(u), Z_{L}^{∗}\overline{Q}_{−}^{∗}(v) ⟩_{W_{G}(L).F}
= ε_{L}q^{dim Z_{L}} (⟨ \overline{Q}_{−}(u), Z_{L}\overline{Q}_{−}(v) ⟩_{W_{G}(L).F} )^{∗} ,

and the result follows as in 6.6. The last remark is a consequence of the evaluation of the right side in 5.7. □

Corollary 6.8. For any pair u, v of unipotent elements of G^{F}, we have

⟨ Γ_{u}, D_{G}Γ_{v} ⟩_{=} \begin{cases}ε_{G}ε_{C_{G}(u)}|C_{G^{F}} (u)|_{q′} & if u ∼_{G^{F}} v \\ 0 & otherwise. \end{cases}

Proof. From 6.4, we see ⟨ Γ_{u}, D_{G}Γ_{v} ⟩_{=} ∑_{I} ⟨ ζ_{I}^{−1}q^{c_{ι}} Γ̃_{u}^{I}, ζ_{I}^{−1}q^{c_{γ}} D_{G}Γ̃_{v}^{I} ⟩_{G^{F}} where the sum is over all blocks which contain two pairs ι, γ whose support respectively contains u and v. By 6.7 this sum is 0 if u and v are not G-conjugate; otherwise we obtain

⟨ Γ_{u}, D_{G}Γ_{v} ⟩_{=}ε_{G}q^{codim(class(u))} ∑_{I} (⟨ \overline{Q}_{−}^{I}(u), Z_{L}\overline{Q}_{−}^{I}(v) ⟩_{W_{G}(L).F} )^{∗} .

We now apply 5.8 and 3.10(ii) to complete the proof. □

To describe the Lusztig restrictions of the Γ̃_{ι}, we shall define the notion of “sign relative to a block”. Suppose M is a rational Levi subgroup which contains a rational G-conjugate of L, as in 2.2 and 2.4. If T_{0} is a maximally split rational maximal torus of L (and hence of G), the element w ∈ W_{G}(L) ⊆ W_{G}(T_{0})/W_{L}(T_{0}). This element is uniquely defined by M and the conditions on L_{w}, up to F -conjugacy in W_{G}(L). The sign ε_{I}(M) of M relative to the block I is defined as −1 raised to the codimension in Y (Z_{L}^{0}) ⊗ ℝ of the subspace of w-fixed points of Y (Z_{L}^{0}) ⊗ ℝ. It has also the following alternative definition:

Definition 6.9. With notation as in the previous paragraph, define ε_{I}(M) := ε^{G}(w) where ε^{G} is the sign character of W_{G}(L).

It follows from the remarks in the last paragraph that the right side depends only on (the G^{F} -conjugacy class of) M.

Lemma 6.10. (i) In the notation of Remarks 2.2 and 2.4, there exist Laurent polyno-

mials R_{ι,γ} in q (ι ∈ I^{F} and γ ∈ I_{M}^{F}) such that Res_{W_{M_{0}} (L).wF}^{W_{G}(L).F} Q̃_{ι} = ∑_{γ∈I_{M}^{F}} R_{ι,γ} Q̃_{γ}.

We have R_{ι,γ} = 0 unless \overline{C_{γ}} ⊂ \overline{C_{ι}} ⊂ \overline{Ind_{M}^{G} C_{γ}}. (ii) Maintaining the above notation, we have Res_{W_{M_{0}} (L).wF}^{W_{G}(L).F} ε̃^{G} = ε_{I}(M)ε̃^{M}, where ε_{I}(M) is defined in 6.9 and where ε̃^{G} (resp. ε̃^{M}) is the preferred extension of the sign character of W_{G}(L) (resp. W_{M_{0}}(L)).

Proof. Let R be the matrix with (ι, γ) coefficient R_{ι,γ} as in (i) of the statement. From 4.3, we obtain the matrix equation

P̃^{G}{⟨ φ̃_{γ}, Res_{W_{M_{0}} (L).wF}^{W_{G}(L).F} φ̃_{ι′} ⟩_{W_{M_{0}} (L).wF} }_{ι′,γ} = RP̃^{M}.

The first statement in (i) is now immediate, since the entries of the unitriangular matrix P̃^{M} are Laurent polynomials, whence the same is true of its inverse. The second statement in (i) follows from 2.3 (ii).

For (ii), let v.wF ∈ W_{M_{0}}(L).wF. Then

ε̃^{G}(v.wF) = ε^{G}(vw) = ε^{G}(v)ε^{G}(w) = ε̃^{M}(v.wF)ε_{I}(M).
□

Proposition 6.11. We have ^{∗}R_{M}^{G}(Γ̃_{ι}) = 0 unless M contains a rational G-conjugate of L; in the latter case, we have ^{∗}R_{M}^{G}(Γ̃_{ι}) = ε_{I}(M) ∑_{γ∈I_{M}^{F}} R_{ι,γ}^{∗} Γ̃_{γ}, where notation is as in 6.5 and 6.10 above.

Proof. Since Γ̃_{ι} ∈ C_{I}(G^{F}), it follows from 3.14 that ^{∗}R_{M}^{G}Γ̃_{ι} is zero unless M contains a rational G-conjugate of L. We therefore take M as in 3.14. Now by 6.1, Γ̃_{ι} = Q^{G}(ε̃Z_{L}Q̃_{ι}^{∗}), and Z_{L} ∈ C(W_{G}(L).F) is defined in 3.5. By 3.14 we need only compute the restriction to W_{M_{0}}(L) of ε̃Z_{L}Q̃_{ι}^{∗}, and a straightforward calculation using 6.10 yields the statement. □

Remark 6.12. As in [DLM2], we refer to a block as regular if it contains a local system supported by the regular unipotent class. It is a consequence of [DLM2, §2] that for regular blocks, ζ̃_{I} is independent of the ambient group and the rational structure, i.e. depends only on the geometric data in the cuspidal system (L, ι_{0}). This is asserted without justification in the proof of [DLM2, 3.4] but can be seen as follows. From [DLM2, 2.1] and [DLM2, 2.5] one has that ζ̃_{I} is equal (in the notation of loc. cit.) to η_{L}σ_{L}σ_{ζ}^{L} up to a power of q. Using the Hasse-Davenport relation, one may compare the product of Gauss sums in [DLM2, 2.4] which applies to the case of twisted L, to that occurring in a split group. One finds that the products also differ by a factor η_{L}σ_{L}. Thus ζ̃_{I} = ζ̃_{I_{M}} in this case. In particular, this applies generally to the principal block (when L is a maximal torus). In general, the question as to whether ζ̃_{I} = ζ̃_{I_{M}} in all cases amounts to the question of whether ζ̃_{I_{L}} is independent of the Frobenius structure on the triple (L, C_{ι_{0}}, ι_{0}). Although this point does not affect the formulation of 6.11, it is relevant to some of the computations later in this work.

Remark 6.13. The equation 6.11 may be expressed as follows.

∗R_{M}^{G}(a_{ι}Γ_{ι}) = ε_{I}(M)ζ_{I}ζ_{I_{M}}^{−1} ∑_{γ∈I_{M}^{F}} R_{ι,γ}^{∗}a_{γ}Γ_{γ} = ε_{G}ε_{M}ζ̃_{I} ζ̃_{I_{M}}^{−1} ∑_{γ∈I_{M}^{F}} R_{ι,γ}^{∗}a_{γ}Γ_{γ},

and the previous remark implies that in the regular case, the factor ζ̃_{I} ζ̃_{I_{M}}^{−1} is equal to 1.

7. Application to the regular and subregular cases

Our objective now is to apply Proposition 6.11 to some specific cases. The general strategy will be first to compute 4.3 explicitly in G and in M by computing certain required values P̃_{ι,κ}, and then to use specific knowledge of restriction of characters from W_{G}(L).F to W_{M_{0}}(L).wF.

As an example, consider first the case when ι = ρ, where ρ is a pair in the block I with support the regular unipotent class (such a pair is then the unique one with regular support in the block I, see [DLM2, 1.10]). Then the only non-zero term in the right hand side of formula 4.3 is φ̃_{ρ}(wF), as P̃_{ρ,ρ} = 1 and P̃_{ρ,γ} = 0 if C_{ρ} ⊄ \overline{C}_{γ}. Moreover, as ρ has regular support we have φ̃_{ρ} = Id. So we get Q̃_{ρ^{G}} = Id_{W_{G}(L).F}, whence Res_{W_{M_{0}} (L).wF}^{W_{G}(L).F} Q̃_{ρ^{G}} = Q̃_{ρ^{M}}. Applying 6.13 we get

∗R_{M}^{G}Γ_{ρ^{G}} = \frac{a_{ρ^{G}}}{a_{ρ^{M}}} ε_{G}ε_{M}Γ_{ρ^{M}}.

Thus we recover lemma 3.6 of [DLM2].

Proposition 7.1. Consider an F -stable pair σ with support a subregular class C_{σ} of G and denote by I the corresponding block; then one of the following holds:

(i) The representation φ_{σ} is a component of the reflection representation r of W_{G}(L). In this case, Q̃_{σ} = qĨd + φ̃_{σ} and the block I is regular.

(ii) The representation φ_{σ} is not a component of r^{∧i} for any i; then Q̃_{σ} = φ̃_{σ}. In this case the block may or may not be regular.

We shall refer to case (i) by saying that σ is standard. Recall that a block I is regular if there exists a local system in I with support the regular class and that in that case this local system is unique and corresponds to the identity representation of W_{G}(L) (cf. [DLM2, 1.10]).

Proof. We prove first that one of the two properties for φ_{σ} and I holds. This is done by checking the tables of the appendix. First we reduce to the case when G is quasi-simple and simply connected. If G is not quasi-simple, a unipotent class is a product of unipotent classes of the quasi-simple components. In particular a subregular class is the product of the regular classes of all the components but one and the subregular class in the last component. Although local systems depend on isogeny, Green functions do not, and hence it clearly suffices to treat the simply connected group in each isogeny class, in which case we may assume that the local system on such a class is the product of local systems on the components. In particular, a cuspidal datum is a product of cuspidal data for the quasi-simple components. All this shows that we can reduce the verification to the quasi-simple (simply connected) case.

It is then apparent from the tables that when φ_{σ} is the reflection representation, the block is regular and that otherwise φ has dimension strictly less than the reflection representation, so appears in no exterior power of the reflection representation.

We now prove the formula for Q̃_{σ} in each case. We know that P_{ι,γ} is zero unless C_{ι} ⊊ \overline{C_{γ}} or ι = γ. So P_{σ,ι} = 0 unless C_{ι} is the regular class or ι = σ.

Consider first the case when I is regular: denote by ρ the unique pair in I with regular support. If we take the rows and columns pertaining to σ and ρ to be the last two, the matrix equation P̃^{−1}Λ̃^{−1}(^{t}P̃^{−1}) = Ω̃ which determines P̃ and Λ̃ has the form:

(\begin{matrix}… & … & … \\ 0 & 1 & Q \\ 0 & 0 & 1 \end{matrix}) (\begin{matrix}… & … & 0 \\ 0 & μ_{σ} & 0 \\ 0 & 0 & μ_{ρ} \end{matrix}) (\begin{matrix}… & 0 & 0 \\ … & 1 & 0 \\ … & Q & 1 \end{matrix}) = (\begin{matrix}… & … & … \\ … & ω̃_{σ,σ} & ω̃_{σ,ρ} \\ … & ω̃_{ρ,σ} & ω̃_{ρ,ρ} \end{matrix})

where Q = (P̃^{−1})_{σ,ρ}, μ_{σ} = (Λ̃^{−1})_{σ,σ} and μ_{ρ} = (Λ̃^{−1})_{ρ,ρ}. We thus get: μ_{σ} + Q^{2}μ_{ρ} = ω̃_{σ,σ}, Qμ_{ρ} = ω̃_{σ,ρ} and μ_{ρ} = ω̃_{ρ,ρ}.

In case (i) we apply 5.3. If G_{1}, …, G_{k} are the quasi-simple components of G, we have r^{∧i} = ∑_{i_{1}+...+i_{k}=i} r_{1}^{∧i_{1}} ⊗ … ⊗ r_{k}^{∧i_{k}}, where r_{i} is the reflection representation of the i-th component

of W_{G}(L). So, using the remarks following 3.13 we have ⟨ φ̃_{σ}, r̃^{∧i} ⟩_{W_{G}(L).F} = \begin{cases}1 & if i = 1 \\ 0 & otherwise \end{cases}.

We then obtain ω̃_{σ,ρ} = −|Z_{G}^{0F} |q^{l−1} and ω̃_{ρ,ρ} = |Z_{G}^{0F} |q^{l} where l is as in 5.3, whence Q = −q, whence P̃_{σ,ρ} = q.

In case (ii), the above computation gives ω̃_{σ,ρ} = 0, so the only non-zero P̃_{σ,ι} is P̃_{σ,σ} = 1. It remains only to consider case (ii) for a non-regular block, where dimension considerations

imply that the only non-zero entry P̃_{σ,ι} is P̃_{σ,σ}.

In either case, the value of Q̃_{σ} by is obtained by applying 4.3. □

Proposition 7.2. Assume that σ is an F -stable standard subregular pair in the regular block I_{G}, and that G is quasi-simple. Let M be a rational Levi subgroup of G, and let C_{1}, …, C_{k} be the F -stable subregular classes in M, which are in bijection with the set of wF -stable irreducible constituents M_{i} of M_{0}. Let σ_{i} be the pair corresponding to the reflection representation of W_{M_{i}}(L); then σ_{i} has support C_{i} and is a standard pair in the regular block I_{M}. Moreover we have

Res_{W_{M_{0}} (L).wF}^{W_{G}(L).F} Q̃_{σ} = ((1 − k)q^{−1} + φ̃_{σ}(wF) − ∑_{i=1}^{i=k} φ̃_{σ_{i}}(wF)) Q̃_{ρ^{M}} + ∑_{i=1}^{i=k} Q̃_{σ_{i}}

where ρ^{M} is the pair with regular support in I_{M}.

Proof. Let V_{G} = Y (Z_{L}^{0}/Z_{G}^{0}) ⊗ ℝ, and V_{M_{0}} = Y (Z_{L}^{0}/Z_{M_{0}}^{0}) ⊗ ℝ. By 3.13, φ̃_{σ} is the extension of the reflection representation of W_{G}(L) which occurs in V_{G}, and by the same remarks we have Trace(vwF | V_{M_{0}}) = ∑_{i} φ̃_{σ_{i}}(vwF) for v ∈ W_{M_{0}}(L) (only the wF -stable components occur when we take the trace of an element in the coset W_{M_{0}}(L).wF). Thus if V is the kernel of the natural map V_{G} → V_{M_{0}}, we have Res_{W_{M_{0}} (L).wF}^{W_{G}(L).F} φ̃_{σ} = ∑_{i=1}^{i=k} φ̃_{σ_{i}} + Trace(wF | V)Ĩd.

Evaluating both sides at wF we get Trace(wF | V) = φ̃_{σ}(wF) − ∑_{i=1}^{i=k} φ̃_{σ_{i}}(wF).

Now by [DLM2, 1.10] since the block I_{G} is regular by assumption, the block I_{M} is also regular. We know from the remark after the statement of 7.1 that the pairs which occur in the restriction of Q̃_{σ} have regular or sub-regular support Since the regular class corresponds to Ĩd in any regular block, σ_{i} must have support C_{i}, and thus σ_{i} is standard, so that by 7.1 we have φ̃_{σ_{i}} = Q̃_{σ_{i}} − qQ̃_{ρ^{M}}.

The formula for the restriction of Q̃_{σ} results from this and the above formula for the restriction of φ̃_{σ}. □

From 6.13 and 7.2 above, we deduce Proposition 7.3. For any standard subregular pair σ, we have

ε_{G}ε_{M}^{∗}R_{M}^{G}Γ_{σ} = \frac{a_{σ}}{a_{σ_{i}}} Γ_{σ_{i}} + \frac{a_{σ}}{a_{ρ^{M}}} ((1 − k)q + φ̃_{σ}(wF) − ∑_{i=1}^{i=k} φ̃_{σ_{i}}(wF)) Γ_{ρ^{M}}.

Similar computations can be made for non-standard pairs; however the end result does not appear to have as clear a statement.

8. The case of SL_{n}

We now discuss the case of G = SL_{n}. According to [LS, §5], cuspidal data are indexed by characters of the centre Z of SL_{n}. Assume that χ is a character of order d of Z where d is a divisor of n; then χ corresponds to an equivariant cuspidal local system on the regular class of a Levi subgroup of type A_{d−1}^{n/d}. We will denote by I_{χ} the corresponding block of G. The unipotent classes of G are indexed by partitions of n. Let C_{λ} be the class indexed by the partition λ of n. There is at most one local system on C_{λ} in I_{χ}; such a system exists when all the parts of λ are divisible by d and we will denote it by ι_{λ}^{χ}. When χ is the trivial character, ι_{λ}^{χ} is the trivial local system on C_{λ}, which is also the only irreducible local system on C_{λ} in GL_{n}. We will denote it simply by ι_{λ} in the latter case.

Theorem 8.1. The Laurent polynomial P̃_{ι_{λ}^{χ},ι_{μ}^{χ}} for SL_{n} is equal to the Laurent polynomial P̃_{ι_{λ/d},ι_{μ/d}} for GL_{n/d}, where λ/d (resp. μ/d) denotes the partition whose parts are 1/d times those of λ (resp. μ).

Proof. The proof consists of merely observing that the equations which determine P̃_{ι_{λ}^{χ},ι_{μ}^{χ}} and P̃_{ι_{λ/d},ι_{μ/d}} coincide. In either case the equation may be written: P̃^{−1}Λ_{1}(^{t}P̃^{−1}) = Ω_{1} where Λ_{1} = |Z_{G}^{0F} |^{−1}Λ̃^{−1} and Ω_{1} = |Z_{G}^{0F} |^{−1}Ω̃. In the present case, F acts trivially on W_{G}(L). If, for φ ∈ Irr(W_{G}(L)), we denote by ι_{φ} the corresponding local system, we have according to 5.3:

(Ω_{1})_{ι_{φ},ι_{φ′}} = ∑_{i=0}^{l} q^{l−i}(−1)^{i}⟨ φ ⊗ φ′, r^{∧i} ⟩_{W_{G}(L)}.

We have two cases to consider: firstly G = SL_{n}, L of type A_{d−1}^{n/d} and secondly G = GL_{n/d}, L a maximal torus. In either case we have W_{G}(L) ≃ 𝔖_{n/d} and l = n/d − 1. Thus the matrices Ω_{1} in the two cases may be identified through the bijection which maps the local system ι_{λ}^{χ} to the local system ι_{λ/d} (since, according to [LS, §5] both correspond under the generalized Springer correspondence to the character of 𝔖_{n/d} indexed by the partition λ/d). To verify that the equations are the same, it remains only to check that the rows and columns of the matrix P̃, both of which are indexed by the irreducible characters of 𝔖_{n/d}, are ordered in the same way in either case. This ordering is induced by the partial order on unipotent classes in either case, and the coincidence follows from the description of this partial order in terms of partitions: we have C_{λ} ≥ C_{μ} if and only if λ ≥ μ where, if λ = {λ_{1}, λ_{2}, …} with λ_{1} ≥ λ_{2} ≥ … (resp. μ = {μ_{1}, μ_{2}, …} with μ_{1} ≥ μ_{2} ≥ …) this means that for all i we have λ_{1} + … + λ_{i} ≥ μ_{1} + … + μ_{i}. This condition is compatible with dividing all parts of λ and μ by the same integer d, whence the result. □

The significance of the previous result is that in view of 6.11, the computation of ^{∗}R_{M}^{G} of the generalized Gelfand-Graev characters, hence of the X_{ι}, and through them of the Y_{ι}, and hence of the characteristic functions of the unipotent conjugacy classes for the group SL_{n}, is reduced to the same problem for various GL_{n′}, which is in principle known. According to the program in [DLM1], this is a step towards determining the character table of SL_{n}(q). The other essential step in this program is the determination of ^{∗}R_{M}^{G} of the irreducible characters, for which the work of C. Bonnafé gives a solution.

9. Appendix: local systems on the subregular unipotent class in good characteristic for simply connected groups

We describe now the generalized Springer correspondence for local systems on the subregular class for simply connected quasi-simple groups. The description for arbitrary quasi-simple groups follows easily.

This appendix contains information extracted from [L2], [LS] and [S]. The table below contains the following information:

• The column “G” contains the type of G.

• The column “C” describes the subregular class C, in Carter’s notation for exceptional groups and by giving the partition associated to the Jordan form for classical groups.

• The column “Dynkin-Richardson” contains the Dynkin-Richardson diagram of C.

• The column “A(u)” describes the group A(u) for an element u ∈ C.

• The column “ι” describes the local system ζ considered on C; it is described by giving the name of the corresponding character of A(u); this last group is when possible described as a Coxeter group so the naming scheme for characters of Coxeter groups (see below) applies. The exceptions are the cyclic group of order 3 whose characters are denoted 1, ζ, ζ^{2} and the cyclic group group of order 4 whose characters are denoted 1, i, −1, −i. If ι = (C, ζ) let (L, ι_{0}) be the corresponding cuspidal datum, where ι_{0} = (ζ_{0}, C_{0}). In general there is only one cuspidal pair in L (which is in most cases a local system on the regular class) so neither C_{0} nor ζ_{0} is mentioned; when there is an ambiguity they are mentioned in the last column.

• When L is not a maximal torus T or equal to G, the column “L” describes the Levi by circling the nodes corresponding to simple roots of L on the Dynkin diagram of G. The simple roots of W_{G}(L) in X(Z_{L}^{0}/Z_{G}^{0}) ⊗ ℝ therefore correspond to the unmarked nodes of the same diagram.

• When W_{G}(L) is neither trivial nor equal to W_{G} it is described in the column “W_{G}(L)” by its Dynkin diagram, which has been decorated by letters a, b, … which appear also on the un-circled nodes in the column “L” to describe the correspondence between simple reflections.

• The column “φ_{ι}” describes the character of W_{G}(L) corresponding to ι. The notation for characters of Coxeter groups is as follows: 1, ε and r always represent the trivial, sign and reflection representation respectively. Other linear characters are represented by the Dynkin diagram labelled by the values of the character on the simple reflections. The notation for characters of F_{4} is that from [C] (the character ϕ′′_{2,4} factors through W (F_{4})/W (D_{4}) = W (A_{2}) and is trivial on the reflections corresponding to a short root; the character ϕ′_{2,4} is deduced from it by the diagram automorphism). The characters of W (B_{n}) are parametrized in the usual way by pairs of partitions.

G  C  Dynkin-Richardson A(u)  ι  L  W_{G}(L)  φ_{ι}

G_{2}  G_{2}(a_{1})  •^{2} > •^{0}  W (A_{2}) 1  T  W (G_{2})  r

r  T  W (G_{2})  •^{−1}> •^{1}
ε  G  1  1

F_{4}  F_{4}(a_{1})  •^{2} •^{2} > •^{0} •^{2}  W (A_{1}) 1  T  W (F_{4})  r
ε  T  W (F_{4})  ϕ′_{2,4}

E_{6}  E_{6}(a_{1})  •^{2} •^{2} _{•2} •^{0} •^{2} •^{2}  ℤ/3ℤ 1  T  W (E_{6})  r

ζ ∘ ∘ _{•b} •^{a} ∘ ∘  •^{b} > •^{a}  •^{−1}> •^{1}

ζ^{2} same description. The cuspidal local system is the other one on the regular class of L ≃ SL_{3} ×_{Z(SL_{3})} SL_{3}

E_{7}  E_{7}(a_{1}) •^{2} •^{2} •^{2} _{•2} •^{0} •^{2} •^{2} W (A_{1}) 1  T  W (E_{7})  r

ε ∘ •^{a} ∘ ∘ •^{b} •^{c} •^{d} •^{a} •^{b} < •^{c} •^{d}  ϕ′′_{2,4}

E_{8}  E_{8}(a_{1}) •^{2} •^{2} •^{2} •^{2} _{•2} •^{0} •^{2} •^{2} 1  1  T  W (E_{8})  r

{A_{n} \atop n even} (1, n-1)  •^{2}⋯•^{2} •^{0} •^{2}⋯•^{2}  1  1  T  W (A_{n})  r

{A_{n} \atop n odd} (1, n-1) •^{2}⋯•^{2} •^{1} •^{1} •^{2}⋯•^{2}  1  1  T  W (A_{n})  r

B_{n} (1, 1, 2n-1)  •^{2}⋯•^{2} > •^{0}  W (A_{1}) 1  T  W (B_{n})  r
ε  T  W (B_{n})  (1.n-1, ∅)

C_{2}  (2, 2)  •^{0} < •^{2}  W (A_{1}) 1  T  W (C_{2})  r
ε  T  W (C_{2})  (∅, 2)

{C_{n} \atop n > 2} (2, 2n-2)  •^{2}⋯•^{2} •^{0} < •^{2}  W (A_{1})^{2} (1, 1)  T  W (C_{n})  r

(ε, ε)  T  W (C_{n})  (∅, n)
(ε, 1) •⋯• • • <∘ W (C_{n−1})  (∅, 1.n-2)
(1, ε) •⋯• ∘ ∘ <∘ W (C_{n−3})  1

G  C Dynkin-Richardson A(u)  ι  L  W_{G}(L)  φ_{ι}
{D_{n} \atop n odd} (3, 2n-3)  •^{2}⋯•^{2} _{•2} •^{0} •^{2}  ℤ/4ℤ  1  T  W (D_{n})  r
−1  •⋯• ∘ ∘ ∘ ∘  W (B_{n−2}) (1.n-3, ∅)
i  \begin{matrix}∘ & • & ⋯ & ∘ & • & ∘ & ∘ & ∘ & ∘ & ∘ \\ (type D_{5} × A_{1}^{(n−5)/2}) \end{matrix} W (B_{\frac{n−5}{2}} )  1

−i same description. The cuspidal local system is also parametrized by −i on the D_{5} component of L

{D_{n} \atop n even} (3, 2n-3)  •^{2}⋯•^{2} _{•2} •^{0} •^{2}  W (A_{1})^{2} (1, 1)  T  W (D_{n})  r
(−1, 1)  •⋯• ∘ ∘ ∘ ∘  W (B_{n−2}) (1.n-3, ∅)
(1, −1) \begin{matrix}∘ & • & ⋯ & ∘ & • & ∘ & • & • & ∘ \\ (type A_{1}^{n/2}) \end{matrix} W (B_{n/2}) (∅, n/2)

(−1, −1) \begin{matrix}∘ & • & ⋯ & ∘ & • & ∘ & ∘ & • & • \\ (type A_{1}^{n/2}) \end{matrix} W (B_{n/2}) (∅, n/2)

A more precise description of the local systems when G = Spin_{2n} (the simply connected semi-simple group of type D_{n}) is as follows: A_{SO_{2n}}(u) is isomorphic to W (A_{1}); when n is odd it is the unique subgroup of order 2 of A_{G}(u), while when n is even it is the first W (A_{1}) in A_{G}(u).

References

[C] R. W. Carter, “Finite groups of Lie type: conjugacy classes and complex characters”, Wiley-Interscience (1985).

[DL] P. Deligne and G. Lusztig, “Representations of reductive groups over finite fields”, Annals of Math. 103 (1976), 103–161.

[DM1] F. Digne et J. Michel, “Representations of finite groups of Lie type”, London Mathematical Society Student Texts 21 (1991).

[DM2] F. Digne et J. Michel, “On Lusztig’s parametrization of characters of finite groups of Lie type”, Astérisque 181-182 (1990), 113–156.

[DLM1] F. Digne, G.I. Lehrer and J. Michel, “The characters of the group of rational points of a reductive group with non-connected centre” Crelle’s Journal 425 (1992), 155–192.

[DLM2] F. Digne, G.I. Lehrer and J. Michel, “On Gelfand-Graev characters of reductive groups with disconnected centre”, Crelle’s Journal 491 (1997), 131–148.

[H] R. Howlett, “Normalizers of parabolic subgroups of reflection groups”, Journal of London Math. Soc. 21 (1980), 62–80.

[K1] N. Kawanaka, “Generalized Gelfand-Graev representations of exceptional simple algebraic groups over finite fields”, Invent. Math. 84 (1986), 575–616.

[K2] N. Kawanaka, “On the irreducible characters of the finite unitary groups”, J. Math. Soc. Japan 29 (1977), 425–450.

[Le] G.I. Lehrer, “Rational tori, semisimple orbits and the topology of hyperplane complements”, Comment. Math. Helv. 67 (1992), 226–251.

[L] G. Lusztig, “A unipotent support for irreducible representations”, Advances in Math. 94 (1992), 139–179.

[L2] G. Lusztig, “Intersection cohomology complexes on a reductive group”, Inventiones 75 (1984), 205–272.

[L3] G. Lusztig, “Character Sheaves”, Advances in Math. 56 (1985), 193–237, 57 (1985), 226–265 and 266–315, 59 (1986), 1–63, 61 (1986), 103–165.

[L4] G. Lusztig, “Green Functions and Character Sheaves”, Annals of Math. 131 (1990) 355–408.

[LS] G. Lusztig and N. Spaltenstein, “On the generalized Springer correspondence for classical groups”, Advanced Studies in Pure Math. 6 (1985), 289–316.

[S] N. Spaltenstein, “On the generalized Springer correspondence for exceptional groups”, Advanced Studies in Pure Math. 6 (1985), 317–338.

[Sp] T. Springer, “ Regular elements of finite reflection groups”, Invent. Math. 25 (1974), 159-198.