ON THE MAKAR-LIMANOV, DERKSEN INVARIANTS, AND FINITE AUTOMORPHISM GROUPS OF ALGEBRAIC VARIETIES

VLADIMIR L. POPOV^{∗}

To Peter Russell on the occasion of his 70th birthday

Abstract. A simple method of constructing a big stock of algebraic varieties with trivial Makar-Limanov invariant is described, the Derksen invariant of some varieties is computed, the generalizations of the Makar-Limanov and Derksen invariants are introduced and discussed, and some results on the Jordan property of automorphism groups of algebraic varieties are obtained.

Introduction

The subject matter of this note are automorphism groups of algebraic varieties. In Section 1 I discuss the Makar-Limanov and Derksen invariants. As is known,

they have been first introduced as the means for distinguishing the Koras-Russell threefolds from affine spaces. Since then studying varieties with certain properties of these invariants (for instance, with trivial Makar-Limanov invariant) became an independent line of research, see, e.g., [Dai], [Dub], [FZ], and references therein. At the conference Affine Algebraic Geometry, June 1–5, 2009, Montreal, I was surprised to find that a simple general method of constructing a big stock of such varieties remained unnoticed by the experts. In Section 1 I expand my comment on this point made after one of the talks and give the related proofs and some illustrating examples. Then I consider the Derksen invariant and show that in many cases in presence of an algebraic group action it coincides with the coordinate algebra. At the end of this section I introduce and discuss the natural generalizations of the Makar-Limanov and Derksen invariants. In Section 2 some results on the Jordan property of automorphism groups of algebraic varieties are obtained.

Conventions and notation.

Below variety means algebraic variety. All varieties are taken over an algebraically closed field k of characteristic zero. I use the standard conventions of [Bo] and [Sp] and the following notation.

A^{∗} is the group of units of the commutative ring A with identity. M_{n×m} is the affine space of all n × m-matrices with entries in k. A_{∗}^{1} is the punctured affine line A^{1} \ {0}.

Date: January 8, 2010.

∗ Supported by grants ���� �����������, ��������������, and the program Contemporary Problems of Theoretical Mathematics of the Russian Academy of Sciences, Branch of Mathematics.

Z_{>0} is the set of positive integers.

|M| is the number of elements of the set M.

Rad G is the radical of the linear algebraic group G.

Rad_{u}G is the unipotent radical of the linear algebraic group G.

(G, G) is the commutator subgroup of the group G.

k[X] is the k-algebra of regular function on the variety X.

k(X) is the field of rational function on the irreducible variety X. T_{x,X} is the tangent space to the variety X at the point x ∈ X. Aut(X) is the automorphism group of the variety X. Bir(X) is the group of birational automorphisms of the irreducible variety X.

Given the varieties X and Y (not necessarily affine), k[X] and k[Y] are naturally identified with the k-subalgebras of k[X × Y]. Recall that then k[X × Y] is generated by k[X] and k[Y] and, moreover, k[X × Y] = k[X] ⊗_{k} k[Y], see [SW]. If A and B are the k-subalgebras of resp. k[X] and k[Y], then the subalgebra of k[X × Y] generated by A and B is A ⊗_{k} B.

Below action of an algebraic group on an algebraic variety means algebraic action. Homomorphism of algebraic groups means algebraic homomorphism.

Let X be a variety endowed with an action of an algebraic group G. Then the natural homomorphism φ: G → Aut(X) defined by this action is called algebraic and φ(G) is called the algebraic subgroup of Aut(X). If φ is injective, φ(G) is identified with G by means of φ.

Acknowledgement. I am grateful to Yu. Prokhorov for the discussions on automorphism groups of surfaces and drawing my attention to references [C], [M].

1. The Makar–Limanov and Derksen invariants

1.1. The Makar–Limanov invariant.

Recall that the Makar-Limanov invariant of a variety X is the following k-subalgeb-ra of k[X]:

ML(X) := ⋂_{H} k[X]^{H}  (1)

where H in (1) runs over the images of all homomorphisms G_{a} → Aut(X).

Below is described a simple method of constructing varieties whose Makar-Limanov invariant is trivial (i.e., equal to k). The starting point is

Lemma 1.1. For every connected linear algebraic group G, the following are equivalent:

(i) G has no nontrivial characters; (ii) G is generated by one-dimensional unipotent subgroups; (iii) G is generated by unipotent elements; (iv) Rad G = Rad_{u}G.

THE MAKAR-LIMANOV, DERKSEN INVARIANTS, AND FINITE AUTOMORPHISM GROUPS 3

Proof. Let G_{0} be the subgroup of G generated by all one-dimensional unipotent subgroups of G; it is normal and, by [Sp, 2.2.7], closed. Since char k = 0, for every nonidentity unipotent element u ∈ G, the closure of {u^{n} | n ∈ ℤ} is a one-dimensional unipotent subgroup of G (cf., e.g., [OV, Chap. 3, §2, no. 2, Theorem 1]). Hence G_{0} coincides with the subgroup generated by all unipotent element of G. This yields (ii)⇔(iii).

Since homomorphisms of algebraic groups preserve Jordan decompositions, G_{0} is contained in the kernel of every character of G and every element of the G/G_{0} is semisimple. The latter yields that G/G_{0} is a torus (cf., e.g., [Bo, I.4.6]). Hence G has no nontrivial characters if and only if G = G_{0}. This proves (i)⇔(ii).

Since char k = 0, there is a reductive subgroup L in G such that G is the semidirect product of Rad_{u}G and L (cf., e.g. [OV, Chap. 6, Sect. 4]). Let Z and Z^{0} be resp. the center of L and the identity component of Z. Put H := (L, L)Rad_{u}G. Then Z^{0} is a torus, F := (L, L)∩Z^{0} is finite, L = Z^{0}(L, L), and H is connected and normal. Being connected semisimple, (L, L) has no nontrivial characters. Hence H is generated by unipotent elements. This yields H ⊆ G_{0}. As G/H is isomorphic to Z^{0}/F and the latter is a torus, all elements of G/H are semisimple. Hence H = G_{0}. Thus, (i) holds if and only if Z^{0} is the identity. Since Rad G = Z^{0}Rad_{u}G, this proves (i)⇔(iv). □

Corollary 1.2. ML(X) = ⋂_{H⊆Aut(X)} k[X]^{H}, where H runs over all connected linear algebraic subgroups of Aut(X) that have no nontrivial characters.

Theorem 1.3. Let X be a variety and let G be a connected linear algebraic subgroup of Aut(X) that has no nontrivial characters. Then

ML(X) ⊆ k[X]^{G}.  (2)

Proof. From Lemma 1.1 we infer that k[X]^{G} = ⋂_{H} k[X]^{H} where H runs over all one-parameter unipotent subgroups of G. This and (1) imply (2). □

Corollary 1.4. Maintain the notation of Theorem 1.3. If G has no nontrivial characters and k[X]^{G} = k, then ML(X) = k.

Since there are no nonconstant invariant functions on orbit closures, this yields the following.

Corollary 1.5. Maintain the notation of Theorem 1.3. If G has no nontrivial characters and X is the closure of a G-orbit, then ML(X) = k.

Corollary 1.6. Let G be a connected algebraic group that has no nontrivial characters. Let H be a reductive subgroup of G. Then G/H is an irreducible affine variety with trivial Makar-Limanov invariant.

Proof. As G acts on G/H transitively, Corollary 1.5 yields ML(G/H) = k. By [Bo, Theorem 6.8] and [PV_{2}, Theorem 4.9] reductivity of H implies that G/H is affine. □

The following generalizes Corollary 1.5.

Theorem 1.7. Let X be a variety endowed with an action of a connected linear algebraic group G. Let d be the dimension of the center of G/Rad_{u}G. If X contains a dense G-orbit, then

tr deg_{k}ML(X) ⩽ d.  (3)

Proof. Let X be the closure of the G-orbit of a point x ∈ X. The morphism G → X, g ↦ g · x, is G-equivariant with respect to the action of G on itself by left translations. Since its image is dense in X, the corresponding comorphism is a Gequivariant embedding of the k-algebras

k[X] ↪ k[G].  (4)

Let L, Z, Z^{0}, F, and H be as in the proof of Lemma 1.1. From (4) and Theorem 1.3 we then infer that

ML(X) ⊆ k[X]^{H} ↪ k[G]^{H}.  (5)

Since G/H is isomorphic to Z^{0}/F and dim Z^{0}/F = dim Z^{0} = dim Z = d, we have dim G/H = d. As k[G]^{H} is isomorphic to k[G/H], this and (5) imply the claim. □

Corollary 1.8. Let X be the closure in P^{n} of an orbit of a connected algebraic subgroup G in Aut(P^{n}). Let X̂ ⊆ k^{n+1} be the affine cone over X. Then tr deg_{k}ML(X̂) ⩽ d + 1, where d is the dimension of the center of G/Rad_{u}G.

Proof. Let Ĝ be the pullback of G with respect to the natural projection Aut(k^{n+1}) → Aut(P^{n}). Then X̂ is the closure of a Ĝ-orbit in k^{n+1} and the dimension of the center of Ĝ/Rad_{u}Ĝ is d + 1; whence the claim by Theorem 1.7. □

Lemma 1.9. For any varieties X_{1} and X_{2},

ML(X_{1} × X_{2}) ⊆ ML(X_{1}) ⊗_{k} ML(X_{2}).  (6)

Proof. Take an element f ∈ ML(X_{1} × X_{2}). Since k[X_{1} × X_{2}] is generated by k[X_{1}] and k[X_{2}], there is a decomposition

f = ∑_{i=1}^{n} s_{i}t_{i}, s_{1}, …, s_{n} ∈ k[X_{1}], t_{1}, …, t_{n} ∈ k[X_{2}].  (7)

We may (and shall) assume that t_{1}, …, t_{n} in (7) are linearly independent over k. As k[X_{1} × X_{2}] = k[X_{1}] ⊗_{k} k[X_{2}], then they are also linearly independent over k[X_{1}].

Consider an action α of G_{a} on X_{1}. Then k[X_{1}] is stable and k[X_{2}] is pointwise fixed with respect to the diagonal action of G_{a} on X_{1} × X_{2} determined by α and trivial action on X_{2}. For every element g ∈ G_{a} and this diagonal action, (1) and (7) imply that

∑_{i=1}^{n} s_{i}t_{i} = f = g · f = ∑_{i=1}^{n}(g · s_{i})t_{i}.  (8)

Since t_{1}, …, t_{n} are linearly independent over k[X_{1}], we infer from (8) that every s_{i} is invariant with respect to α. As α is arbitrary, (1) implies that s_{1}, …, s_{n} ∈ ML(X_{1}).

Hence f is decomposed as

f = ∑_{i=1}^{m} s′_{i}t′_{i}, s′_{1}, …, s′_{m} ∈ ML(X_{1}), t′_{1}, …, t′_{m} ∈ k[X_{2}],  (9)

where s′_{1}, …, s′_{m} are linearly independent over k. The same argument as above then yields t′_{1}, …, t′_{m} ∈ ML(X_{2}). Now (6) follows from (9). □

Corollary 1.10. For any varieties X_{1} and X_{2}, the following are equivalent: (i) ML(X_{1}) and ML(X_{2}) lie in ML(X_{1} × X_{2}); (ii) ML(X_{1} × X_{2}) = ML(X_{1}) ⊗_{k} ML(X_{2}).

Corollary 1.11. If ML(X_{1}) = k and ML(X_{2}) = k, then ML(X_{1} × X_{2}) = k.

Corollary 1.12. Let X_{1} and X_{2} be the varieties such that ML(X_{1}) and ML(X_{2}) are generated by units. Then ML(X_{1} × X_{2}) = ML(X_{1}) ⊗_{k} ML(X_{2}).

Proof. This follows from Corollary 1.10 since k[X_{1}]^{∗} and k[X_{2}]^{∗} lie in k[X_{1} × X_{2}]^{∗} and k[X_{1} × X_{2}]^{∗} ⊂ ML(X_{1} × X_{2}), cf. [F, 1.4]. □

Definition 1.13. A variety is called toral if it is isomorphic to a closed subvariety of a linear algebraic torus.

Note that closed subvarieties and products of toral varieties are toral.

Lemma 1.14. Let X be an affine variety.

(a) The following are equivalent: (a_{1}) X is toral; (a_{2}) k[X] is generated by k[X]^{∗}.

(b) For every finite subgroup G of Aut(X), there is a covering of X by G-stable open toral sets.

(c) If X is toral, then

(c_{1}) for every unipotent linear algebraic group H, every algebraic homomor-

phism φ: H → Aut(X) is trivial; (c_{2}) ML(X) = k[X].

Proof. (a) Every character of a linear algebraic torus T is an element of k[T]^{∗} and k[T]^{∗} is the k-linear span of the set of all characters [Bo, Sect. 8.2]; this and Definition 1.13 imply (a_{1})⇒(a_{2}).

Conversely, if (a_{2}) holds, let k[X] = k[f_{1}, …, f_{n}] for some f_{i} ∈ k[X]^{∗}. Then ι: X → A^{n}, x ↦ (f_{1}(x), …, f_{n}(x)), is a closed embedding since X is affine. The standard coordinate functions on A^{n} do not vanish on ι(X) since f_{i} does not vanish on X. Hence ι(X) ⊂ (G_{m})^{n}. This proves (a_{2})⇒(a_{1}) and completes the proof of (a).

(b) Let x be a point of X. We have to show that x is contained in a G-stable open toral subset of X. Let k[X] = k[h_{1}, …, h_{s}]. Replacing h_{i} by h_{i}+α_{i} for an appropriate α_{i} ∈ k, we may )and shall) assume that every h_{i} vanishes nowhere on the G-orbit G·x of x. Enlarging the set {h_{1}, …, h_{s}} by including in it g · h_{i} for every i and g ∈ G, we may (and shall) assume that {h_{1}, …, h_{s}} is G-stable. Then h := h_{1} ⋯ h_{s} ∈ k[X]^{G}.

Hence the affine open set X_{h} := {z ∈ X | h(z) ≠ 0} is G-stable and contains G · x. Since k[X_{h}] = k[h_{1}, …, h_{s}, 1/h] we have h_{i} ∈ k[X_{h}]^{∗} for every i. Hence, X_{h} is toral by (a). This proves (b).

(c) Consider the action of H on X determined by φ. Let H · x be the H-orbit of a point x ∈ X. Since char k = 0, H · x is isomorphic to A^{d} for some d, see [P_{1}, Cor. of Theorem 2]. Since H is unipotent and X is affine, H · x is closed in X, cf. [Bo, 4.10]. Hence H · x is toral. Since k[A^{d}]^{∗} = k^{∗}, from (a) we then infer that d = 0, i.e., x is a fixed point. This proves (c_{1}). In turn, (c_{1}) implies (c_{2}) by (1). □

Corollary 1.15. If ML(X_{1}) = k and X_{2} is toral, then ML(X_{1} × X_{2}) = k[X_{2}].

Utilizing the above statements one gets many interesting varieties with trivial Makar-Limanov invariant. The following construction is typical (but not the only possible, see Example 1.21).

Let G be a connected semisimple algebraic group acting on an affine variety X. By Hilbert’s theorem, k[X]^{G} is a finitely generated k-algebra. Let k[X]^{G} = k[f_{1}, …, f_{n}]. For every α_{1}, …, α_{n} ∈ k, denote by X(α_{1}, …, α_{n}) the closed subvariety of X whose underlying topological space is {x ∈ X | f_{1}(x) = α_{1}, …, f_{n}(x) = α_{n}} (warning: in general, the ideal (f_{1} − α_{1}, …, f_{n} − α_{n}) of k[X] is not radical). Let Y be a Gstable closed suvariety of X. It is well-known that k[X]^{G} → k[Y]^{G}, f ↦ f|_{Y}, is an epimorphism [PV_{2}, 3.4]. Hence k[Y]^{G} = k if and only if Y is contained in some X(α_{1}, …, α_{n}). From Theorem 1.3 we then infer that the Makar-Limanov invariant of every G-stable closed subvariety of X(α_{1}, …, α_{n}) is trivial.

There are many instances where f_{1}, …, f_{n} can be explicitly described. E.g., classical invariant theory yields such a description for a number of finite-dimensional modules X of classical linear groups G; for some of them, it is proved that (f_{1} − α_{1}, …, f_{n} − α_{n}) is radical. If the latter happens, one obtains the instances of affine algebras with trivial Makar-Limanov invariant that are explicitly described by equations.

Below are several illustrating examples.

Example 1.16 (Closures of adjoint orbits). Let f_{s} be the sum of all principal s × sminors of the n × n-matrix (x_{ij}) where x_{11}, …, x_{nn} are variables considered as the standard coordinate functions on M_{n×n}. For α_{1}, …, α_{n} ∈ k,

M_{n×n}(α_{1}, …, α_{n}) := {a ∈ M_{n×n} | f_{1}(a) = α_{1}, …, f_{n}(a) = α_{n}}

is the set of all matrices whose characteristic polynomial is t^{n} + ∑_{i=1}^{n}(−1)^{i}α_{i}t^{n−i}.

Consider the action of SL_{n} on M_{n×n} by conjugation. Then k[M_{n×n}]^{SL^{n}} is freely generated by f_{1}, …, f_{n} (cf., e.g., [PV_{2}, 0.6]). Moreover, M_{n×n}(α_{1}, …, α_{n}) is irreducible and the ideal (f_{1} − α_{1}, …, f_{n} − α_{n}) of k[M_{n×n}] is radical (see the next paragraph). Hence, M_{n×n}(α_{1}, …, α_{n}) is a closed subvariety M_{n×n} of such that

ML(M_{n×n}(α_{1}, …, α_{n})) = k

and k[…, x_{ij}, …]/(f_{1} − α_{1}, …, f_{n} − α_{n}) is the k-domain with trivial Makar-Limanov invariant.

This admits the following generalization. Let G be a connected reductive algebraic group and let Lie(G) be the Lie algebra of G endowed with the adjoint action of G. By [K] the graded k-algebra k[Lie(G)]^{G} is free and, for every minimal system of its homogeneous generators f_{1}, …, f_{r} and constants α_{1}, …, α_{r} ∈ k,

(i) Lie(G)(α_{1}, …, α_{r}) := {a ∈ Lie(G) | f_{1}(a) = α_{1}, …, f_{r}(a) = α_{r}} is the closure of a G-orbit;

(ii) the ideal (f_{1} − α_{1}, …, f_{r} − α_{r}) of k[Lie(G)] is radical. Since the center Z of G acts trivially on Lie(G) and G/Z is semisimple, this yields

ML(Lie(G)(α_{1}, …, α_{r})) = k

and k[Lie(G)]/(f_{1} −α_{1}, …, f_{r} −α_{r}) is the k-domain with trivial Makar-Limanov invariant.

For G = GL_{n}, we have Lie(G)(α_{1}, …, α_{r}) = M_{n×n}(α_{1}, …, α_{n}).

Example 1.17 (Determinantal varieties). Given positive integers n ⩾ m > r, let {x_{ij} | i = 1, …, n, j = 1, …, m} be the set of variables considered as the standard coordinates functions on M_{n×m}. Let I_{n,m,r} be the ideal of k[M_{n×m}] = k[…, x_{ij}, …] generated by all (r + 1) × (r + 1)-minors of the matrix (x_{ij}). Then I_{n,m,r} is radical, cf., e.g., [Pr]. The (affine) determinantal variety D_{n,m,r} is the subvariety of M_{n×m} defined by I_{n,m,r}. Its underlying set is that of n × m-matrices of rank ⩽ r. It is stable with respect to the action of SL_{n} ×SL_{m} on M_{n×m} by (g, h)·a := gah^{−1} and contains a dense orbit. Whence

ML(D_{n,m,r}) = k

and k[M_{n×m}]/I_{n,m,r} is the k-domain with trivial Makar-Limanov invariant.

Example 1.18 (S-varieties in the sense of [PV_{1}]). Denote by S^{d}k^{n} the dth symmetric power of the coordinate vector space (of columns) k^{n}. The natural SL_{n}-action on k^{n} induces that on S^{d}k^{n}. The (affine) Veronese morphism

ν_{n}^{d}: k^{n} → S^{d}k^{n}, v ↦ v^{d},

is SL_{n}-equivariant. Its image ν_{n}^{d}(k^{n}) is closed and contains a dense SL_{n}-orbit. The ideal of ν_{n}^{d}(k^{n}) is generated by all 2 × 2-minors of a certain symmetric matrix whose entries are the coordinates on S^{d}k^{n}, cf. [Ha]. Thus,

ML(ν_{n}^{d}(k^{n})) = k

and the coordinate algebra of ν_{n}^{d}(k^{n}) is the explicitly described k-domain with trivial Makar-Limanov invariant.

More generally, the following combination of the Veronese and Segre morphisms

ν_{n_{1},...,n_{s}}^{d_{1},...,d_{s}}: k^{n_{1}} × ⋯ × k^{n_{s}} → S^{d_{1}}k^{n_{1}} ⊗ ⋯ ⊗ S^{d_{s}}k^{n_{s}},
(v_{1}, …, v_{s}) ↦ v^{d_{1}} ⊗ ⋯ ⊗ v^{d_{s}},

is equivariant with respect to the natural SL_{n_{1}} ×⋯×SL_{n_{s}}-actions, its image is closed and contains a dense orbit. Whence,

ML(ν_{n_{1},...,n_{s}}^{d_{1},...,d_{s}}(k^{n_{1}} × ⋯ × k^{n_{s}})) = k.

In turn, this construction admits a further generalization. Namely, any matrix

A = (\begin{matrix}a_{11} … a_{1s} \\ … & … & … \\ a_{r1} … a_{rs} \end{matrix})

with the entries in Z_{>0} defines the diagonal morphism

ν_{n_{1},...,n_{s}}^{A} := ν_{n_{1},...,n_{s}}^{a_{11},...,a_{1s}} × ⋯ × ν_{n_{1},...,n_{s}}^{a_{r1},...,a_{rs}}.

This morphism is SL_{n_{1}} × ⋯ × SL_{n_{s}}-equivariant and its image

H_{n_{1},...,n_{s}}^{A} := ν_{n_{1},...,n_{s}}^{A}(k^{n_{1}} × ⋯ × k^{n_{s}})

is closed and contains a dense orbit. Thus,

ML(H_{n_{1},...,n_{s}}^{A}) = k.

In fact, H_{n_{1},...,n_{s}}^{A}’s are special examples of varieties with trivial Makar-Limanov invariant obtained by the following general construction [PV_{1}].

Let G be a connected semisimple algebraic group and let E(λ) be a simple Gmodule with the highest weight λ (with respect to a fixed Borel subgroup and its maximal torus). Let v_{λ_{i}} be a highest vector of E(λ_{i}). For x = v_{λ_{1}} + ⋯ + v_{λ_{s}} ∈ E(λ_{1}) ⊕ ⋯ ⊕ E(λ_{s}), let X(λ_{1}, …, λ_{s}) be the closure of the G-orbit of x. Up to G-isomorphism, X(λ_{1}, …, λ_{s}) depends only on λ_{1}, …, λ_{s}. By Corollary 1.5

ML(X(λ_{1}, …, λ_{s})) = k.

The ideal I(λ) of X(λ) in k[E(λ)] is generated by quadratic forms that can be explicitly described. Namely, k[E(λ)] is the symmetric algebra of the dual G-module E(λ)^{∗} = E(λ^{∗}). The submodule S^{2}E(λ^{∗}) of the G-module k[E(λ)] contains a unique simple submodule with the highest weight 2λ^{∗}, the Cartan component of S^{2}E(λ^{∗}). Hence S^{2}E(λ^{∗}) contains a unique submodule L complement to the Cartan component. This L generates I(λ), see [Br, Theorem 4.1].

For G = SL_{n_{1}} × ⋯ × SL_{n_{s}} and λ_{i} = a_{i1}ϖ_{1}^{(1)} + ⋯ + a_{is}ϖ_{1}^{(s)} where ϖ_{1}^{(j)} is the highest weight of the natural SL_{n_{j}}-module k^{n_{j}}, we have

X(λ_{1}, …, λ_{s}) = H_{n_{1},...,n_{s}}^{A}.

Example 1.19 (Irreducible affine surfaces quasihomogeneous with respect to an algebraic group in the sense of [G]). By [P_{1}], up to isomorphism, such surfaces are exhausted by the following list (we maintain the notation of Example 1.18):

(i) smooth surfaces:

A^{2}, A^{1} × A_{∗}^{1}, A_{∗}^{1} × A_{∗}^{1}, (P^{1} × P^{1}) \ Δ, P^{2} \ C,  (10)

where Δ is the diagonal in P^{1} × P^{1}, and C is a nondegenerate conic in P^{2}; (ii) singular surfaces:

V(n_{1}, …, n_{r}) := H_{2}^{A} for A = (n_{1}, …, n_{r})^{T}, n_{1}, …, n_{r} ⩾ 2.

Each of these surfaces but A^{1} × A_{∗}^{1} and A_{∗}^{1} × A_{∗}^{1} admits an SL_{2}-action with a dense orbit. Namely, (P^{1}×P^{1})\Δ = SL_{2}/T and P^{2}\C = SL_{2}/N(T), where T is a maximal torus of SL_{2} and N(T) its normalizer, see [P_{1}, Lemma 2]. The surface V(n_{1}, …, n_{r}) is the closure of the SL_{2}-orbit of v_{1} + ⋯ + v_{r} ∈ R_{n_{1}} ⊕ ⋯ ⊕ R_{n_{r}}, where v_{i} is a highest vector of the simple SL_{2}-module R_{n_{i}} of dimension n_{i} + 1 (such a module is unique up to isomorphism),

see [P_{1}, §2]. By Corollary 1.5 this yields

ML((P^{1} × P^{1}) \ Δ) = ML(P^{2} \ C) = ML(V(n_{1}, …, n_{r})) = k,  (11)

As A_{∗}^{m} := (A_{∗}^{1})^{m} is toral and ML(A^{n}) = k, Corollary 1.15 implies that

ML(A^{n} × A_{∗}^{m}) = k[A_{∗}^{m}].  (12)

From (12) we get the Makar-Limanov invariants of the remaining three surfaces in (10).

Example 1.20 (Irreducible affine threefolds quasihomogeneous with respect to an algebraic group in the sense of [G]). We maintain the notation of Examples 1.18 and 1.19. Identify Pic((P^{1} × P^{1}) \ Δ) with Z by a fixed isomorphism φ. Let X_{n} be the total space of the one-dimensional vector bundle over (P^{1} × P^{1}) \ Δ corresponding to n ∈ Z, and let X_{n}^{∗} be the complement of the zero section in X_{n}. In fact, X_{n} is isomorphic to X_{−n} and X_{n}^{∗} to X_{−n}^{∗}, so X_{n} and X_{n}^{∗} do not depend on the choice of φ, see [P_{2}].

The group Pic(P^{2} \ C) has order 2. Let Y_{0} and Y_{1} be the total spaces of, resp., trivial and nontrivial one-dimensional vector bundles over P^{2} \ C. Let Y_{n}^{∗} be the complement of the zero section in Y_{n}.

Let T̃, Õ, Ĩ, and D̃_{n} be, resp., the binary tetrahedral, octahedral, icosahedral, and dihedral subgroup of order 4n in SL_{2}. Put S_{3} = SL_{2}/T̃, S_{4} = SL_{2}/Õ, S_{5} = SL_{2}/Ĩ, and W_{n} = SL_{2}/D̃_{n}.

By [P_{1}] up to isomorphism irreducible affine threefolds quasihomogeneous with respect to an algebraic group in the sense of [G] are exhausted by the following list:

(i) smooth threefolds:

X_{n}, X_{n}^{∗}, W_{n}, Y_{0}, Y_{0}^{∗}, Y_{1}^{∗}, S_{3}, S_{4}, S_{5},

A^{3}, A^{2} × A_{∗}^{1}, A^{1} × A_{∗}^{2}, A_{∗}^{3}; (13)

(ii) singular threefolds:

P(A) := H_{3}(A) where all entries of A are ⩾ 1, Q(B) := H_{2,2}(B) where rk B = 1.

By construction, S_{3}, S_{4}, S_{5}, W_{n} are homogeneous with respect to SL_{2} while P(A) and Q(B) admit an action of resp. SL_{2}×SL_{2} and SL_{3} with a dense orbit. In fact, X_{n}^{∗} for n ≠ 0 is homogeneous with respect to SL_{2} as well (it is the quotient of SL_{2} modulo a cyclic subgroup of order |n|). By [P_{2}, Theorem 9] every X_{n} is homogeneous with respect to the nonreductive linear algebraic group SL_{2,|n|} := SL_{2} ⋉R_{|n|} (see Example 1.19); the radical of SL_{2,|n|} is unipotent. By [P_{2}, Prop. 18], Y_{0} is homogeneous with respect to SL_{2,d} for every even d > 0. From Theorem 1.7 we then deduce that

ML(S_{3}) = ML(S_{4}) = ML(S_{5}) = ML(Y_{0}) = k,
ML(X_{n}) = ML(W_{n}) = ML(P(A)) = ML(Q(B)) = k,
ML(X_{n}^{∗}) = k for n ≠ 0.

As X_{0}^{∗} = ((P^{1} × P^{1}) \ Δ) × A_{∗}^{1} and Y_{0}^{∗} = (P^{2} \ C) × A_{∗}^{1}, we deduce from (11) and Corollary 1.15 that

ML(X_{0}^{∗}) = ML(Y_{0}^{∗}) = k[A_{∗}^{1}].

By [P_{2}, Prop. 16], Y_{1}^{∗} is homogeneous with respect to SL_{2} × G_{m}. Since the latter is a reductive group with one-dimensional center, Theorem 1.7 implies that tr deg_{k}ML(Y_{1}^{∗}) ⩽ 1. On the other hand, by [P_{2}, Prop. 19], k[Y_{1}^{∗}]/k^{∗} is a free abelian group of rank 1. Since k[X]^{∗} ⊆ ML(X) for every X, this yields

tr deg_{k}ML(Y_{1}^{∗}) = 1.

Finally, (12) yields the Makar-Limanov invariants of the last four threefolds in (13).

Example 1.21 (Schubert varieties). Let G be a connected semisimple algebraic group and let PE be the projective space of 1-dimensional linear subspaces in a nonzero simple G-module E. There is a unique closed G-orbit O in PE. Let U be a maximal unipotent subgroup of G. There are only finitely many U-orbits in O; their closures are called Schubert varieties, cf., e.g., [Sp, 8.3–8.5]. Let X ⊆ O be a Schubert variety and let X̂ be the affine cone over X in E. As U is unipotent, Corollary 1.8 yields

tr deg_{k}ML(X̂) ⩽ 1.

The ideal of X̂ in k[E] is generated by certain forms of degree ⩽ 2 that admit an explicit description, see, e.g., [BL, 2.10].

1.2. The Derksen invariant.

Let X be a variety. Recall that the Derksen invariant D(X) of X is the ksubalgebra of k[X] generated by all k[X]^{H}’s where H runs over all subgroups of Aut(X) isomorphic to G_{a}. If there are no such subgroups, we put D(X) = ∅.

Example 1.22. If X is toral, then D(X) = ∅ by Lemma 1.14(c_{1}).

In this section we deduce some information on D(X) in case when Aut(X) contains a connected noncommutative reductive algebraic subgroup.

Recall that if an algebraic group G acts linearly on a (not necessarily finite-dimen-sional) k-vector space V, then the G-module V is called algebraic if every element of V is contained in an algebraic finite-dimensional G-submodule of V, cf., e.g., [PV_{2}, 3.4].

The starting point is Lemma 1.23. Let G be a connected noncommutative reductive algebraic group. Then every algebraic G-module V is a k-linear span of the set

⋃_{H⊂G} V^{H},  (14)

where H in (14) runs over all one-parameter unipotent subgroups of G.

Proof. The assumptions that G is reductive, char k = 0, and V is algebraic imply that V is a sum of simple G-submodules. Hence we may (and shall) assume that V is a nonzero simple G-module. Since G is a connected noncommutative reductive algebraic group, it contains a one-dimensional unipotent subgroup U (indeed, since (G, G) is a nontrivial semisimple group, a root subgroup of (G, G) with respect to a maximal torus may be taken as U). By the Lie–Kolchin theorem V^{U} ≠ {0}. Let v be a nonzero vector of V^{U}. As g · v ∈ V^{gUg^{−1}} for every element g ∈ G, the G-orbit G · v of v is contained in set (14). Hence the k-linear span of G · v is contained in the k-linear span of this set. But the k-linear span of G · v is G-stable and therefore coincides with V since V is simple. This completes the proof. □

Theorem 1.24. Let X be a variety. If Aut(X) contains a connected noncommutative reductive algebraic subgroup, then

D(X) = k[X].  (15)

Proof. Let G be a connected noncommutative reductive algebraic subgroup of Aut(X). Since the G-module k[X] is algebraic (see [PV_{2}, Lemma 1.4]), the claim follows from Lemma 1.23 and the definition on D(X). □

Remark 1.25. The following are equivalent:

(i) Aut(X) contains a connected noncommutative reductive algebraic subgroup; (ii) Aut(X) contains SL_{2} or PSL_{2}. Indeed, SL_{2} and PSL_{2} are connected noncommutative reductive algebraic groups

and every connected noncommutative reductive algebraic group contains SL_{2} or PSL_{2}, cf. [Bo, Theorem 13.18(4)], [Sp, 7.2.4].

The following example shows that the assumption of noncommutativity in Theorem 1.24 cannot be dropped.

Example 1.26. By [Der], for the Koras–Russell cubic threefold X, the following inequality distinguishing X from A^{3} holds:

D(X) ≠ k[X].  (16)

On the other hand, since X is defined in A^{4} by x_{1}+x_{1}^{2}x_{2}+x_{3}^{2}+x_{4}^{3} = 0 where x_{1}, …, x_{4} are the standard coordinate functions on A^{4}, it is stable with respect to the action of G_{m} on A^{4} defined by t · (a_{1}, a_{2}, a_{3}, a_{4}) = (t^{6}a_{1}, t^{−6}a_{2}, t^{3}a_{3}, t^{2}a_{4}). Hence Aut(X) contains a one-dimensional connected commutative reductive subgroup, cf. [DM-JP, Sect. 3].

One can apply Theorem 1.24 to proving that, for some varieties X, there are no connected noncommutative reductive algebraic subgroups in Aut(X).

Example 1.27. For the Koras–Russell cubic threefold X, Theorem 1.24 and (16) imply that Aut(X) contains no connected noncommutative reductive algebraic subgroups.

Since A^{n} for n ⩾ 2 contains a connected noncommutative reductive algebraic subgroup (for instance, GL_{n}), the next corollary generalizes the well-known fact that D(X × A^{n}) = k[X × A^{n}] for n ⩾ 2 (see, e.g., [CM]).

Corollary 1.28. Let Z be a variety such that Aut(Z) contains a connected noncommutative reductive algebraic subgroup. Then, for every variety X,

D(X × Z) = k[X × Z].  (17)

Proof. Consider the natural action of Aut(Z) on Z and its trivial action on X. Then the diagonal action of Aut(Z) on X ×Z identifies Aut(Z) with a subgroup of Aut(X × Z). Whence the claim by Theorem 1.24. □

The following example shows that the assumption of noncommutativity in Corollary 1.28 cannot be dropped.

Example 1.29. Let x_{1}, x_{2} be the standard coordinate functions on A^{2}. The principal open set Y in A^{2} defined by x_{1} ≠ 0 is isomorphic to A_{∗}^{1} × A^{1} and

k[Y] = k[t, t^{−1}, s],  where t := x_{1}|_{Y} , s := x_{2}|_{Y} .  (18)

Since t is the unit of k[Y], for every action of G_{a} on Y we have

t, t^{−1} ∈ k[Y]^{G_{a}}.  (19)

As, clearly, Aut(Y) contains a one-dimensional unipotent subgroup, (19) and the definition of D[Y] yield k[t, t^{−1}] ⊆ D[Y]. We also deduce from (19) that, for every point y ∈ Y, the G_{a}-orbit of y lies in the line defined by the equation t = t(y). But this orbit is closed in Y since Y is affine and G_{a} is unipotent, cf. [Bo, 4.10]. Hence, if y is not a fixed point, this orbit coincides with the aforementioned line. Therefore, if G_{a} acts on Y nontrivially, t separates orbits in general position. Since char k = 0, by [PV_{2}, Lemma 2.1] this means that k(Y)^{G_{a}} = k(t). Whence by (18) we have k[Y]^{G_{a}} = k[t, t^{−1}]. From this, (12) and (18) we then infer that

k[t, t^{−1}] = ML(A_{∗}^{1} × A^{1}) = D(A_{∗}^{1} × A^{1}) ⊊ k[A_{∗}^{1} × A^{1}] = k[t, t^{−1}, s].

Thus, (17) does not hold for X = A_{∗}^{1}, Z = A^{1} while both Aut(A_{∗}^{1}) and Aut(A^{1}) contain a one-dimensional connected commutative reductive algebraic subgroup.

Theorem 1.30. If X_{i} is a variety such that ML(X_{i}) ≠ k[X_{i}], i = 1, 2, then

D(X_{1} × X_{2}) = k[X_{1} × X_{2}].

Proof. As ML(X_{1}) ≠ k[X_{1}], there is a nontrivial G_{a}-action α on X_{1}. The diagonal G_{a}-action on X_{1} × X_{2} determined by α and trivial action on X_{2} is a nontrivial G_{a}- action for which k[X_{2}] lies in the algebra of invariants. Hence, k[X_{2}] ⊆ D(X_{1} × X_{2}). Similarly, k[X_{2}] ⊆ D(X_{1} × X_{2}). As k[X_{1} × X_{2}] is generated by k[X_{1}] and k[X_{2}], the claim follows. □

Example 1.31. If X is the Koras–Russell cubic threefold X, then D(X) ≠ k[X] by [Der]. But for the square of X we have D(X × X) = k[X × X] — since ML(X) = k[x_{1}|_{X}] ≠ k[X] (cf., e.g., [F, Chap. 9]), this follows from Theorem 1.30.

1.3. Generalizations.

The Makar-Limanov and Derksen invariants can be naturally generalized.

Namely, let X be a variety and let F be an algebraic group.

Definition 1.32. The F -kernel of X is the following k-subalgebra of k[X]:

Ker_{F} (X) := ⋂_{H} k[X]^{H},  (20)

where H in (20) runs over the images of all algebraic homomorphisms F → Aut(X).

Definition 1.33. The F -envelope of X is the k-subalgebra

Env_{F} (X)

of k[X] generated by all k[X]^{H}’s where H runs over all subgroups of Aut(X) isomorphic to H. If there are no such subgroups, we put Env_{F} (X) = ∅.

Example 1.34. The definitions imply that

Ker_{G_{a}}(X) = ML(X), Env_{G_{a}}(X) = D(X).

Definition 1.35. We say that an algebraic group G is F -generated if it is generated by the images of all homomorphisms F → G.

Examples 1.36. (1) By Lemma 1.1 a connected linear algebraic group G is G_{a}- generated if and only if G has no nontrivial characters that, in turn, is equivalent to the condition Rad G ≠ Rad_{u}G.

(2) Every connected reductive algebraic group G is G_{m}-generated. This is clear if G is a torus. In the general case this follows from the fact that the subgroup generated by algebraic subgroups is closed (see, e.g., [Sp, 2.2.7]) and the union of maximal tori of G contains a dense open subset ([Sp, 6.4.5(iii), 7.6.4(ii)].

(3) Clearly, the subgroup generated by the images of all homomorphisms F → G is normal. Hence, if G is simple as abstract group and there exists a nontrivial homomorphism F → G, then G is F -generated.

The following are the generalizations of the above statements on ML(X) and D(X).

Theorem 1.37. If a variety X is endowed with an action of an F -generated algebraic group G, then Ker_{F} (X) ⊆ k[X]^{G}.

Proof. This follows from Definitions 1.32 and 1.35. □

Corollary 1.38. If a variety X is endowed with an action of an F -generated algebraic group G and X contains a dense G-orbit, then Ker_{F} (X) = k.

Corollary 1.39. If H is a reductive subgroup of an F -generated linear algebraic group G, then G/H is an affine variety with Ker_{F} (G/H) = k.

Corollary 1.40. Let X be an irreducible variety. If there is an action of G_{m} on X with a fixed point and without other closed orbits, then

Ker_{G_{m}}(X) = k.  (21)

Proof. The assumptions imply that the fixed point is unique and lies in the closure of every G_{m}-orbit; whence k[X]^{G_{m}} = k. In turn, this and (20) yield (21). Note that, in fact, X is affine [P_{4}]. □

Corollary 1.41. Let X be a closed subset of P^{n} and let X̂ ⊆ k^{n+1} be the affine cone over X. Then Ker_{G_{m}}(X̂) = k.

Example 1.42. Consider the case F = G_{m}. If G is a connected reductive subgroup of Aut(X) and X contains a dense G-orbit, then Corollary 1.38 and Example 1.36(2) imply that (21) holds. In particular, this is so for every toric variety X; for instance,

Ker_{G_{m}}(A^{n} × A_{∗}^{m}) = k.

(compare with (12)). Applying this to the varieties considered in Examples 1.16–1.20, we see that (21) holds for every X from the following list:

Lie(G)(α_{1}, …, α_{n}) (see Example 1.16);
D_{n,m,r} (see Example 1.17);
X(λ_{1}, …, λ_{s}) (see Example 1.18);

(P^{1} × P^{1}) \ Δ, P^{2} \ C, V(n_{1}, …, n_{r}) where n_{1}, …, n_{r} ⩾ 2 (see Example 1.19); S_{3}, S_{4}, S_{5}, W_{n}, P(A), Q(B), X_{n}^{∗} where n ≠ 0, Y_{1}^{∗} (see Example 1.20).

The threefold X_{n} from Example 1.20 is homogeneous with respect to SL_{2,|n|}. One can prove that SL_{2,|n|} is G_{m}-generated; whence Ker(X_{n}) = k.

The remaining threefolds X_{0}^{∗}, Y_{0}, and Y_{0}^{∗} from Example 1.20 are considered in Example 1.45 below.

The same proof as that of Lemma 1.9 yields

Lemma 1.43. For any varieties X_{1} and X_{2},

Ker_{F} (X_{1} × X_{2}) ⊆ Ker_{F} (X_{1}) ⊗_{k} Ker_{F} (X_{2}).

Corollary 1.44. For any varieties X_{1} and X_{2}, the following are equivalent: (i) Ker_{F} (X_{1}) and Ker_{F} (X_{2}) lie in Ker_{F} (X_{1} × X_{2}); (ii) Ker_{F} (X_{1} × X_{2}) = Ker_{F} (X_{1}) ⊗_{k} Ker_{F} (X_{2}).

Example 1.45. Since X_{0}^{∗} = ((P^{1} × P^{1}) \ Δ) × A_{∗}^{1}, Y_{0} = (P^{2} \ C) × A^{1}, and Y_{0}^{∗} = (P^{2} \ C) × A_{∗}^{1} (see Example 1.20), we deduce from Lemma 1.43 and Example 1.42 that Ker_{G_{m}}(X_{0}^{∗}) = Ker_{G_{m}}(Y_{0}) = Ker_{G_{m}}(Y_{0}^{∗}) = k.

Lemma 1.46. For any connected algebraic group F that has no nontrivial characters,

k[X]^{∗} ⊆ Ker_{F} (X).  (22)

Proof. Let H be the image of an algebraic homomorphism F → Aut(X). We claim that k[X]^{∗} ⊆ k[X]^{H}; by virtue of Definition 1.32 this inclusion implies (22). Since H is connected, every irreducible component of X is H-stable, so proving the claim we may (and shall) assume that X is irreducible. In this case every element of k[X]^{∗} is H-semiinvariant by [PV_{2}, Theorem 3.1], hence lies in k[X]^{H} since H has no nontrivial characters. This completes the proof. □

Corollary 1.47. Let F be a connected algebraic group that has no nontrivial characters. Let X_{1} and X_{2} be varieties such that Ker_{F} (X_{1}) and Ker_{F} (X_{2}) are generated by units. Then Ker_{F} (X_{1} × X_{2}) = Ker_{F} (X_{1}) ⊗_{k} Ker_{F} (X_{2}).

Lemma 1.48. Let G be a connected reductive algebraic group of rank ⩾ 2. Then every algebraic G-module V is a k-linear span of the set

⋃_{H⊆G} V^{H},  (23)

where H in (23) runs over all one-dimensional tori of G.

Proof. Like in the proof of Lemma 1.23 we may (and shall) assume that V is a nonzero simple G-module. Let T be a maximal torus of G and let v ∈ V, v ≠ 0, be a weight vector of T. Since dim T ⩾ 2, the T -stabilizer T_{v} of v is a diagonalizable group of dimension ⩾ 1. Hence T_{v} contains a one-dimensional torus S. Thus, v ∈ V^{S}. Like in the proof of Lemma 1.23 we then conclude that the orbit G · v is contained in set (23). Since V is simple, the k-linear span of G · v coincides with V; whence the claim. □

Theorem 1.49. Let X be a variety such that Aut(X) contains a connected reductive algebraic group G of rank ⩾ 2. Then

Env_{G_{m}}(X) = k[X].

Proof. Since the G-module k[X] is algebraic, the claim follows from Lemma 1.48 and Definition 1.33. □

Remark 1.50. Clearly, Env_{G_{m}}(A^{1}) = k. This shows that in Lemma 1.48 and Theorem 1.49 the condition “⩾ 2” can not be replaced by “⩾ 1”.

2. Finite automorphism groups of algebraic varieties

2.1. Jordan groups.

The following definition is inspired by the classical Jordan theorem (Theorem 2.2).

Definition 2.1. A group G is called a Jordan group if there exists a positive integer J_{G}, depending only on G, such that every finite subgroup K of G contains a normal abelian subgroup whose index in K is at most J_{G}.

Jordan’s theorem (see [CR, Theorem 36.13]) can be then reformulated as follows:

Theorem 2.2. Every GL_{n}(k) is Jordan.

Remark 2.3. For G = GL_{n}(k), the explicit upper bounds J_{G} are known, see [CR, §36].

Since subgroups of Jordan groups are Jordan and every linear algebraic group is isomorphic to a subgroup of some GL_{n}(k) (see [Sp, 2.3.7]), Theorem 2.2 yields the following more general

Theorem 2.4. Every linear algebraic group is Jordan.

Lemma 2.5. Let H be a finite normal subgroup of a group G. If G is Jordan, then G/H is Jordan.

Proof. Let π: G → G/H be the natural projection and let K be a finite subgroup of G/H. Since H is finite, π^{−1}(K) is a finite subgroup of G. As G is Jordan, π^{−1}(K) contains a normal abelian subgroup A whose index is at most J_{G}. Hence π(A) is a normal abelian subgroup of K whose index is at most J_{G}. □

Lemma 2.6. If the groups G_{1} and G_{2} are Jordan, then G_{1} × G_{2} is Jordan.

Proof. Let π_{i}: G := G_{1} × G_{2} → G_{i} be the natural projection and let K be a finite subgroup of G. Since G_{i} is Jordan, K_{i} := π_{i}(K) contains an abelian normal subgroup A_{i} such that

[K_{i} : A_{i}] ⩽ J_{K_{i}}.  (24)

The subgroup Ã_{i} := π_{i}^{−1}(A_{i}) ∩ K is normal in K and K/Ã_{i} is isomorphic to K_{i}/A_{i}. From (24) we then conclude that

[K : Ã_{i}] ⩽ J_{K_{i}}.  (25)

Since A := Ã_{1} ∩ Ã_{2} is the kernel of the diagonal homomorphism

K ⟶ ∏_{i=1}^{2} K/Ã_{i}

determined by the canonical projections K → K/Ã_{i}, we infer from (25) that

[K : A] = |K/A| ⩽ | ∏_{i=1}^{2} K/Ã_{i}| ⩽ J_{K_{1}}J_{K_{2}}  (26)

By construction, A ⊆ A_{1} × A_{2}. Since A_{i} is abelian, this implies that A is abelian. As A is normal in K, this and (26) complete the proof. □

The following definition distinguishes a special class of Jordan groups.

Definition 2.7. A group G is called bounded if there is a positive integer b_{G}, depending only on G, such that the order of every finite subgroup of G is at most b_{G}.

Examples 2.8. (1) Finite groups and torsion free groups are bounded.

(2) Every finite subgroup of GL_{n}(Q) is conjugate to a subgroup of GL_{n}(Z) (see, e.g., [CR, Theorem 73.5]). On the other hand, by Minkowski’s theorem (see, e.g., [Hu, Theorem 39.4]) GL_{n}(Z) is bounded. Hence GL_{n}(Q) is bounded. Note that H. Minkowski and I. Schur obtained explicit upper bounds of the orders of finite subgroups in GL_{n}(Z), see [Hu, §39].

(3) It is immediate from the definition that every extension of a bounded group by bounded is bounded as well.

Lemma 2.9. Let H be a normal subgroup of a group G such that G/H is bounded. Then G is Jordan if and only if H is Jordan.

Proof. A proof is needed only for the sufficiency. So assume that H is Jordan; we have to prove that G is Jordan. Let K be a finite subgroup of G. By Definition 2.1

L := K ∩ H  (27)

contains an abelian normal subgroup A such that

[L : A] ⩽ J_{H}.  (28)

Let g be an element of K. Since L is a normal subgroup of K, we infer that gAg^{−1} is a normal abelian subgroup of L and

[L : A] = [L : gAg^{−1}].  (29)

Consider now the group

M := ⋂_{g∈K} gAg^{−1}.  (30)

It is a normal abelian subgroup of K. We claim that [K : M] is upper bounded by a constant not depending on K. To prove this, fix the representatives g_{1}, …, g_{|K/L|} of all cosets of L in K. Then (30) and normality of A in L imply that

M = ⋂_{i=1}^{|K/L|} g_{i}Ag_{i}^{−1}.  (31)

From (31) we deduce that M is the kernel of the diagonal homomorphism

L ⟶ ∏_{i=1}^{|K/L|} L/g_{i}Ag_{i}^{−1}

determined by the canonical projections L → L/g_{i}Ag_{i}^{−1}. This, (29), and (28) yield

[L : M] ⩽ [L : A]^{|K/L|} ⩽ J_{H}^{|K/L|}.  (32)

Let π: G → G/H be the canonical projection. By (27) the finite subgroup π(K) of G/H is isomorphic to K/L. Since G/H is bounded, this yields |K/L| ⩽ b_{G/H}. We then deduce from (32) and [K : M] = [K : L][L : M] that

[K : M] ⩽ b_{G/H}J_{H}^{b_{G/H}} .

This completes the proof. □

Corollary 2.10. Let H be a finite normal subgroup of a group G such that the center of H is trivial. If G/H is Jordan, then G is Jordan.

Proof. The conjugating action of G on H determines a homomorphism φ: G → Aut(H). The definition of φ and triviality of the center of H implies that

H ∩ ker φ = {1}.  (33)

In turn, (33) yields that the restriction of the natural projection G → G/H to ker φ is an embedding ker φ ↪ G/H. Hence ker φ is Jordan since G/H is Jordan. But G/ker φ is finite since it is isomorphic to a subgroup of Aut(H) for the finite group H. Whence G is Jordan by Lemma 2.9. This completes the proof. □

Example 2.12, Theorems 2.13, 2.16 and their corollaries below give, for some varieties X, the affirmative answer to the following

Question 2.11. Let X be an irreducible affine variety. Is it true that Aut(X) is Jordan?

Example 2.12. Aut(A^{n}) is Jordan for n ⩽ 2. For n = 1 this clear, for n = 2 follows from Theorem 2.2 and the well-known fact that every finite subgroup of Aut(A^{2}) is linearizable, i.e., conjugate to a subgroup of GL_{2}(k) (see also Subsection 2.2 below).

Theorem 2.13. The automorphism group of every irreducible toral variety (see Definition 1.13) is Jordan.

Proof. By [R], for any irreducible variety X, the abelian group

Γ := k[X]^{∗}/k^{∗}

is free and of finite rank. Let X be toral and let H be the kernel of the natural action of Aut(X) on Γ. We claim that H is abelian. Indeed, for every element f ∈ k[X]^{∗}, the line spanned by f in k[X] is H-stable. Since GL_{1} is abelian, this yields that

h_{1}h_{2} · f = h_{2}h_{1} · f  for any elements h_{1}, h_{2} ∈ H.  (34)

As X is toral, k[X]^{∗} generates the k-algebra k[X] by Lemma 1.14. Hence (34) holds for every f ∈ k[X]. Since X is affine, the automorphisms of X coincide if and only if they induce the same automorphisms of k[X]. Whence H is abelian, as claimed.

Let n be the rank of Γ. Then Aut(Γ) is isomorphic to GL_{n}(Z). By the definition of H, the natural action of Aut(X) on Γ induces an embedding of Aut(X)/H in Aut(Γ). Hence Aut(X)/H is isomorphic to a subgroup of GL_{n}(Z). Example 2.8(2) then implies that Aut(X)/H is bounded. Thus, Aut(X) is an extension of a bounded group by an abelian group, hence Jordan by Lemma 2.9. This completes the proof. □

Remark 2.14. Maintain the notation of the proof of Theorem 2.13. Let f_{1}, … f_{n} be a basis of Γ. There are the homomorphisms λ_{i}: H → k^{∗}, i = 1, …, n, such that g · f_{i} = λ(g)f_{i} for every g ∈ H and i. Since k[X]^{∗} generates k[X], the diagonal map H → (k^{∗})^{n}, h ↦ (λ_{1}(g), …, λ_{n}(g)), is injective. This and the proof of Theorem 2.13 show that the automorphism group of X is an extension of a subgroup of GL_{n}(Z) by a subgroup of the torus (k^{∗})^{n}.

The following lemma is well-known (see, e.g., [FZ, Lemma 2.7(b)]).

Lemma 2.15. Let X be a variety and let G be a reductive algebraic subgroup of Aut(X). Let x ∈ X be a fixed point of G. Then the kernel of the induced action of G on T_{x,X} is trivial.

Theorem 2.16. Let ∼ be the equivalence relation on the set of points of a variety X defined by

x ∼ y ⟺ the local rings of X at x and y are k-isomorphic.

If there is a finite equivalence class of ∼, then Aut(X) is Jordan.

Proof. Every equivalence class of ∼ is Aut(X)-stable. Let C be a finite equivalence class of ∼ and let G be the kernel of the action of Aut(X) on C. Then G is a normal subgroup of finite index in Aut(X). By Lemma 2.9 it suffices to prove that G is Jordan.

Let K be a finite subgroup of G and let x be a point of C. As x is fixed by K, the action of K on X induces an action of K on T_{x,X}. The latter is linear and hence determined by a homomorphism τ: K → GL(T_{x,X}). Being finite, K is reductive. Hence τ is injective by Lemma 2.15. Theorem 2.2 then yields that K contains an abelian normal subgroup A such that [K : A] ⩽ J_{GL_{n}(k)}, n := dim T_{x,X}. This completes the proof. □

Given a variety X, we say that its point x is a vertex of X if

dim T_{x,X} ⩾ dim T_{y,X} for every point y ∈ X.

Clearly, an irreducible X is smooth if and only if every its point is a vertex.

Corollary 2.17. The automorphism group of every variety with only finitely many vertices is Jordan.

Corollary 2.18. Let ≈ be the equivalence relation on the set of points of a variety X defined by

x ≈ y ⟺ the tangent cones of X at x and y are isomorphic.

If there is a finite equivalence class of ≈, then Aut(X) is Jordan.

Corollary 2.19. The automorphism group of every nonsmooth variety with only finitely many singular points is Jordan.

Corollary 2.20. Let X̂ ⊂ k^{n+1} be the affine cone of a smooth closed proper subvariety X in P^{n} = P(k^{n+1}) that does not lie in any hyperplane. Then Aut(X̂) is Jordan.

Proof. The assumptions imply that the singular locus of X̂ consists of a single point, the origin; whence the claim by Corollary 2.19. □

Remark 2.21. Smoothness in Corollary 2.20 may be replaced by the assumption that X is not a cone. Indeed, in this case the origin constitutes a single equivalence class of ≈ for points of X̂; whence the claim by Corollary 2.18.

Theorem 2.22. For every variety X, every finite subgroup G of Aut(X) such that X^{G} ≠ ∅ contains an abelian normal subgroup whose index in G is at most J_{GL_{d}(k)} where d = max_{x} dim T_{x,X}.

Proof. Like in the above proof of Theorem 2.16, this follows from Lemma 2.15 and Theorem 2.2. □

Corollary 2.23. Let p be a prime number. Then every finite p-subgroup G of Aut(A^{n}) contains an abelian normal subgroup whose index in G is at most J_{GL_{n}(k)}.

Proof. This follows from Theorem 2.22 since in this case (A^{n})^{G} ≠ ∅, see [Se_{3}, Theorem 1.2]. □

Remark 2.24. To date, it is not known whether or not (A^{n})^{G} ≠ ∅ for every finite subgroup G of Aut(A^{n}). By Theorem 2.22 the affirmative answer would imply that Aut(A^{n}) is Jordan.

Remark 2.25. The statement of Corollary 2.23 remains true if A^{n} is replaced by any p-acyclic variety X and n in J_{GL_{n}(k)} by max_{x} dim T_{x,X}. This is because in this case X^{G} ≠ ∅ for every finite p-subgroup G of Aut(X), see [Se_{3}, Sect. 7–8].

Theorem 2.26. For every variety X, there is an integer m_{X} such that any finite subgroup G of any connected linear algebraic subgroup L of Aut(X) contains an abelian normal subgroup whose index in G is at most m_{X}.

Proof. Being reductive, G is contained in a maximal reductive subgroup R of L. Then R is a Levi subgroup, i.e., L is a semidirect product of R and Rad_{u}L, cf., e.g., [OV, Chap. 6]. As L is connected, R is connected as well. Since the kernel of the action of R on X is trivial, rk R ⩽ dim X, see [P_{2}, §3]. The claim then follows from Theorem 2.4 as there are only finitely many connected reductive groups of rank at most dim X. □

2.2. Generalizations. One may ask whether “affine” in Question 2.11 can be dropped:

Question 2.27. Is there an irreducible variety X such that Aut(X) is not Jordan?

The negative answer to Question 2.27 would follow from that to

Question 2.28. Is there an irreducible variety X such that Bir(X) is not Jordan?

If X is a curve, then the answer to Question 2.28 is negative.

Indeed, we may assume that X is smooth and projective. Then Bir(X) = Aut(X). If g(X), the genus of X, is 0, then X = P^{1}, hence Bir(X) = PGL_{2}(k), so Bir(X)

is Jordan by Theorem 2.4.

If g(X) = 1, then X is an elliptic curve; whence Bir(X) is the extension of a finite group by the abelian algebraic group X, hence Jordan by Lemma 2.9.

If g(X) ⩾ 2, then Bir(X) is finite, hence Jordan.

Note that all curves (not necessarily smooth and projective) with infinite automorphism group are classified in [P_{3}].

Answering Question 2.28 for surfaces X, one may assume that X is a smooth projective minimal model.

If X is of general type, then by Matsumura’s theorem Bir(X) is finite, hence Jordan. If X is rational, then Bir(X) is the Cremona group of rank 2 over k, hence Jordan

by [Se_{1}, Theorem 5.3], [Se_{2}, Théorème 3.1].

If X is a nonrational ruled surface, it is birationally isomorphic to P^{1} × B where B is a smooth projective curve such that g(B) > 0; we may then take X = P^{1} × B. As g(B) > 0, there are no dominant rational maps P^{1} → B, hence the elements of Bir(X) permute the fibers of the natural projection P^{1} ×B → B. The set of elements inducing trivial permutation is a normal subgroup Bir_{B}(X) of Bir(X). The definition implies that Bir_{B}(X) = PGL_{2}(k(B)), hence Jordan by Theorem 2.4. Naturally identifying Aut(B) with the subgroup of Bir(X), we get the decomposition Bir(X) = Bir_{B}(X) ⋊ Aut(B). Note that Aut(X) = PGL_{2}(k) × Aut(B) ≠ Bir(X) (see [M, pp. 98–99]), so Aut(X) is Jordan by Lemma 2.6. Let g(B) ⩾ 2. Then Aut(B) is finite, hence [Bir(X) : Bir_{B}(X)] < ∞. Lemma 2.9 then implies that Bir(X) is Jordan. For g(B) = 1, this argument does not work as B is an elliptic curve, so Aut(B) is infinite.

The canonical class of all other surfaces X is numerically effective, so, for them, Bir(X) = Aut(X) (cf. [IS, Sect. 7.3, Theorem 2].

If X is an abelian surface, then Bir(X) is an extension of a subgroup of GL_{4}(Z) by the abelian algebraic group X, hence Jordan by Lemma 2.9 and Example 2.8(2).

If X is a bielliptic surface, then X is the quotient of an abelian surface X̃ by a finite automorphism group F and Aut(X) is isomorphic to Aut(X̃)/F. Since Aut(X̃) is Jordan, Lemma 2.5 then yields that Aut(X) is Jordan.

In the other cases let K be the kernel of the natural action of Bir(X) on H^{2}(X, Q) (we may assume that k = C) and let D be the image of Bir(X) in GL(H^{2}(X, Q)) defined by this action. Since Bir(X)/K is isomorphic to D and D is bounded (see Example 2.8(2)), Lemma 2.9 implies that Bir(X) is Jordan if K is Jordan (and Bir(X) is bounded if K is bounded, see Example 2.8(3)). In many cases this yields that Bir(X) is Jordan. Namely:

• If X is a K3-surface, then K is trivial (cf. [IS, Sect. 12.4]); whence Bir(X) is bounded.

• If X is an Enriques surface, then K is finite by [MM]; whence Bir(X) is bounded.

• For elliptic surfaces X of Kodaira dimension one, there is an extensive information on the cases when K is finite (hence Bir(X) is bounded), see [C] and references therein. For instance, by [L] if H^{0}(X, T_{X}) = 0 (i.e., X admits no nonzero regular vector fields), then K is finite.

So to answer Question 2.28 for surfaces it only remains to consider the following two types of elliptic surfaces that are not covered by the above considerations:

(a) X = P^{1} × B where B is an elliptic curve;

(b) elliptic surfaces X of Kodaira dimension one for which the kernel of the natural action of Bir(X) on H^{2}(X, Q) is infinite (note that every such X admits a nonzero regular vector field).

It looks plausible that Bir(X) is Jordan for both types (a) and (b). If this is indeed so, this would complete the proof of the following

Conjecture 2.29. Bir(X) is Jordan for every surface X.

References

[Bo] A. Borel, Linear Algebraic Groups, 2nd ed., Graduate Text in Mathematics, Vol. 126, Springer-Verlag, New York, 1991.

[Br] M. Brion, Represéntations exceptionnelles des groupes semi-simples, Ann. Sci. Éc. Norm. Sup. 4e sér. 18 (1985), 345–387.

[BL] S. Billey, V. Lakshmibai, Singular Loci of Schubert Varieties, Progress in Math., Vol. 182, Birkhäuser, Boston, 2000.

[C] J.-X. Cai, Automorphism groups of elliptic surfaces, inducing the identity in cohomology, J. Algebra, 322 (2009), 4228–4246.

[CM] A. Crachiola, S. Maubach, The Derksen invariant vs. the Makar-Limanov invariant, Proc. Amer. Math. Soc. 131 (2003), no. 11, 3365–3369.

[CR] C. W. Curtis, I. Reiner, Representation Theory of Finite Groups and Associative Algebras, Wiley, New York, 1962.

[Dai] D. Daigle, Affine surfaces with trivial Makar-Limanov invariant, J. Algebra 319 (2008), no. 8, 3100–3111.

[Der] H. Derksen, Constructive Invariant Theory and the Linearization Problem, Ph.D. thesis, Univ. Basel, 1997.

[Dub] A. Dubouloz, Completions of normal affine surfaces with a trivial Makar-Limanov invariant, Michigan Math. J. 52 (2004), no. 2, 289–308.

[DM-JP] A. Dubouloz, L. Moser-Jausin, P.-M. Poloni, Inequivalent embeddings of the Koras–

Russell cubic threefold, arXiv:0903.4278 (2009).

[F] G. Freudenburg, Algebraic Theory of Locally Nilpotent Derivations, Springer, Berlin, 2006.

[FZ] H. Flenner, M. Zaidenberg, Locally nilpotent derivations on affine surfaces with a C^{∗}- action, Osaka J. Math. 42 (2005), 931–974.

[G] M. H. Gizatullin, Affine surfaces which are quasihomogeneous with respect to an algebraic group, Math. USSR Izv. 5 (1971), 754–769.

[Ha] J. Harris, Algebraic Geometry, Graduate Texts in Mathematics, Vol. 133, Springer, New York, 1992.

[Hu] B. Huppert, Character Theory of Finite Groups, De Gruyter Expositions in Mathematics, Vol. 25, Walter de Gruyter, Berlin, 1998.

[IS] V. A. Iskovskikh, I. R. Shafarevich, Algebraic surfaces, in: Algebraic Geometry, II, Encyclopaedia Math. Sci., Vol. 35, Springer, Berlin, 1996, pp. 127–262.

[K] B. Kostant, Lie group representations on polynomial rings, Amer. J. Math. 85 (1963), 327–404.

[L] D. Lieberman, Compactness of the Chow scheme: applications to automorphisms and deformations of Kähler manifolds, Sem. Norguet 1976.

[M] M. Maruyama, On automorphisms of ruled surfaces, J . Math. Kyoto Univ. 11-1 (1971), 89–112.

[MM] S. Mukai, Y. Namikawa, Automorphisms of Enriques surfaces which act trivially on the cohomology groups, Invent. Math. 77 (1984), 383-397.

[OV] A. L. Onishchik, E. B. Vinberg, Lie Groups and Algebraic Groups, Springer Series in Soviet Mathematics, Springer-Verlag, Berlin, 1990.

[P_{1}] V. L. Popov, Classification of affine algebraic surfaces that are quasihomogeneous with re-

spect to an algebraic group, Math. USSR Izv. 7 (1973), no. 5, 1039–1055.

[P_{2}] V. L. Popov, Classification of three-dimensional affine varieties that are quasi-

homogeneous with respect to an algebraic group, Math. USSR Izv. 9 (1975), no. 3, 535–576.

[P_{3}] V. L. Popov, Algebraic curves with an infinite automorphism group, Math. Notes 23 (1978),

102–108.

[P_{4}] V. L. Popov, Algebraic cones, Math. Notes 86 (2009), no. 6, 892–894. [PV_{1}] V. L. Popov, E. B. Vinberg, On a class of quasihomogeneous affine varieties, Math.

USSR, Izv. 6 (1973), 743–758.

[PV_{2}] V. L. Popov, E. B. Vinberg, Invariant Theory, in: Algebraic Geometry IV, Encyclopaedia

of Mathematical Sciences, Vol. 55, Springer-Verlag, Berlin, 1994, pp. 123–284.

[Pr] C. Procesi, Lie Groups. An Approach Through Invariants and Representations, Springer, New York, 2007.

[R] M. Rosenlicht, Some rationality questions on algebraic groups, Ann. Mat. Pura Appl. (4) 43 (1957), 25–50.

[Se_{1}] J-P. Serre, A Minkowski-style bound for the orders of the finite subgroups of the Cremona

group of rank 2 over an arbitrary field, Moscow Math. J. 9 (2009), no. 1, 183–198.

[Se_{2}] J-P. Serre, Le groupe de Cremona et ses sous-groupes finis, Séminaire Bourbaki, no. 1000,

Novembre 2008, 24 pp.

[Se_{3}] J-P. Serre, How to use finite fields for problems concerning infinite fields, Contemporary

Math. 487 (2009), 183–193.

[Sp] T. A. Springer, Linear Algebraic Groups. 2nd ed., Progress in Mathematics, Vol. 9, Birkhäuser, Boston, 1998.

[SW] J. N. Sampson, G. Washnitzer, A Künneth formula for coherent algebraic sheaves, Illinois J. Math. 3 (1959), 389–402.

Steklov Mathematical Institute, Russian Academy of Sciences, Gubkina 8, Moscow, 119991, Russia

   E-mail address: popovvl@orc.ru