TWO ORBITS: WHEN IS ONE IN THE CLOSURE OF THE OTHER?

VLADIMIR L. POPOV

To V. A. Iskovskikh on the occasion of his 70th birthday

Abstract. Let G be a connected linear algebraic group, let V be a finite dimensional algebraic G-module, and let O_{1}, O_{2} be two G-orbits in V. We describe a constructive way to find out whether or not O_{1} lies in the closure of O_{2}.

1. Introduction

1.1. Fix an algebraically closed ground field k of arbitrary characteristic.

Let G be a connected linear algebraic group and let V be a finite dimensional algebraic G-module. Consider two points a and b ∈ V and their G-orbits G · a and G · b.

The following problem continually arises in algebraic transformation group theory and its applications:

How can one find out whether or not the orbit G · a lies in the closure in V of the orbit G · b?

(∗)

(here and further topological terms are related to the Zarisky topology).

Example 1.2. If the group G is reductive and a = 0, then Problem (∗) means finding out whether or not the point b is unstable in the sense of Geometric Invariant Theory [M65]. A description of the cone of unstable points is provided by the Hilbert–Mumford theory.

Example 1.3. Let G be a torus and let X(G) be the group of its characters in additive notation. For every λ ∈ X(G), g ∈ G, and v ∈ V denote by g^{λ} and v_{λ} respectively the value of λ at g and the projection of v to the λwight subspace of the G-module V parallel to the sum of the other weight subspaces. Let supp v := {λ ∈ X(G) | v_{λ} ≠ 0}. Then by [PV72] Problem (∗) means finding out whether or not the following conditions hold: (i) the cone generated by supp a in X(G) ⊗_{ℤ} ℝ is a face of the cone generated by supp b, and (ii) there is an element g ∈ G such that g^{λ}a_{λ} = b_{λ} for every λ ∈ supp a.

Example 1.4. If the group G is unipotent, then every G-orbit is closed in V, see [R61]. Therefore Problem (∗) means finding out whether or not the points a and b lie in one and the same G-orbit.

Supported by Russian grants ���� �����������, ��������������, and a granting program Contemporary Problems of Theoretical Mathematics of the Mathematics Branch of the Russian Academy of Sciences.

Example 1.5. Let char k = 0. Assume that G is a simple group, V is its Lie algebra endowed with the adjoint action of G, and the elements a and b are nilpotent. If G is a classical group (i.e., of type A_{l}, B_{l}, C_{l}, or D_{l}), then the answer to (∗) is given by the known rule formulated in terms of the sizes of Jordan blocks of the Jordan normal forms of a and b, see, e.g., [CM93]. If the group G is exceptional (i.e., of type E_{6}, E_{7}, E_{8}, F_{4}, or G_{2}), then this answer, obtained by means of ad hoc methods, is given by the explicit Hasse diagrams of the set of nilpotent orbits endowed with the Bruhat order, i.e., partially ordered according to the rule

O_{1} ⩽ O_{2} ⟺ O_{1} ⊆ \overline{O_{2}}

(as usual, bar means the closure in V), see [Spa82], [C85].

Example 1.6. Apart from the classical case of orbits of a Borel subgroup of a reductive group G on the generalized flag variety G/P, the Hasse diagrams of the sets of orbits endowed with the Bruhat order are found utilizing the ad hoc methods in some other special cases, see, e.g., [Kas90], [Pe04], [BHRZ99], [GHR07], [MWZ99]. On the other hand, in a number of cases the orbits are classified, but the Hasse diagrams are not found: for instance, this is so for nilpotent 3-vectors of the n-dimensional spaces where n ⩽ 9, for 4-vectors of an 8-dimensional space, for spinors of the m-dimensional spaces where m ⩽ 14 and 16, see the relevant references in [PV94].

Example 1.7. Let L be a finite dimensional vector space over k. Let G = GL(L) and V = L^{∗} ⊗L^{∗} ⊗L. Points of V are the structures of (not necessarily associative) algebras on the vector space L. The algebras defined by the structures a and b are isomorphic if and only if G · a = G · b. In the language of the theory of algebras Problem (∗) is formulated as follows: How can one find out whether or not the algebra defined by the structure a is a degeneration of the algebra defined by the structure b? In general case it is considered to be a difficult problem. There is a number of papers where a classification of degenerations in various special cases is obtained by means of ad hoc methods, see, e.g., [B05], [BS99], [See90], and survey [OVG94, Chap. 7].

Example 1.8. Consider the conjugation action of the group G = GL_{d}(k) on Mat_{d,d}(k). Let A be a finite dimensional associative k-algebra. The set of structures of left A-modules on a fixed d-dimensional vector space over k is naturally identified with a closed invariant subset Mod_{A}^{d} of the direct sum of dim_{k}A copies of the G-module Mat_{d,d}(k). Denote by M_{x} the A-module corresponding to a point x ∈ Mod_{A}^{d}. Then the A-modules M_{a} and M_{b} are isomorphic if and only if G · a = G · b, and in this theory the condition G · a ⊆ \overline{G · b} is expressed by saying that M_{a} is a degeneration of M_{b}. If A is the path algebra of a quiver obtained by fixing an orientation of the extended Dynkin graph of a root system of type A_{l}, D_{l}, E_{6}, E_{7}, or E_{8}, then in [B95] a characterization of the degeneration relation in terms of A-module structures of M_{a} and M_{b} and an algorithm that finds out whether or not M_{a} is the degeneration of M_{b} are obtained.

Example 1.9. In case of the natural action of the group GL_{n}(k) on the space of n-ary forms of degree d with the coefficients in k Problem (∗) (under the name The orbit closure problem) is of fundamental importance in application of geometric invariant theory to complexity theory, see [MS01].

1.10. In this paper we give a constructive solution of Problem (∗): in Section 2 we suggest an algorithm that provides an answer to (∗) by means of a finite number of effectively feasible operations. Namely, we explicitly point out a finite system of linear equations in finitely many variables over the field k such that the inclusion G · a ⊆ \overline{G · b} is equivalent to its inconsistency (the precise formulation is contained in Theorem 2.12). By Kronecker–Capelli theorem this reduces answering (∗) to comparing ranks of two explicitly given matrices with the coefficients in k that can be executed constructively. Of course, the existence of a constructive solution of Problem (∗) immediately leads to the problem of finding a most effective algorithm. But this is another problem that we do not consider here.

1.11. In Section 3 we suggest another algorithm for finding an answer to Problem (∗). It is less effective than the algorithm from Section 2, but it provides more information and concerns a more general problem. To wit, let L be a linear subvariety of V. We show how one can constructively find a finite system of polynomial functions q_{1}, …, q_{m} on V such that

\overline{G · L} = \{x ∈ V | q_{1}(x) = … = q_{m}(x) = 0\}.  (1)

For L = b this provides the following constructive answer to Problem (∗):

G · a ⊆ \overline{G · b} ⟺ q_{1}(a) = … = q_{m}(a) = 0.

Note that varieties of the form \overline{G · L} are ubiquitous in algebraic transformation group theory: apart from orbit closures, to them also belong irreducible components of Hilbert null-cones and, more generally, closures of Hesselink strata [Po03], closures of sheets [PV94], and closures of Jordan (a.k.a. decomposition) classes [TY05]. Also note that if a system of polynomials q_{1}, … …, q_{m} satisfying (1) is given, modern commutative algebra provides algorithms to constructively find a system of generators of the ideal of all polynomials vanishing on \overline{G · L}, see, e.g., [CLO98, Chap. 4, §2]. In particular, this provides methods to constructively find generators of the ideal of polynomials vanishing on the closure of orbit. In some special cases (for instance, for nilpotent orbits of the adjoint action of the group SL_{n}(k) and for “rank varieties”) such generators have been found, see [W89].

1.12. In essence, both algorithms are based on the possibility to rationally parametrize an open subset of G by means of a variety of the form

𝔸^{r,s} := {(ε_{1}, …, ε_{r+s}) ∈ 𝔸^{r+s} | ε_{1} ⋯ ε_{r} ≠ 0}, r, s ∈ ℕ

(we denote by ℕ the set of all nonnegative integers), more precisely, on the existence of a dominant morphism

ι: 𝔸^{r,s} → G.  (2)

1.13. As every normal quasiprojective variety endowed with an algebraic action of G can be equivariantly embedded in a projective space [PV94], in algebraic transformation group theory and its applications continually arises the problem analogous to (∗), but for an action of G on a projective space (in fact, this is so in Example 1.9). However, this problem is reduced to Problem (∗) for actions on vector spaces, see Subsection 2.7.

1.14. We close this introduction by noting that as

G · a = G · b ⟺ G · a ⊆ \overline{G · b} and G · b ⊆ \overline{G · a},

a constructive solution of Problem (∗) provides a constructive solution of the following problem:

How can one find out whether or not two given points of V lie in one and the same G-orbit?

2. Main result

2.1. Our further considerations are based on the following fact.

Lemma 2.2. For some r, s ∈ ℕ, there is a dominant morphism (2). Moreover, for r = rk G, there is an open embedding (2).

Proof. Let R_{u}(G) be the unipotent radical of the group G. By [R56], as R_{u}(G) is a connected solvable group, the canonical projection G → G/R_{u}(G) is a torsor with the base G/R_{u}(G) and the structural group R_{u}(G) that is locally trivial in the Zariski topology. By [G58], as the base is affine and the structural group is connected and unipotent, this torsor is trivial. Hence the variety G is isomorphic to the product of varieties R_{u}(G) and G/R_{u}(G). As R_{u}(G) is a connected unipotent group, the first of them is isomorphic to 𝔸^{dim R_{u}(G)}, see [G58]. On the other hand, the big Bruhat cell of the reductive group G/R_{u}(G) is isomophic to 𝔸^{r,dim G/R_{u}(G)−r}, where r = rk G/R_{u}(G) = rk G, see [Spr98]. As the variety 𝔸^{r,dim G−r} is isomorphic to 𝔸^{dim R_{u}(G)} × 𝔸^{r,dim G/R_{u}(G)−r}, this shows that there is its open embedding in G. □

2.3. Notation

Fix a basis e_{1}, …, e_{n} in V. As the case n = 1 is clear, in the further we assume that n > 1. There are functions ρ_{i,j}, 1 ⩽ i, j ⩽ n, regular on G such that the action of G on V is given by the matrix representation

ρ: G → Mat_{n,n}(k), ρ(g) = [\begin{matrix}ρ_{1,1}(g) & & ⋯ & & ρ_{1,n}(g) \\ . & . & . & . & . & . & . & . & . & . & . & . & . & . & . \\ ρ_{n,1}(g) & & ⋯ & & ρ_{n,n}(g) \end{matrix}] , g ∈ G,  (3)

i.e., ρ(g) is the matrix of the linear V → V, v ↦ g · v in the basis e_{1}, …, e_{n}, so that

g · (∑_{i=1}^{n} γ_{i}e_{i}) = ∑_{i=1}^{n}(∑_{j=1}^{n} ρ_{i,j}(g)γ_{j})e_{i} g ∈ G γ_{1}, …, γ_{n} ∈ k.  (4)

Fix a dominant morphism (2): this is possible by Lemma 2.2. Denote by x_{1}, …, x_{r+s} the standard coordinate functions on 𝔸^{r,s}:

x_{i}(a) = ε_{i} for a = (ε_{1}, …, ε_{r+s}) ∈ 𝔸^{r,s}.  (5)

As x_{1}, …, x_{r+s}, x_{1}^{−1}, …, x_{r}^{−1} generate the k-algebra k[𝔸^{r,s}] of regular functions on 𝔸^{r,s} and x_{1}, …, x_{r+s} are algebraically independent over k, all the monomials of the form

x_{1}^{i_{1}} ⋯ x_{r+s}^{i_{r+s}}, i_{1}, …, i_{r+s} ∈ ℤ i_{r+1}, …, i_{r+s} ∈ ℕ,  (6)

constitute a basis of the vector space k[𝔸^{r,s}] over k.

2.4. The degree of the variety ρ(G)

Recall [M76] that the degree of a locally closed subset Y of 𝔸^{l} is the cardinality deg Y of the intersection of Y with an (l − dim Y)-dimensional linear subvariety of 𝔸^{l} in general position. For us, the degree deg ρ(G) of the subvariety ρ(G) of the space of matrices Mat_{n,n}(k) is of the special interest. In the important case where char k = 0 and G is a reductive group there is the following formula for computation of this number.

Fix a maximal torus T in G. Let X(T) be its character group in additive notation. The latter is a free abelian group of rank r = dim T = rk G naturally embedded in the real vector space E = X(T) ⊗_{ℤ} ℝ. Fixing a basis in X(T), we fix an isomorphism between E and the coordinate space ℝ^{r}. Identify these spaces by means of this isomorphism. Then the group X(T) is identified with the lattice ℤ^{r} in ℝ^{r}. The Weyl group W := N_{G}(T)/T naturally acts on E. We denote by dν the standard volume form on E. Let P_{V} be the convex hull in E of the union of zero and the system of weights of the T -module V. Replacing G by the quotient group of G by the unity connected component of the kernel of the G-action on V, we may (and shall) assume that this kernel is finite and hence dim P_{V} = r. Fix a system R_{+} of positive roots of the root system of G with respect to T. For every root α ∈ R_{+}, denote by α^{∨} the corresponding coroot, i.e., the linear form on E defined by the formula α^{∨}: E → ℝ, α^{∨} (v) = 2⟨α | v⟩/⟨α | α⟩, where ⟨ | ⟩ is a W -invariant inner product on E, see [B68]. Let m_{1} + 1, …, m_{r} + 1 be the set of degrees of homogeneous free generators of the algebra of W -invariant polynomial functions on the space E, i.e., m_{1}, …, m_{r} are the exponents of W, see [B68].

Theorem 2.5 (Kazarnovskiǐ [Kaz87]). Let char k = 0 and let ρ be a representation of a connected reductive group G with finite kernel. Then

deg ρ(G) := \frac{dim G!}{|W |(m_{1}! ⋯ m_{r}!)^{2}|ker ρ|} ∫_{P_{V}} ∏_{α∈R_{+}}(α^{∨})^{2}dν.  (7)

Example 2.6. Let char k = 0. Consider the main object of pre-Hilbertian classical invariant theory: G = SL_{2}(k) and V = V_{s} is the space of binary forms of degree s in variables z_{1}, z_{2} over the field k on which G acts by linear substitutions of variables:

g · z_{1} = αz_{1} + γz_{2}, g · z_{2} = βz_{1} + δz_{2}, if g = [\begin{matrix}α & β \\ γ & δ \end{matrix}] ∈ G.  (8)

In this case dim G = 3, |W | = 2, r = 1, m_{1} = 1, E = ℝ, X(T) = ℤ. The set R_{+} consists of a single root α = 2 and α^{∨} is the standard coordinate function on ℝ, i.e., α^{∨}(a) = a for every a ∈ ℝ. Take the sequence of monomials z_{1}^{s}, z_{1}^{s−1}z_{2}, …, z_{1}z_{2}^{s−1}, z_{2}^{s} as a basis e_{1}, …, e_{s+1} of V. It follows from (7) that e_{i+1} is a weight vector of the diagonal torus T = {diag(t, t^{−1}) | t ∈ k \ {0}} with weight t ↦ t^{s−2i}. Therefore the weight system of the T -module V is the arithmetic progression {s, s − 2, …, −s + 2, −s}. Hence P_{V_{s}} = [−s, s]. The kernel of the representation ρ = ρ_{s} given by formula (3) is trivial if s odd and has order 2 if s is even. Therefore we deduce from (7) that

deg ρ_{s}(SL_{2}) = \frac{3!}{2| ker ρ_{s}|} ∫_{−s}^{s} x^{2}dx = \begin{cases}2s^{3} & if s is odd, \\ s^{3} & if s is even. \end{cases}  (9)

2.7. The reduction to conic case

Let L be a finite dimensional vector space over k. Let H be an algebraic group (algebraically) acting on a projective space ℙ(L) of one-dimensional linear subspaces of L. Keeping H-orbits in ℙ(L), we may replace the group H by its quotient group by the kernel of action and assume that H is a subgroup of Aut(ℙ(L)). Let H̃ be the inverse image of H with respect to the natural homomorphism GL(L) → Aut(ℙ(L)) (note that H is reductive if and only if H̃ shares this property). Let π: L\{0} → ℙ(L) be the natural projection. We call a subset in L conic if it is stable with respect to scalar multiplication by every nonzero element k.

Lemma 2.8. Let U be a nonempty open H-stable subset of ℙ(L) and let p, q ∈ U be two its points. Take any points p̃ ∈ π^{−1}(p) and q̃ ∈ π^{−1}(q). Then the following properties are equivalent:

(i) the orbit H · p lies in the closure of the orbit H · q in ℙ(L); (ii) the orbit H · p lies in the closure of the orbit H · q in U; (iii) the orbit H̃ · p̃ lies in the closure of the orbit H̃ · q̃ in L.

The orbits H̃ · p̃ and H̃ · q̃ are conic.

Proof. As H̃ contains all scalar multiplications of the space L by nonzero scalars, H̃ · p̃ = π^{−1}(H · p) and H̃ · q̃ = π^{−1}(H · q). The reader will easily check that the statement follows from this and the definitions. □

We shall need the following application of Lemma 2.8. Let L be the coordinate space k^{n+1} and let H be the group G from Subsection 1.1 that acts on ℙ(L) according to the rule (see (3))

g · (α_{0} : α_{1} : …: α_{n}) := (α_{0} : ∑_{i=1}^{n} ρ_{1,i}(g)α_{i} : …: ∑_{i=1}^{n} ρ_{n,i}(g)α_{i}).

The standard principal open subset {(α_{0} : α_{1} : …: α_{n}) | α_{0} ≠ 0} of ℙ(L̃) is G-stable and is equivariantly isomorphic to the G-module V. Hence, by Lemma 2.8, answering question (∗) is equivalent to answering whether or not the orbit G̃ · ã lies in the closure of the orbit G̃ · b̃. This means that replacing the group G by G̃, the space V by L, and the points a and b by, respectively, ã and b̃, we reduce solving Problem (∗) to the case where both orbits G · a and G · b are nonzero and conic. Given this,

searching for an answer to Problem (∗), we may assume that G · a and G · b are nonzero conic orbits.

(10)

Note also that by Lemma 2.8 the problem analogous to Problem (∗), but for an action on a projective space is reduced to Problem (∗) for an action on a linear space.

2.9. The input of the algorithm We assume that the following data are known:

— The degree of the variety ρ(G),

d := deg ρ(G).  (11)

— The functions

ι^{∗}(ρ_{p,q})∈k[𝔸^{r,s}]=k[x_{1}, …, x_{r+s}, x_{1}^{−1}, …, x_{r}^{−1}], 1 ⩽ p, q ⩽ n.

Example 2.10. Consider the same situation as in Example 2.6. Number (11) is given by formula (9). It follows from (8) that the functions ρ_{p,q} in (3) are defined by the equality

(αz_{1} + γz_{2})^{s−j}(βz_{1} + δz_{2})^{j} =∑_{i=0}^{s} ρ_{i+1,j+1}(g)z_{1}^{s−i}z_{2}^{i}, g =[\begin{matrix}α & β \\ γ & δ \end{matrix}]∈G. (12)

Take ι to be the morphism

ι: 𝔸^{1,2} ↪ SL_{2}(k),
(ε_{1}, ε_{2}, ε_{3}) ↦ [\begin{matrix}1 & ε_{2} \\ 0 & 1 \end{matrix}] [\begin{matrix}ε_{1} & 0 \\ 0 & ε_{1}^{−1} \end{matrix}] [\begin{matrix}1 & 0 \\ ε_{3} & 1 \end{matrix}] = [\begin{matrix}ε_{1}^{−1}ε_{2}ε_{3} + ε_{1} & ε_{1}^{−1}ε_{2} \\ ε_{1}^{−1}ε_{3} & ε_{1}^{−1} \end{matrix}] . (13)

Then it follows from (5), (12), (13) that the function ι^{∗}(ρ_{i+1,j+1}) is equal to the coefficient z_{1}^{s−i}z_{2}^{i} in the decomposition of the binary form

((x_{1} + x_{1}^{−1}x_{2}x_{3})z_{1} + (x_{1}^{−1}x_{3})z_{2})^{s−j}((x_{1}^{−1}x_{2})z_{1} + (x_{1}^{−1})z_{2})^{j}

in the variables z_{1}, z_{2} with the coefficients in the field k(x_{1}, x_{2}, x_{3}) as sum of monomials in z_{1}, z_{2}. For instance, if d = 2, then ι^{∗}(ρ_{2,2}) = 1 + 2x_{1}^{−2}x_{2}x_{3}.

2.11. The algorithm

Now we turn to the formulation and proof of the main result. We utilize the notation and conventions introduced above and exclude the trivial case \overline{G · b} = V, i.e., assume that

dim G · b < dim V  (14)

(as the number dim G · b is equal to the rank of the system of vectors {dρ(Y_{i}) · b}_{i∈I} where {Y_{i}}_{i∈I} is a basis of the vector space Lie(G), condition (14) can be verified constructively if the operators dρ(Y_{i}) are known).

The following sequence of steps together with Theorem 2.12 provide a constructive method to answer Problem (∗):

(1) Find the coordinates of the vectors a and b in the basis e_{1}, …, e_{n}:

a = α_{1}e_{1} + … + α_{n}e_{n}, b = β_{1}e_{1} + … + β_{n}e_{n},

and, changing the basis e_{1}, …, e_{n} if necessary, achieve that the following condition holds:

β_{1} ⋯ β_{n} ≠ 0.  (15)

(3) Consider n “generic” polynomials F_{1}, …, F_{n} of degree 2d − 2 (where d is defined by formula (11)) in the variables y_{1}, …, y_{n},

F_{p} := ∑_{\begin{matrix}q_{1},...,q_{n}∈ℕ \\ q_{1}+···+q_{n}⩽2d−2 \end{matrix}} c_{p,q_{1},...,q_{n}}y_{1}^{q_{1}} ⋯ y_{n}^{q_{n}}, p = 1, …, n,  (16)

i.e., such that, apart from y_{1}, …, y_{n}, all the coefficients c_{p,q_{1},...,q_{n}} are the indeterminates over k as well, and put

H(y_{1}, …, y_{n}) := (y_{1} − α_{1})F_{1} + … + (y_{n} − α_{n})F_{n} − 1.  (17)

(4) Replacing in H(y_{1}, …, y_{n}) every variable y_{i} by ∑_{j=1}^{n} β_{j}ι^{∗}(ρ_{i,j}), obtain a linear combination of monomials of form (6) with the coefficients in the ring k[…, c_{p,q_{1},...,q_{n}}, …] of polynomials in variables c_{p,q_{1},...,q_{n}} over the field k:

H(∑_{j=1}^{n} β_{j}ι^{∗}(ρ_{1,j}), …, ∑_{j=1}^{n}β_{j}ι^{∗}(ρ_{n,j}))
= ∑_{(i_{1},...,i_{r+s})∈M} ℓ_{i_{1},...,i_{r+s}}x_{1}^{i_{1}} ⋯ x_{r+s}^{i_{r+s}}, (18)

where ℓ_{i_{1},...,i_{r+s}} ∈ k[…, c_{p,q_{1},...,q_{n}}, …] and M is a finite subset in ℤ^{r} ×ℕ^{s}. By (16) and (17), every coefficient ℓ_{i_{1},...,i_{r+s}} in (18) is a linear function in the variables c_{p,q_{1},...,q_{n}} with the coefficients in the field k.

(5) Consider the following finite system of linear equations in the variables c_{p,q_{1},...,q_{n}} with the coefficients in the field k:

ℓ_{i_{1},...,i_{r+s}} = 0, where (i_{1}, …, i_{r+s}) ∈ M.  (19)

Theorem 2.12. Let G · b be a nonzero conic orbit (see (10)). The following properties are equivalent:

(i) the closure of the orbit G · b in V contains the orbit G · a; (ii) system of linear equations (19) is inconsistent.

Proof. We split it into several steps.

1. Let z_{1}, …, z_{n} be the basis of V^{∗} dual to e_{1}, …, e_{n}. Let t_{i} be the restriction to \overline{G · b} of the function z_{i}. As the set \overline{G · b} is closed in V and k[V] = k[z_{1}, …, z_{n}], we have

k[\overline{G · b}] = k[t_{1}, …, t_{n}].  (20)

Note that the function t_{i} is not a constant. Indeed, as the orbit G · b is conic, 0 ∈ \overline{G · b}. The definition of t_{i} implies that t_{i}(0) = 0 and t_{i}(b) = β_{i}. But β_{i} ≠ 0 because of (15).

Consider the orbit morphism

φ: G → \overline{G · b}, φ(g) = g · b.  (21)

As the morphism ι is dominant, the image of the morphism

ψ := φ ∘ ι: 𝔸^{r,s} → \overline{G · b}  (22)

is dense in \overline{G · b}; whence the corresponding comorphism is an embedding of the algebra of regular functions:

ψ^{∗}: k[\overline{G · b}] ↪ k[𝔸^{r,s}].  (23)

It follows from (20) that

ψ^{∗}(k[\overline{G · b}]) = k[ψ^{∗}(t_{1}), …, ψ^{∗}(t_{n})],  (24)

and (21), (22), (4), and the definition of t_{i} imply that

ψ^{∗}(t_{i}) = ∑_{j=1}^{n} β_{j}ι^{∗}(ρ_{i,j}).  (25)

There is only one point of V where all the functions z_{1} − α_{1}, …, z_{n} − α_{n} vanish, the point a. Taking into account that the set \overline{G · b} is G-stable, we deduce from this that the following properties are equivalent:

\begin{matrix}(c_{1}) & the orbit G · a does not lie in the closure of the orbit G · b; \\ (c_{2}) & the point a does not lie in the closure of the orbit G · b; \\ (c_{3}) & there are no points of \overline{G · b} where all the functions \\ & t_{1}−α_{1}, …, t_{n}−α_{n} vanish. \end{matrix} \} (26)

2. Now we shall use the effective form of Hilbert’s Nulltellensatz obtained in [J05]. In order to formulate this result we shall introduce some notation and definitions.

First, for every positive integers d_{1} ⩾ ⋯ ⩾ d_{m} and q, set

N(d_{1}, …, d_{m}; q) = \{ ; \begin{matrix}∏_{i=1}^{m} d_{i} & if q ⩾ m ⩾ 1, \\ (∏_{i=1}^{q−1} d_{i})d_{m} & if m > q > 1, \\ d_{m} & if q = 1, \end{matrix}

and also

N′(d_{1}, …, d_{m}; q) = N(d_{1}, …, d_{m}; q) if q > 1, and N′(d_{1}, …, d_{m}; 1) = d_{1}.

Further, if a nonzero regular function h on an irreducible closed subset X of an affine space 𝔸^{l} is given, the minimum of degrees of polynomial functions on 𝔸^{l} whose restriction to X is h will be called the degree of h and denoted by deg h. It is easily seen that, for every nonzero regular functions f and h on X, the inequality deg fh ⩽ deg f + deg h holds and, in general, it may be strict. However, if the set X is conic, then necessarily

deg fh = deg f + deg h.  (27)

Theorem 2.13 (Z. Jelonek [J05]). Let X be an irreducible closed subset of 𝔸^{l} of positive dimension. Let h_{1}, …, h_{m} be the nonconstant regular functions on X such that

deg h_{1} ⩾ ⋯ ⩾ deg h_{m}.  (28)

Then the following conditions are equivalent:

(a) there are no points of X where all the functions h_{1}, …, h_{m} vanish; (b) there are regular functions f_{1}, …, f_{m} on X such that 1 = ∑_{i=1}^{m} f_{i}h_{i}

and, for every i, the following inequality holds:

deg f_{i}h_{i} ⩽ \begin{cases}deg X ·N′(deg h_{1}, …, deg h_{m}; dim X), & if m ⩽ dim X, \\ 2deg X ·N′(deg h_{1}, …, deg h_{m}; dim X) − 1, & if m > dim X. \end{cases}

Now we take as 𝔸^{l} and X respectively V and \overline{G · b}. As the nonconstant function t_{i} is the restriction to \overline{G · b} of a linear function on V, we have

deg (t_{1} − α_{1}) = … = deg (t_{n} − α_{n}) = 1.  (29)

Further, in Theorem 2.13 put m = n and h_{i} = t_{i} − α_{i}, i = 1, …, n (by (29) condition (28) is fulfilled). Then it follows from (26), Theorem 2.13, and (14),

(29) that every property (c_{1}), (c_{2}), (c_{3}) in (26) is equivalent to the property

(c_{4}) there are functions f_{1}, …, f_{n} regular on \overline{G · b} such that ∑_{i=1}^{m}(t_{i} − α_{i})f_{i} − 1 = 0 and, for every i, the following inequality holds:

deg (t_{i} − α_{i})f_{i} ⩽ 2 deg \overline{G · b} − 1.  (30)

As \overline{G · b} is a conic (irreducible) subvariety of V, it follows from (27) and (29) that deg (t_{i} − α_{i})f_{i} = 1 + deg f_{i}. Therefore inequality (30) is equivalent to the inequality

deg f_{i} ⩽ 2 deg \overline{G · b} − 2.  (31)

3. The degree of the variety \overline{G · b} can be upper bounded. Namely, the orbit G · b is the image of the variety ρ(G) ⊂ End(V) under the linear map End(V)→ V, g ↦ g ·b. But it is easy to prove (see, e.g., [DK02, Prop. 4.7.10]) that degree does not increase under affine maps: if Y is a locally closed subset of 𝔸^{l} and φ: 𝔸^{l} → 𝔸^{m} is an affine map, then deg Y ⩾ deg \overline{φ(Y)}. Therefore deg \overline{G · b} is not bigger than the degree of the subvariety ρ(G) in End(V), i.e., the number d. Hence, by virtue of (31), for every i, the following inequality holds

deg f_{i} ⩽ 2d − 2.  (32)

4. As the comorphism ψ^{∗} (see (23)) is an embedding, we obtain the equivalence

∑_{i=1}^{m}(t_{i} − α_{i})f_{i} − 1 = 0 ⟺ ∑_{i=1}^{m}(ψ^{∗}(t_{i}) − α_{i})ψ^{∗}(f_{i}) − 1 = 0. (33)

By virtue of (25) and (33), it follows from inequality (32) and the definitions of functions t_{i}, numbers deg f_{i}, the “generic” polynomials F_{p} (see (16)), and the polynomial H (see (17)) that property (c_{4}) is equivalent to the following property:

(c_{5}) for every coefficient c_{p,q_{1},...,q_{n}} of every “generic” polynomial F_{p}, there is a constant ν_{p,q_{1},...,q_{n}} ∈ k such that after substitution of ν_{p,q_{1},...,q_{n}} in place of c_{p,q_{1},...,q_{n}} for every p, q_{1}, …, q_{n}, the right-hand side of formula (19) becomes zero of the field of rational functions in x_{1}, …, x_{r+s} with the coefficients in k:

∑_{(i_{1},...,i_{r+s})∈M} ℓ_{i_{1},...,i_{r+s}}(…, ν_{p,q_{1},...,q_{n}}, …) x_{1}^{i_{1}} ⋯ x_{r+s}^{i_{r+s}} = 0.  (34)

It remains to notice that as monomials x_{1}^{i_{1}} ⋯ x_{r+s}^{i_{r+s}}, where (i_{1}, …, i_{r+s}) ∈ M, are linearly independent over k, equality (34) is equivalent to vanishing of all the coefficients of the left-hand side,

ℓ_{i_{1},...,i_{r+s}}(…, ν_{p,q_{1},...,q_{n}}, …) = 0,

i.e., to that c_{p,q_{1},...,q_{n}} = ν_{p,q_{1},...,q_{n}} is a solution of system of linear equations (19) in variables c_{p,q_{1},...,q_{n}}. This completes the proof of the theorem. □

Remark 2.14. The proof shows that the claim of Theorem 2.12 remains true if the constant d in the definition of the “generic” polynomials F_{1}, …, F_{n} is replaced by deg \overline{G · b} = deg G · b. If, from some reasons, the number deg G · b is known, this permits to decrease the number of variables and equations in the system of linear equations (19). In some cases the degrees of orbits indeed have been computed.

Example 2.15. Consider the same situation as in Examples 2.6 and 2.10. Take a nonzero binary form v ∈ V_{s} and decompose it as a product v = v_{1}^{n_{1}} ⋯ v_{p}^{n_{p}}, where v_{1}, …, v_{p} are pairwize nonproportional forms from V_{1}. Assume that p ⩾ 3 and s/n_{i} ⩾ 2 for every i. Then the G-stabilizer G_{v} of the form v is finite [Po74] and |G_{v}|deg G·v = −2(p−1)s^{3}−4 ∑_{i=1}^{p}(s−n_{i})^{3}+3s^{2} ∑_{i=1}^{p}(s− n_{i}) + 3s ∑_{i=1}^{p}(s − s_{i})(s − 2n_{i}) (see the proof in [MJ92, Sect. 8]). In particular, if all the roots of the form v are simple, i.e., p = s, n_{1} = … = n_{s} = 1, then

|G_{v}|deg(G · v) = 2s(s − 1)(s − 2).  (35)

Formula (35) can also be deduced from a calculation of 1897 by Enriques and Fano; this has been done in 1983 by Mukai and Umemura (with a gap fixed in [MJ92, Sect. 8, Remark] where one can find the relevant references).

3. Defining the set \overline{G · L} by equations

3.1. Let L be a linear subvariety of V. Then there is a morphism

τ: 𝔸^{l} → V,

whose image is dense in L: for instance, one can take τ to be an affine embedding of 𝔸^{l} into V whose image is L. We fix such a morphism τ. Besides, like above we assume that a dominant morphism (2) is fixed.

We maintain the notation from Subsection 2.3. Like in the proof of Theorem 2.12, we denote by z_{1}, …, z_{n} the basis of V^{∗} dual to e_{1}, …, e_{n}. Besides, we denote by y_{1}, …, y_{l} the standard coordinate functions on 𝔸^{l}:

y_{i}(a) = δ_{i} for a = (δ_{1}, …, δ_{l}) ∈ 𝔸^{l}.

Then

τ(v) = ∑_{i=1}^{n} τ^{∗}(z_{i})(v)e_{i} for every v ∈ 𝔸^{l}.  (36)

3.2. The functions x_{1}, …, x_{r+s}, y_{1}, …, y_{l} can be naturally extended to the functions on 𝔸^{r,s}×𝔸^{l}; we denote these extensions by the same letters. Consider the morphism

μ: A^{r,s} × 𝔸^{l} → V, μ(u, v) = ι(u) · τ(v)  (37)

Then (4) and (36) imply that

f_{p} := μ^{∗}(z_{p}) = ∑_{q=1}^{n} ι^{∗}(ρ_{pq})τ^{∗}(z_{q}),  1 ⩽ p, q ⩽ n.  (38)

We identify 𝔸^{r,s} × 𝔸^{l} with the open subset of 𝔸^{r+s+l} by means of the embedding

𝔸^{r,s} × 𝔸^{l} ↪ 𝔸^{r+s+l}, ((ε_{1} …, ε_{r+s}), (δ_{1}, …, δ_{l})) ↦ (ε_{1} …, ε_{r+s}, δ_{1}, …, δ_{l})

and naturally extend the functions x_{1}, …, x_{r+s}, y_{1}, …, y_{l} to the functions on 𝔸^{r+s+l} maintaining the notation. Besides, we identify V with 𝔸^{n} by means of the isomorphism

V → 𝔸^{n}, ∑_{i=1}^{n} γ_{i}e_{i} ↦ (γ_{1}, …, γ_{n}).

Then morphism (37) becomes the rational map ϱ of the affine space 𝔸^{r+s+l} to the affine space 𝔸^{n}:

ϱ: 𝔸^{r+s+l} −−→ 𝔸^{n}, a ↦ (f_{1}(a), …, f_{n}(a)).

As \overline{ι(𝔸^{r,s}) · L} = \overline{G · L}, we have the equality

\overline{ϱ(𝔸^{r+s+l})} = \overline{G · L}.  (39)

This makes it possible to apply eliminaton theory to finding the equations that cut out \overline{G · L} in V. An algorithmic solution of this problem is obtained by means of Gröbner bases as follows.

3.3. The input of the algorithm We assume that the following data are known:

— The functions

ι^{∗}(ρ_{p,q})∈k[x_{1}, …, x_{r+s}, x_{1}^{−1}, …, x_{r}^{−1}] ⊂ k(𝔸^{r+s+l}),  1 ⩽ p, q ⩽ n. (40)

— The functions

τ^{∗}(z_{i}) ∈ k[y_{1}, …, y_{l}] ⊂ k[𝔸^{r+s+l}],  1 ⩽ i ⩽ n.  (41)

Example 3.4. Fix a point v ∈ L and a sequence f_{1}, …, f_{m} of linear independent vectors defining a parametric presentation L = {v+∑_{i=1}^{m} λ_{i}f_{i} | λ_{d}, …, λ_{m} ∈ k}. Take τ to be the embedding τ: 𝔸^{m} ↪ 𝔸^{n}, τ(λ_{1}, …, λ_{m}) = v + ∑_{i=m}^{l} λ_{i}f_{i}. Let v = ∑_{j=1}^{n} γ_{j}e_{j} and f_{i} = ∑_{j=1}^{n} ν_{ji}e_{j}. Then

τ^{∗}(z_{i}) = ∑_{i=1}^{n} ν_{ij}y_{j} + γ_{i}, 1 ⩽ i ⩽ n.

See Example 2.10 regarding the functions ι^{∗}(ρ_{p,q}).

3.5. The algorithm

The following sequence of steps together with Theorem 3.6 provide a constructive method to obtain the equations defining \overline{G · L} in V:

(1) Compute the rational functions f_{p} using formula (38) and write down each of them as a fraction of polynomials:

f_{p} = \frac{g_{p}}{h_{p}} , g_{p} ∈ k[x_{1}, …, x_{r+s}, y_{1}, …, y_{t}], h_{p} ∈ k[x_{1}, …, x_{r}]

(see (40) and (41)).

(2) Consider the polynomial ring k[t, x_{1}, …, x_{r+s}, y_{1}, …, y_{t}, z_{1}, …, z_{n}], where t is a new variable, and find for its ideal generated by the polynomials

h_{1}z_{1} − g_{1}, …, h_{n}z_{n} − g_{n}, 1 − h_{1} ⋯ h_{n}t.

a Gröbner basis with respect to an order of monomials such that every variable t, x_{i} y_{j} is bigger than every variable z_{p}.

Theorem 3.6. Let q_{1}, …, q_{m} be all the elements of this Gröbner basis that lie in k[z_{1}, …, z_{n}]. Then

\overline{G · L} = \{v ∈ 𝔸^{n} | q_{1}(v) = … = q_{m}(v) = 0\}.

Proof. We have \overline{ϱ(𝔸^{r+s+l})}=\{v ∈𝔸^{n} | q_{1}(v)=…=q_{m}(v)=0\}—this is a general fact about the closure of image of every rational map of one affine space to another, see, e.g., [CLO98, Chap. 3, § 3, Theorem 2]. Now the claim that we wish to prove follows from equality (39). □

Remark 3.7. Although the elements q_{1}, …, q_{m}, interesting for us, constitute a part of the Gröbner basis, for finding them by means of the described algorithm, we should find the whole of this basis.

References

[B95] K. Bongartz, Degenerations for representations of tame quivers, Ann. Sci. ÉNS 28 (1995), no. 5, 647–668.

[B68] N. Bourbaki, Groupes et Algèbres de Lie, Chap. IV, V, VI, Hermann, Paris, 1968.

[BHRZ99] T. Brustle, L. Hille, G. Röhrle, G. Zwara, The Bruhat–Chevalley order of parabolic group actions in general linear groups and degeneration for Δ-filtered modules, Adv. in Math. 148 (1999), no. 2, 203–242.

[B05] D. Burde, Degenerations of 7-dimensional nilpotent Lie algebras, Commun. Algebra 33 (2005), no. 4, 1259–1277.

[BS99] D. Burde, C. Steinhoff, Classification of orbit closures of 4-dimensional complex Lie algebras, J. Algebra 214 (1999), 729–739.

[C85] R. Carter, Finite Groups of Lie Type: Conjugacy Classes and Complex Characters, John Wiley & Sons, London, 1985.

[CM93] D. H. Collingwood, W. M. McGovern, Nilpotent Orbits in Semisimple Lie Algebras, Van Nostrand Reinhold, New York, 1993.

[CLO98] D. Cox, J. Little, D. O’Shea, Ideals, varieties, and Algorithms. An Introduction to Computational Algebraic Geometry and Commutative Algebra, 2nd ed., Undergraduate Texts in Mathematics, Springer, New York, 1998.

[DK02] H. Derksen, G. Kemper, Computational Invariant Theory, Encyclopaedia of Mathematical Sciences, Vol. 130, Subseries Invariant Theory and Algebraic Transformation Groups, Vol. I, Springer, Berlin, Heidelberg, 2002.

[GHR07] S. Goodwin, L. Hille, G. Röhrle, Orbits of parabolic subgroups on metabelian

ideals, 2007, arXiv:0711.3711.

[G58] A. Grothendieck, Torsion homologique et sections rationnelles, in: Anneaux de Chow et Applications, Séminaire C. Chevalley ENS 1958, Sec. math. 11 rue Pierre Cirue, Paris, 1958, pp. 5-10–5-29.

[J05] Z. Jelonek, On the effective Nulltellensatz, Invent. Math. 162 (2005), 1–17.

[Kas90] V. V. Kashin, Orbits of adjoint and coadjoint actions of Borel subgroups of semisimple algebraic groups, in: Problems in Group Theory and Homological Algebra, Yaroslavl’, 1990, pp. 141–159 (Russian).

[Kaz87] B. Ya. Kazarnovskiĭ, Newton polyhedra and the Bezout formula for matrix-valued functions of finite-dimensional representations, Funct. Anal. Appl. 21 (1987), no. 4, 319–321.

[MWZ99] P. Magyar, J. Weyman, A. Zelevinsky, Multiple flag varieties of finite type, Adv. Math. 141 (1999), no. 1, 97–118.

[MJ92] L. Moser-Jauslin, The Chow rings of smooth complete SL_{2}-embeddings, Compositio Math. 82 (1992), 67–106.

[MS01] K. Mulmuley, M. Sohoni, Geometric complexity theory I: An approach to the P vs. NP and related problems, SIAM J. Comput. 31 (2001), no. 2, 496–526.

[M65] D. Mumford, Geometric Invariant Theory, Ergebn. Math., Bd. 34, Springer-Verlag, Berlin, 1965.

[M76] D. Mumford, Algebraic Geometry I. Complex Projective Varieties, Grundlehren de mathematischen Wissenschaften, Vol. 221, Springer-Verlag, Berlin, 1976.

[Pe04] D. D. Pervouchine, Hierarchy of closures of matrix pencils, J. Lie Theory 14 (2004), 443–479.

[OVG94] A. L. Onishchik, E. B. Vinberg, V. V. Gorbatsevich, Structure of Lie groups and Lie algebras, in: Lie Groups and Lie Algebras III, Encyclopaedia of Mathematical Sciences, Vol. 41, Springer-Verlag, Berlin, 1994, pp. 1–248.

[Po74] V. L. Popov, Structure of the closure of orbits in spaces of finite-dimensional linear SL(2) representations, Math. Notes 16 (1974), 1159–1162.

[Po81] V. L. Popov, Constructive invariant theory, Astérisque 87–88 (1981), 303–334.

[Po03] V. L. Popov, The cone of Hilbert nullforms, Proc. Steklov Inst. of Math. 241 (2003), 177–194.

[PV72] V. L. Popov, E. B. Vinberg, On a class of quasihomogeneous affine varieties, Math. USSR-Izv. 6, 1972, 743–758.

[PV94] V. L. Popov, E. B. Vinberg, Invariant Theory, in: Algebraic Geometry IV, Enc. Math. Sci., Vol. 55, Springer, Berlin, 1994, pp. 123–284.

[R56] M. Rosenlicht, Some basic theorems on algebraic groups, Amer. J. Math. 78 (1956), 401–443.

[R61] M. Rosenlicht, On quotient varieties and the affine embedding of certain homogeneous spaces, Trans. Amer. Math. Soc. 101 (1961), 211–223.

[See90] C. Seeley, Degenerations of 6-dimensional nilpotent Lie algebras over ℂ, Comm. in Algebra 18 (1990), 3493–3505.

[Spa82] N. Spaltenstein, Classes Unipotentes et Sous-Groupes de Borel, Lecture Notes in Mathematics, Vol. 946, Springer-Verlag, New York, 1982.

[Spr98] T. A. Springer, Linear Algebraic Groups, 2nd Edition, Progress in Mathematics, Vol. 9, Birkhäser, Boston, 1998.

[TY05] P. Tauvel, R. W. T. Yu, Lie Algebras and Algebraic Groups, Springer Monographs in Mathematics, Springer, Berlin, 2005.

[W89] J. Weyman, The equations of conjugacy classes of nilpotent matrices, Invent. Math. 98 (1989), 229–245.

Steklov Mathematical Institute, Russian Academy of Sciences, Gubkina 8, Moscow, 119991, Russia

   E-mail address: popovvl@orc.ru