Decomposing some finitely generated groups into free products with amalgamation ^{1}

V.V. Benyash-Krivets

Introduction

. We shall say that a group G is a non-trivial free product with amalgamation if G = G_{1} ∗_{A} G_{2}, where G_{1} ≠ A ≠ G_{2} (see [1]). Wall [2] has posed the following question:

Which one-relator groups are non-trivial free products with amalgamation?

Let G = ⟨g_{1}, …, g_{m} | R_{1} = ⋯ = R_{n} = 1⟩ be a group with m generators and n relations such that def G = m − n ≥ 2. It is proved in [4] that G is a non-trivial free product with amalgamation. In particular, if G is a group with m ≥ 3 generators and one relation, then G is a non-trivial free product with amalgamation. A case of groups with two generators and one relation is more complicated. For example, the free abelian group G = ⟨a, b | [a, b] = 1⟩ of rank 2, where [a, b] = aba^{−1}b^{−1}, obviously, is not a non-trivial free product with amalgamation. Other examples are given by groups G_{n} = ⟨a, b | aba^{−1} = b^{n}⟩. For any n, the group G_{n} is solvable and using results from [3], it is easy to show that for n ≠ −1 the group G_{n} is not decomposable into a non-trivial free product with amalgamation. The following conjecture was stated in [4].

Conjecture 1 Let G = ⟨a, b | R^{m}(a, b) = 1⟩, m ≥ 2, be a group with two generators and one relation with torsion. Then G is a non-trivial free product with amalgamation.

Zieschang [5] has studied the problem of decomposing into non-trivial free products with amalgamation for discontinuous groups of transformations of the plane. He has given a complete answer to the question when such a group is a non-trivial free product with amalgamation in all cases except for the groups H_{1} = ⟨a, b | [a, b]^{n} = 1⟩ and H_{2} = ⟨a, b | a^{2} = [a, b]^{n} = 1⟩, n ≥ 2. Rosenberger [6] has proved that the groups H_{1} and H_{2} are non-trivial free products with amalgamation if n is not a power of 2. In recent papers [7, 8] it was proved that H_{1} is a non-trivial free product with amalgamation for arbitrary n ≥ 2. The independent proof of this fact was given in [9, 10].

1This research was carried out with the financial support of the Institute of Mathematics of the National Academy of Sciences of Belarus in the framework of the National Programme of Fundamental Research “Mathematical Structures”.

In the present paper we study the more general case, namely, we consider so-called generalized triangle groups, that is, groups G having a presentation of the form

G = ⟨a, b | a^{m} = b^{n} = R^{l}(a, b) = 1⟩,

where l ≥ 2, R(a, b) is a cyclically reduced word in the free group on a, b. Not all of these groups are decomposable into non-trivial free products with amalgamation. For example, Zieschang [5] has proved that the ordinary triangle group

T (m, n, l) = ⟨a, b | a^{m} = b^{n} = (ab)^{l} = 1⟩,

where m, n, l ≥ 2, is not a non-trivial free product with amalgamation. On the other hand, it was shown in [10] that the group G = ⟨a, b | a^{2m} = R^{l}(a, b) = 1⟩, where m ≥ 1, l ≥ 2, is a free product with amalgamation. Theorems 2 and 3 of the present paper contain more general results about decomposing of the generalized triangle groups into non-trivial free products with amalgamation.

In Theorem 1 we prove that a finitely generated group Γ is a non-trivial free product with amalgamation if the dimension of some algebraic variety (so-called character variety of irreducible representations of the group Γ into SL_{2}(ℂ)) is more than 1. To formulate this result we recall some notations and facts from the geometric representation theory (see also [11, 12, 13, 14]).

Let Γ = ⟨g_{1}, …, g_{m}⟩ be a finitely generated group and let G ⊂ GL_{n}(K) be a connected linear algebraic group defined over an algebraically closed field K of characteristic zero. Obviously, for each homomorphism ρ : Γ → G(K) the set of elements

(ρ(g_{1}), …, ρ(g_{m})) ∈ G(K) × ⋯ × G(K)

satisfies all defining relations of Γ. So the correspondence ρ → (ρ(g_{1}), …, ρ(g_{m})) is a bijection between the set Hom (Γ, G(K)) and the set of K-points in some affine Kvariety R(Γ, G) ⊂ G^{m}. The variety R(Γ, G) is usually called the representation variety of the group Γ into the algebraic group G.

The group G acts on R(Γ, G) in a natural way (by simultaneous conjugation of components) and its orbits are in one-to-one correspondence with the equivalence classes of representations of Γ. In a general case the orbits of the group G under this action are not necessarily closed and hence the variety of orbits (the geometric quotient) is not an algebraic variety. However, if G is a reductive group, then one can consider the categorical quotient X(Γ, G) = R(Γ, G)/G (see [15]). Its points parametrize closed G-orbits. In the case G = GL_{n}(K) or G = SL_{n}(K) an orbit of G is closed if and only if the corresponding representation is completely reducible. Therefore in this case points of the variety X(Γ, G) are in one-to-one correspondence with the equivalence classes of completely reducible representations of the group Γ into G or, in other words, with characters of representations of Γ into G.

Throughout we shall consider only the case G = SL_{2}(K) and for brevity we shall denote R(Γ), SL_{2}(K)) = R(Γ), X(Γ, SL_{2}(K)) = X(Γ). One can find all used below information about varieties R(Γ), X(Γ) in [12, 16, 17, 18]. We set

R^{s}(Γ) = {ρ ∈ R(Γ) | ρ is irreducible}, X^{s}(Γ) = π(R^{s}(Γ)),

where π : R(Γ) → X(Γ) is the canonical projection. It is shown in [12] that R^{s}(Γ), X^{s}(Γ) are open (in Zariski topology) subset of R(Γ), X(Γ) respectively. The aim of the present paper is to prove the following theorems.

Theorem 1 Let Γ be a finitely generated group such that dim X^{s}(Γ) ≥ 2. Then Γ is a non-trivial free product with amalgamation.

Theorem 2 Let k, m ∈ ℤ, k, m ≥ 2, let F_{2} = ⟨g, h⟩ be the free group of rank 2 with generators g, h, and let R(g, h) = g^{u_{1}}h^{v_{1}} … g^{u_{s}}h^{v_{s}} be a cyclically reduced word in F_{2} such that u_{i} ≠ 0, 0 < v_{i} < k, s ≥ 1. Suppose that there exists i ∈ {1, …, s} such that |u_{i}| ≥ 2. Let p be a prime number such that u_{i}p does not divide u_{j} for j ≠ i and let 0 ≠ f ∈ ℤ. We set n = u_{i}pf. Then in the following cases the group

Γ_{n} = ⟨a, b | a^{n} = b^{k} = R^{m}(a, b) = 1⟩

is a non-trivial free product with amalgamation.

1) m = 2, p does not belong to some finite set of prime numbers S. The set S is completely determined by the exponent k and the word R.

2) m = 3 or m = 2^{l} > 3, p ≠ 2;

3) m > 3 and m ≠ 2^{l}.

Note that the condition u_{i}p ∤ u_{j} for j ≠ i in Theorem 2 is carried out automatically if u_{i} = max_{1≤j≤s} u_{j} ≥ 2 or u_{i} ∤ u_{j} for each j ≠ i.

Theorem 3 Let Γ = ⟨a, b | a^{n} = R^{m}(a, b) = 1⟩, where n = 0 or n ≥ 2, m ≥ 2, R(a, b) = a^{u_{1}}b^{v_{1}} … a^{u_{s}}b^{v_{s}}, s ≥ 1, v_{i} ≠ 0, 0 < u_{i} < n. Then Γ is a non-trivial free product with amalgamation.

As a direct consequence of Theorem 3 we obtain the proof of Conjecture 1.

Corollary 1 Let Γ = ⟨a, b | R^{m}(a, b) = 1⟩, m ≥ 2, be a group with two generators and one relation with torsion. Then Γ is a non-trivial free product with amalgamation.

At the end of Section 2 we shall prove that the group Γ from Corollary 1 satisfies the assumptions of Theorem 1, that is, dim X_{2}^{s}(Γ) = 2 so that we obtain an another proof of Conjecture 1.

Corollary 2 Fuchsian groups H_{1} = ⟨a, b | [a, b]^{n} = 1⟩ and H_{2} = ⟨a, b | a^{2} = [a, b]^{n} = 1⟩, n ≥ 2, are non-trivial free products with amalgamation.

1. The proof of Theorem 1

In what follows we shall denote the field of p-adic numbers by ℚ_{p}, the ring of p-adic integers by ℤ_{p}, the group of invertible elements in ℤ_{p} by ℤ_{p}^{∗} , the p-adic valuation by | |_{p}, the trace of a matrix A by tr A, the identity 2 × 2 matrix by E.

We recall some facts about the character variety X(Γ) of representations of a finitely generated group Γ into SL_{2}(ℂ) (see [12]). For an arbitrary element g ∈ Γ one can consider the regular function

τ_{g} : R(Γ) → ℂ, τ_{g}(ρ) = tr ρ(g).

Usually, τ_{g} is called a Fricke character of the element g. It is known that the ℤ-algebra T (Γ) generated by all functions τ_{g}, g ∈ Γ is finitely generated. Moreover, if τ_{g_{1}}, …, τ_{g_{s}} are the generators of T (Γ) then the ℂ-algebra of SL_{2}(ℂ)-invariant regular functions ℂ[R(Γ)]^{SL_{2}(ℂ)} is equal to ℂ[τ_{g_{1}}, …, τ_{g_{s}}]. Consider the morphism

π : R(Γ) → 𝔸^{s}, π(ρ) = (τ_{g_{1}}(ρ), …, τ_{g_{s}}(ρ)).

It is shown in [12] that the image π(R(Γ)) is closed in 𝔸^{s}. Since X(Γ) and π(R(Γ)) are biregularly isomorphic, then throughout we shall identify X(Γ) and π(R(Γ)).

The idea of the proof of Theorem 1 is to construct a representation ρ : Γ → SL_{2}(ℚ_{p}) for some prime p such that the group ρ(Γ) is dense in SL_{2}(ℚ_{p}) in p-adic topology. After that Theorem 1 will follow from the following well known facts.

1) If H is a dense in p-adic topology subgroup of SL_{2}(ℚ_{p}) then H is a non-trivial free product with amalgamation (see [19]).

2) If f : G_{1} → G_{2} is an epimorphism of groups and G_{2} is a non-trivial free product with amalgamation then G_{1} is a non-trivial free product with amalgamation.

Lemma 1 Let H be a subgroup of SL_{2}(ℚ_{p}). Then H is dense in SL_{2}(ℚ_{p}) in p-adic topology if and only if H is absolute irreducible (that is, irreducible over algebraic closure of ℚ_{p}), unbounded, and non-discrete.

Proof. If H is dense in SL_{2}(ℚ_{p}) then obviously H is absolute irreducible, unbounded and non-discrete. Prove the inverse assertion. The unboundedness of H means that there exists an element h ∈ H such that | tr h|_{p} > 1. Indeed, otherwise the traces of all elements from H belong to ℤ_{p} whence the group H is conjugated to a subgroup of SL_{2}(ℤ_{p}) (see [20] or [12, lemma I.4.3]), that is, H is bounded. Let us show that the eigenvalues of the matrix h belong to ℚ_{p}. Let tr h = p^{−s}α, where α ∈ ℤ_{p}^{∗}, s > 0. Then the characteristic polynomial of h is of the form f(y) = y^{2} − p^{−s}αy + 1 and its discriminant is equal to D = p^{−2s}α^{2} − 4 = p^{−2s}(α^{2} − 4p^{2s}). Thus, D is a square in ℚ_{p} whence the roots of f(y) belong to ℚ_{p}. So h is conjugate in SL_{2}(ℚ_{p}) to a diagonal matrix of the form

diag(λ, λ^{−1}), λ = p^{−s}γ, s ≥ 0, γ ∈ ℤ_{p}^{∗}.  (1)

Without loss of generality we can suppose considering if necessary a group conjugated to H that h = diag(λ, λ^{−1}) ∈ H. Consider the following unipotent subgroups of SL_{2}(ℚ_{p}):

U_{1} = \{( \begin{matrix}1 & 0 \\ a & 1 \end{matrix} ) |a ∈ ℚ_{p}\} , U_{2} = \{( \begin{matrix}1 & a \\ 0 & 1 \end{matrix} ) |a ∈ ℚ_{p}\} .

It is well known that U_{1} and U_{2} generate SL_{2}(ℚ_{p}). Therefore it is sufficient to show that U_{1}, U_{2} ⊂ \overline{H}, where \overline{H} is the closure of H in p-adic topology. For example, let us prove that U_{1} ⊂ \overline{H}. To this end, first we shall show that \overline{H} contains a non-trivial unipotent element u = ( \begin{matrix}1 & 0 \\ a & 1 \end{matrix} ), a ∈ ℚ_{p}^{∗}. Let

Γ_{j} = \{( \begin{matrix}1 + p^{j}a & p^{j}b \\ p^{j}c & 1 + p^{j}d \end{matrix} ) ∈ SL_{2}(ℚ_{p})|a, b, c, d ∈ ℤ_{p}\}

be the principal congruence subgroup of a level j in SL_{2}(ℚ_{p}). The groups Γ_{j}, j ≥ 1, form a base for the neighborhoods of the identity in SL_{2}(ℚ_{p}). Then non-discreteness of H implies that for each j > 0 there exists an element E ≠ x_{j} ∈ H ∩ Γ_{j}. Let

x_{j} = ( \begin{matrix}1 + p^{j}a_{j} & p^{j}b_{j} \\ p^{j}c_{j} & 1 + p^{j}d_{j} \end{matrix} ) ,

where a_{j}, b_{j}, c_{j}, d_{j} ∈ ℤ_{p}. Then lim_{j→∞} x_{j} = E. Since H is an absolutely irreducible subgroup of SL_{2}(ℚ_{p}), then without loss of generality we can suppose that almost all elements b_{j}, c_{j} (with the exception of a finite number) are not equal to 0 (otherwise one can conjugate all x_{j} by a suitable element y from H, and again we shall have lim_{j→∞} yx_{j}y^{−1} = E). Let c_{j} = p^{k_{j}}ε_{j}, where k_{j} ≥ 0, ε_{j} ∈ ℤ_{p}^{∗}, and let t_{j} = [(k_{j} + j)/(2s)] be the integer part of the number (k_{j} + j)/(2s), where s is defined in (1). Then we have r_{j} = k_{j} + j − 2st_{j} ∈ {0, 1, …, 2s − 1}. Now consider the following sequence {x′_{j}} of elements of H:

x′_{j} = h^{−t_{j}}x_{j}h^{t_{j}} = ( \begin{matrix}1 + p^{j}a_{j} & p^{j+2st_{j}}γ^{−2t_{j}}b_{j} \\ p^{r_{j}}γ^{2t_{j}}ε_{j} & 1 + p^{j}d_{j} \end{matrix} ) .  (2)

The infinite sequence {p^{r_{j}}γ^{2t_{j}}ε_{j}} is contained in ℤ_{p} hence it is bounded. Therefore one can choose from it the convergent subsequence

a_{j_{m}} = p^{r_{jm}}γ^{2t_{jm}}ε_{j_{m}}.

Since |a_{j_{m}}|_{p} = p^{−r_{jm}} ≥ p^{−2s+1}, then lim_{j_{m}→∞} a_{j_{m}} = a ≠ 0. Thus, it follows from (2) that

lim_{j_{m}→∞} x′_{j_{m}} = ( \begin{matrix}1 & 0 \\ a & 1 \end{matrix} ) = u ∈ \overline{H}.

Conjugating u by a suitable power of h, we can suppose that a ∉ ℤ_{p}. Furthermore,

u^{n} = ( \begin{matrix}1 & 0 \\ an & 1 \end{matrix} ) for arbitrary n ∈ ℤ. Since the closure of aℤ in ℚ_{p} contains ℤ_{p}, then \overline{H}

contains a group U′_{1} = \{( \begin{matrix}1 & 0 \\ x & 1 \end{matrix} ) |x ∈ ℤ_{p}\}. Furthermore, let u_{1} = ( \begin{matrix}1 & 0 \\ p^{−r}β & 1 \end{matrix} ) ∈ U_{1} be an element such that r > 0 and β ∈ ℤ_{p}^{∗}. Choose an integer m such that 2sm − r ≥ 0. Then h^{m}u_{1}h^{−m} ∈ U′_{1} whence u_{1} ∈ \overline{H}. Thus, we have proved that U_{1} ⊂ \overline{H}. Similarly, one can proved that U_{2} ⊂ \overline{H}. Hence \overline{H} = SL_{2}(ℚ_{p}). Lemma 1 is proved.

Lemma 2 Let X,Y be irreducible ℚ-defined affine varieties, let dim Y ≥ 1, and let f : X → Y be a dominant ℚ-defined regular morphism. Then there exists a prime number p ≠ 2 and a point x ∈ X(ℚ_{p}) such that not all coordinates of the image f(x) ∈ Y (ℚ_{p}) belong to a ring ℤ_{p}.

Proof. Let K be the algebraic closure of ℚ. Let D be an arbitrary irreducible curve in Y (K), and let L be an arbitrary irreducible curve such that L ⊂ f^{−1}(D) and f(L) is dense in D. Let \overline{D} and \overline{L} be the projective closure of D and L respectively, and let L̃ be the smooth projective model of \overline{L}. The regular morphism f : L → D determines a rational morphism f̃: L̃ → \overline{D}. Since any rational morphism from a smooth curve to a projective variety is regular and the image of a projective variety under regular map is closed (see [21]), then f̃ is a regular surjective morphism. Let v ∈ \overline{D}\D be a point at infinity on \overline{D} and let w ∈ f̃^{−1}(v). The coordinates of points v and w generate the finite extension K_{1}/ℚ. By Chebotarev’s density theorem there exist infinitely many prime numbers p such that K_{1} ⊂ ℚ_{p}. Choose one of such p. Then w ∈ L̃(ℚ_{p}), v ∈ \overline{D}(ℚ_{p}). Since w is a non-singular point on L̃, then w has p-adic neighborhood W ⊂ L̃(ℚ_{p}) such that W is homeomorphic to an area in ℚ_{p} (see [21, chapter II]). This means that there exists an infinite sequence of elements w_{i} ∈ W such that w_{i} ∈ L(ℚ_{p}) and lim_{i→∞} w_{i} = w in p-adic topology. Then by continuity of f̃we have lim_{i→∞} f̃(w_{i}) = v. Since v ∈ \overline{D}(ℚ_{p}) is a point at infinity, then the sequence of elements f(w_{i}) = f̃(w_{i}) ∈ D(ℚ_{p}) is not bounded. This means that there exists i such that not all of the coordinates of the point f(w_{i}) belong to ℤ_{p}. Lemma 2 is proved.

The proof of Theorem 1. Let g_{1} …, g_{s} ∈ Γ be elements such that the functions τ_{g_{1}}, …, τ_{g_{s}} generate the ring T (Γ). Then the projection π : R(Γ) → X(Γ) is defined by the formula π(ρ) = (τ_{g_{1}}(ρ), …, τ_{g_{s}}(ρ)). Since by assumption of Theorem 1 we have dim X^{s}(Γ) ≥ 2, then there exists an irreducible component Z of the variety X(Γ) such that dim Z ≥ 2 and U = Z ∩ X^{s}(Γ) ≠ ø . Let p_{i} : Z → 𝔸^{1} be the projection defined by the formula p_{i}(z_{1}, …, z_{s}) = z_{i}. Since dim Z ≥ 2, then there exists i such that the projection p_{i} is dominant, that is p_{i}(U) is dense in 𝔸_{1}. Therefore there exists an integer n > 2 such that n ∈ p_{i}(U). Let Y = p_{i}^{−1}(n) ⊂ Z. Then by the Dimension Theorem dim Y ≥ dim Z − 1 ≥ 1 and Y ∩ U ≠ ø . Furthermore, let X be an irreducible component of π^{−1}(Y) such that π(X)) is dense in Y. Applying Lemma 2 to varieties X, Y and morphism π, we obtain that there exists a prime p such that X(ℚ_{p}) contains a representation ρ with the following property: ρ is irreducible and not all coordinates of the point π(ρ) belong to ℤ_{p}. The last means that there exists j such that τ_{g_{j}}(ρ) = tr ρ(g_{j}) ∉ ℤ_{p}. Hence the group ρ(Γ) is an unbounded subgroup of SL_{2}(ℚ_{p}). Moreover, it follows from the construction of the representation ρ that τ_{g_{i}}(ρ) = tr ρ(g_{i}) = n > 2. Thus, the cyclic subgroup of ρ(Γ) generated by ρ(h_{i}) is infinite and bounded. Hence, ρ(Γ) is a non-discrete subgroup of SL_{2}(ℚ_{p}). It follows from Lemma 1 that ρ(Γ) is dense in SL_{2}(ℚ_{p}) in p-adic topology whence ρ(Γ) (and consequently Γ) is a non-trivial free product with amalgamation. Theorem 1 is proved.

2. Some auxiliary results

In this section we shall prove several auxiliary results used in the proofs of Theorems 2 and 3. Throughout we shall denote the ring of algebraic integers in ℂ by O, the group of units in O by O^{∗}, the free group of a rank 2 with generators g and h by F_{2} =< g, h >, the greatest common divisor of integers a and b by (a, b). If K ⊃ L is a finite extension of fields and x ∈ K, then we shall denote the norm of x by N_{K/L}(x). The following lemma characterizes elements of the finite order in SL_{2}(ℂ).

Lemma 3 Let 2 < m ∈ ℤ and ±E ≠ X ∈ SL_{2}(ℂ). Then X^{m} = E if and only if tr X = ε + ε^{−1}, where ε^{m} = 1, ε ≠ ±1 (in other words, tr X = 2 cos(2rπ/m) for some r ∈ {1, …, m − 1}). In particular, if tr X = 0, then X^{2} = −E.

Proof. If X^{m} = E, then the assertion is obvious. If tr X = ε + ε^{−1}, then ε, ε^{−1} are the eigenvalues of the matrix X. Hence, X is conjugated with the matrix diag(ε, ε^{−1}), that is, X^{m} = E as required.

Obviously, the representation variety R(F_{2}) of the free group F_{2} = ⟨g, h⟩ is equal to SL_{2}(ℂ) × SL_{2}(ℂ). It is known that the ring T (F_{2}) is generated by the functions τ_{g}, τ_{h}, τ_{gh} (see [12, 16, 17]). For an element u ∈ F_{2} the function τ_{u} is usually called a Fricke character of the element u.

Lemma 4 For all α, β, γ ∈ ℂ there exist matrices A, B ∈ SL_{2}(ℂ) such that τ_{g}(A, B) = tr A = α, τ_{h}(A, B) = trB = β, τ_{gh}(A, B) = tr AB = γ.

This lemma can be easily proved by straightforward computations.

Lemma 4 implies that X(F_{2}) = π(R(F_{2})) = 𝔸^{3}. Moreover, the functions τ_{g}, τ_{h}, τ_{gh} are algebraically independent over ℂ and for all u ∈ F_{2} we have

τ_{u} = Q_{u}(τ_{g}, τ_{h}, τ_{gh}),

where Q_{u} ∈ ℤ[x, y, z] is a uniquely determined polynomial with integer coefficients. The polynomial Q_{u} is usually called the Fricke polynomial of the element u. The following relations for Fricke characters follow from the relations between the traces of arbitrary matrices in SL_{2}(ℂ):

1)τ_{u^{−1}} = τ_{u}; 2)τ_{uv} = τ_{vu}; 3)τ_{vuv^{−1}} = τ_{u}; 4)τ_{uv} = τ_{u}τ_{v} − τ_{uv^{−1}}.  (3)

Furthermore, we require the more detailed information on the Fricke polynomials (see [22]). Consider polynomials P_{n}(λ) satisfying the initial conditions P_{−1}(λ) = 0, P_{0}(λ) = 1 and the recurrence relation

P_{n}(λ) = λP_{n−1}(λ) − P_{n−2}(λ).

If n < 0, then we set P_{n}(λ) = −P_{|n|−2}(λ). The degree of the polynomial P_{n}(λ) is equal to n if n > 0 and to |n| − 2 if n < 0. It is easy to verify by induction on n that

P_{n}(2 cos(φ)) = \frac{sin((n + 1)φ)}{sin(φ)}.  (4)

It follows from (4) that the polynomial P_{n}(λ), n ≥ 1, has n zeros described by the formula

λ_{n,k} = 2 cos(\frac{kπ}{n+1}),  k = 1, 2, …, n.  (5)

Moreover, it is easy to verify by induction that for n ≥ 0 we have

P_{2n}(λ) = λ^{2n} + ⋯ + (−1)^{n}
P_{2n−1}(λ) = λ(λ^{2n−2} + ⋯ + (−1)^{n−1}n).  (6)

Furthermore, let w = g^{α_{1}}h^{β_{1}} … g^{α_{s}}h^{β_{s}} be a cyclically reduced word in F_{2} and let x = τ_{g}, y = τ_{h}, z = τ_{gh}. Let us treat the Fricke polynomial Q_{w}(x, y, z) as a polynomial in z. Let

Q_{w}(x, y, z) = M_{n}(x, y)z^{n} + M_{n−1}(x, y)z^{n−1} + … + M_{0}(x, y)

Lemma 5 ([22]) The degree of the Fricke polynomial Q_{w}(x, y, z) with respect to z is equal to s, that is, the number of blocks of the form g^{α_{i}}h^{β_{i}} in w. The leading coefficient M_{s}(x, y) of the polynomial Q_{w}(x, y, z) has the following form

M_{s}(x, y) = ∏_{i=1}^{s} P_{α_{i}−1}(x))P_{β_{i}−1}(y).  (7)

The following lemma plays an important role in the proof of Theorems 2 and 3.

Lemma 6 Let Γ = ⟨a, b | a^{n} = R^{m}(a, b) = 1⟩, where n = 0 or n ≥ 2, m ≥ 2, R(a, b) is a cyclically reduced containing b word in the free group on a and b. Assume that there exists matrices A, B ∈ SL_{2}(ℂ) such that tr A = α = 2 cos(tπ/n) for some t ∈ {1, …, n − 1} and tr R(A, B) = Q_{R}(α, y, z) = c, where Q_{R} is the Fricke polynomial of an element R(g, h) ∈ F_{2}, c = 2 cos(rπ/m) for some r ∈ {1, …, m − 1}, y = tr B, z = tr AB. Let H =< A, B > be the group generated by matrices A and B. Assume that the following two conditions hold:

1) there exists a unipotent (or finite-order) element W ∈ H of the form W = A^{α_{1}}B^{β_{1}} … A^{α_{s}}B^{β_{s}}, where α_{i}, β_{i} ≠ 0 for i = 1, …, s, such that l = ∑_{i=1}^{s} β_{i} ≠ 0.

2) There exists an element h ∈ H such that tr h ∉ O.

Then the group Γ is a non-trivial free product with amalgamation.

If instead of the condition 1) the following condition holds

1′) B has finite order, that is, tr B = 2 cos(k_{1}π/k) for some k ≥ 2, (k_{1}, k) = 1, then the group Γ_{1} = ⟨a, b | a^{n} = b^{kv} = R^{m}(a, b) = 1⟩ is a non-trivial free product

with amalgamation for any integer v.

The proof of this lemma is based on the Bass’ classification of finitely generated subgroups in SL_{2}(ℂ) [23].

Proposition 1 ([23]) Let H ⊂ GL_{2}(ℂ) be a finitely generated subgroup. Then one of the following cases must occur:

1) there exists an epimorphism f : H → ℤ such that f(u) = 0 for all unipotent elements u ∈ H;

2) tr h ∈ O for any element h ∈ H;

3) H is a non-trivial free product with amalgamation.

Proof of lemma 6. It is easy to see that the group H does not satisfy the conditions 1), 2) of Proposition 1. Indeed, assume that f : H → ℤ is an epimorphism such that f(z) = 0 for all unipotent elements z ∈ H. Then f(A) = 0 because A^{2n} = E by Lemma 3. Furthermore, f(u) = lf(B) = 0 whence f(B) = 0 because by assumption u is either unipotent or has finite order and l ≠ 0. Thus, f(H) = {0} which is a contradiction. By assumption H does not satisfy condition 2) of Proposition 1. Hence H is a non-trivial free product with amalgamation, that is, H = H_{1} ∗_{F} H_{2}, where H_{1} ≠ F ≠ H_{2}. Let \overline{A}, \overline{B}, \overline{H}, \overline{H}_{1}, \overline{H}_{2}, \overline{F} be the images of A, B, H, H_{1}, H_{2}, F in PSL_{2}(ℂ), respectively. If −E ∉ H, then groups H and \overline{H} are isomorphic. If −E ∈ H, then −E belongs to the centre of H hence −E ∈ F. In all of these cases, \overline{H}_{1} ≠ \overline{F} ≠ \overline{H}_{2} and hence \overline{H} = \overline{H}_{1}∗\overline{F} \overline{H}_{2} is a non-trivial free product with amalgamation. The conditions tr A = α and Q_{R}(α, y, z) = c imply by Lemma 3 that A^{2n} = R^{2m}(A, B) = E. Hence \overline{A}^{n} = R^{m}(\overline{A}, \overline{B}) = 1 in PSL_{2}(ℂ). Thus, \overline{H} is an epimorphic image of Γ therefore Γ is a non-trivial free product with amalgamation too.

Furthermore, if we replace the condition 1) to 1′), then again the group \overline{H} is a nontrivial free product with amalgamation. Moreover, we have \overline{A}^{n} = \overline{B}^{k} = R^{m}(\overline{A}, \overline{B}) = 1 in PSL_{2}(ℂ). Hence \overline{H} is an epimorphic image of Γ_{1}. Thus, Γ_{1} is a non-trivial free product with amalgamation. Lemma 6 is proved.

Lemma 7 1) Let r, s ∈ ℤ, where s ≥ 3 and (r, s) = 1. Then cos(rπ/s) ∉ O. 2) Let s ∈ ℤ, s ≥ 1, and r ≢ 0(mod 2s + 1). Then 2 cos(rπ/(2s + 1)) ∈ O^{∗}.

3) Let u ∈ ℤ, |u| ≥ 1, and let p be a prime number. Set

x_{r} = 2 cos(\frac{rπ}{2pu}), y_{r} = 2 sin(\frac{rπ}{2pu}), K_{r} = ℚ(x_{r}), L_{r} = ℚ(y_{r}).

Then there exist r, r_{1} ≢ 0(mod p) such that p divides N_{K_{r}/ℚ}(x_{r}) and N_{L_{r_{1}}/ℚ}(y_{r_{1}}). In particular, x_{r}, y_{r_{1}} ∉ O^{∗}.

4) Let u, c ∈ ℤ, |u| ≥ 2, c ≠ 0, and let p be a prime number not dividing c. Set x_{0} = −2 cos(π/u), x_{r} = 2 cos(rπ/(pu)). Then there exists r ≢ 0(mod p) such that c/(x_{r} − x_{0}) ∉ O.

5) Let p > 2 be a prime number. Then for each r ≢ 0(mod p) and s ≥ 1 we have sin(rπ/p^{s}) ∉ O^{∗}.

6) Let t ≥ 1. Then for each odd r we have 2sin(rπ/2^{t}) ∉ O^{∗}.

Proof. 1) Assume that cos(rπ/s) ∈ O. Then for each d ∈ ℤ we have cos(drπ/s) ∈ O. Since by assumption (r, s) = 1, then for each integer l there exists d such that dr ≡ l (mod s). Hence for each integer l we have cos(lπ/s) ∈ O. By (5) the polynomial P_{s−1}(λ) has the roots 2 cos(lπ/s), l = 1, …, s − 1. Then the polynomial P_{s−1}(2λ) has the roots cos(lπ/s), l = 1, …, s − 1. If s = 2s_{1} + 1 is odd, then by (6) we have P_{2s_{1}}(2λ) = 2^{2s_{1}}λ^{2s_{1}} + ⋯ + (−1)^{s_{1}}. Since 1/2^{2s_{1}} ∉ ℤ, then P_{2s_{1}}(2λ) has a root not belonging to O, that is, there exists l such that cos(lπ/s) ∉ O. If s = 2s_{1} is even, then (6) implies that P_{2s_{1}−1}(2λ) = 2λ(2^{2s_{1}−2}λ^{2s_{1}−2} + ⋯ + (−1)^{s_{1}−1}s_{1}). By assumption s ≥ 3 hence s_{1} ≥ 2. Then s_{1}/2^{2s_{1}−2} ∉ ℤ and P_{2s_{1}−1}(2λ) has a root not belonging to O.

2) By (5), (6) the number 2 cos(rπ/(2s + 1)) is a root of the polynomial P_{2s}(λ) = λ^{2s} + ⋯ + (−1)^{s} and therefore belongs to O^{∗}.

3) Since y_{r} = 2 cos((pu − r)π/(2pu)) = x_{pu−r}, then it is sufficient to prove the assertion for x_{r}. Let u = p^{f}u′, where f ≥ 0, p ∤ u′, and let r = r_{1}u′, where p ∤ r_{1}. Then x_{r} = 2 cos(r_{1}π/(2p^{f+1})). By (5), (6) the polynomial

P_{2p^{f+1}−1}(λ) = λ(λ^{2p^{f+1}−2} + ⋯ + (−1)^{p^{f+1}−1}p^{f+1})

has the roots 2 cos(r′π/(2p^{f+1})), r′ = 1, …, 2p^{f+1} − 1, and the polynomial

P_{2p^{f} −1}(λ) = λ(λ^{2p^{f} −2} + ⋯ + (−1)^{p^{f} −1}p^{f})

has the roots 2 cos(r′π/(2p^{f})), r′ = 1, …, 2p^{f} − 1. Hence the polynomial P_{2p^{f} −1}(λ) divides the polynomial P_{2p^{f+1}−1}(λ), that is,

P_{2p^{f+1}−1}(λ) = P_{2p^{f} −1}(λ)F (λ),  (8)

where, as it is easy to see, F (λ) is the polynomial of degree 2(p^{f+1} − p^{f}) with the leading coefficient 1 and the constant term p. The roots of F (λ) are the numbers 2 cos(r′π/(2p^{f+1})), r′ ≢ 0(mod p). Let K be the splitting field of F (λ). Obviously, there exists r′ ≢ 0(mod p) such that N_{K/ℚ}(2 cos(r′π/(2p^{f+1}))) = ±p as required.

4) Note that

x_{r} − x_{0} = 2 cos(\frac{rπ}{pu}) + 2 cos(\frac{π}{u}) = (2 cos(\frac{(r+p)π}{2pu})) (2 cos(\frac{(r−p)π}{2pu})) .

Therefore it is sufficient to show that for some r ≢ 0(mod p) we have c/α_{r} ∉ O, where α_{r} = 2 cos((r + p)π/(2pu)). Let K_{r} = ℚ(α_{r}). By item 3) proven above, there exists r ≢ 0 (mod p) such that p divides N_{K_{r}/ℚ}(α_{r}). Then

N_{K_{r}/ℚ} (\frac{c}{α_{r}} ) = \frac{c}{N_{K_{r}/ℚ}(α_{r})} ∉ ℤ

because by assumption p ∤ c. Hence c/α_{r} ∉ O, as required.

5) Note that 1/ sin(rπ/p^{s}) = 2/(2 cos((p^{s} − 2r)π/(2p^{s}))). It follows from the proof of item 4) that there exists r_{0} ≢ 0(mod p) such that

\frac{2}{2 cos((p^{s} − 2r_{0})π/(2p^{s}))} ∉ O.

Now we shall show that for every r ≢ 0(mod p) we have sin(rπ/p^{s}) ∉ O. Assume the contrary. Let for some r, (r, p) = 1, we have 1/ sin(rπ/p^{s}) ∈ O. Since p ∤ (p^{s} − 2r_{0}), then there exists d such that r ≡ d(p^{s} − 2r_{0})(mod p^{s}). Then by (4) we have

P_{d} (2 cos(\frac{(p^{s}−2r_{0})π}{p^{s}})) = \frac{sin(d(p^{s} − 2r_{0})π/p^{s})}{sin((p^{s} − 2r_{0})π/p^{s})} = ±\frac{sin(rπ/p^{s})}{sin((p^{s} − 2r_{0})π/p^{s})}.

So we immediately obtain that

\frac{1}{sin((p^{s} − 2r_{0})π/p^{s})} = ±\frac{1}{sin(rπ/p^{s})}P_{d} (2 cos(\frac{(p^{s}−2r_{0})π}{p^{s}})) ∈ O

which is a contradiction.

6) For t = 1 the assertion is obvious. Suppose that t > 1. By item 3) there exists an odd r_{0} such that 2 sin(r_{0}π/2^{t}) ∉ O^{∗}. Prove that for each odd r we have 2 sin(rπ/2^{t}) ∉ O^{∗}. Assume the contrary. Let for some odd r we have 2 sin(rπ/2^{t}) ∈ O^{∗}. Obviously, there exists an integer d such that r ≡ dr_{0} (mod 2^{t}). Then by (4) we have

P_{d} (2 cos(\frac{r_{0}π}{2^{t}})) = ±\frac{2 sin(rπ/2^{t})}{2 sin(r_{0}π/2^{t})}.

The last equality implies that 2 sin(r_{0}π/2^{t}) ∈ O^{∗} which is a contradiction. Lemma 7 is proved.

Lemma 8 1) Let s, t ≥ 0. Then

P_{s}(λ)P_{t}(λ) = ∑_{i=0}^{t} P_{s−t+2i}(λ).  (9)

2) The polynomial P_{s}(λ) − P_{s−1}(λ) has the roots λ_{r} = 2 cos((2r + 1)π/(2s + 1)), r ∈ {0, 1, …, s − 1}.

3) If γ = 2 cos(2rπ/(2s + 1)), where s ≥ 1, r ∈ {1 …, s}, (r, 2s + 1) = 1, then P_{s}(γ) − P_{s−1}(γ) ∉ O^{∗}.

4) If γ = 2 cos((2r + 1)π/(2s)), where s ≥ 2, (s, 2r + 1) = 1, then 0 ≠ P_{s−1}(γ) ∉ O^{∗}.

5) Let γ ∈ O. Assume that γ is not equal to 2 cos(rπ/s), where r, s ∈ ℤ. Then there exists an integer l > 0 such that P_{l}(γ) ∉ O^{∗}.

Proof. 1) We fix s and proceed the induction on t. If t = 0, then P_{s}(λ)P_{0}(λ) = P_{s}(λ). If t = 1, then P_{s}(λ)P_{1}(λ) = P_{s}(λ)λ = P_{s+1}(λ) + P_{s−1}(λ) by definition. Furthermore, we have by induction

P_{s}(λ)P_{t}(λ) = P_{s}(λ)(λP_{t−1}(λ) − P_{t−2}(λ)) = λ ∑_{i=0}^{t−1} P_{s−t+1+2i}(λ) − ∑_{i=0}^{t−2} P_{s−t+2+2i}(λ)

= ∑_{i=0}^{t−1}(P_{s−t+2+2i}(λ) + P_{s−t+2i}(λ)) − ∑_{i=0}^{t−2} P_{s−t+2+2i}(λ)

= P_{s+t}(λ) + ∑_{i=0}^{t−1} P_{s−t+2i}(λ) = ∑_{i=0}^{t} P_{s−t+2i}(λ)

as required.

2) Bearing in mind (4), we obtain

P_{s}(λ_{r}) − P_{s−1}(λ_{r}) = \frac{sin((2r + 1)(s + 1)π/(2s + 1)) − sin((2r + 1)sπ/(2s + 1))}{sin((2r + 1)π/(2s + 1))} = 0.

3) Taking into account (4), we have

\frac{1}{P_{s+1}(γ) − P_{s}(γ)} = \frac{sin(2rπ/(2s + 1))}{2 sin(rπ/(2s + 1)) cos(rπ)} = ± cos(\frac{rπ}{2s+1}) ∉ O

by item 1) of Lemma 7.

4) Using (4), we obtain

\frac{1}{P_{s−1}(γ)} = \frac{sin((2r + 1)π/(2s)}{sin((2r + 1)π/2)} = (−1)^{r} cos (\frac{(s−2r−1)π}{2s}) ∉ O

by item 1) of Lemma 7.

5) Since by (5) the polynomial P_{l}(λ) has the roots 2 cos(rπ/(l + 1)), r = 1, …, l, then one can write P_{l}(γ) = ∏_{r=1}^{l}(γ − 2 cos(rπ/(l + 1))). Hence it is sufficient to prove that γ − (ε + ε^{−1}) ∉ O^{∗}, where ε ≠ ±1 is some root of 1. Let f(λ) be the irreducible polynomial for γ over ℚ, let K_{0} be the splitting field of f(λ) and let K_{1} = K_{0}(x_{0}), where x_{0} is a root of the equation x+x^{−1} = γ. Let Z_{1} be the integer closure of ℤ in K_{1} and let p ≠ 2 be some prime number. Let 𝔭_{1} be some prime ideal in Z_{1} lying above (p). Then k_{1} = Z_{1}/𝔭_{1} ⊃ ℤ/pℤ = k is the finite extension of fields. We have x_{0}, y_{0} ∈ Z_{1}. Let \overline{x}_{0}, \overline{γ} be the images of x_{0}, γ in the field k_{1}, respectively. Then the equality holds

\overline{x}_{0} + \overline{x}_{0}^{−1} = \overline{γ}.

Let l = |k_{1}^{∗}| be the order of the multiplicative group of the field k_{1}. Then \overline{x}_{0}^{l} = 1 in k_{1}. Let K_{2} = K_{1}(ξ), where ξ is a primitive root of 1 of degree l in ℂ. Let Z_{2} be the integer closure of Z_{1} in K_{2}, let 𝔭_{2} be some prime ideal of Z_{2} lying above 𝔭_{1}, k_{2} = Z_{2}/𝔭_{2} ⊃ k_{1}. Let Δ be the group of roots of 1 of the degree l in K_{2} and let \overline{Δ} be its image in k_{2}. Prove that \overline{Δ} = k_{1}^{∗}. Assume the contrary, that is, \overline{Δ} ≠ k_{1}^{∗}. Then for some integer r, 0 < r < l, we have \overline{ξ}^{r} = 1, where \overline{ξ} is the image of ξ in k_{2}. Then (1 + y)^{l} = 1, that is, 1 + C_{l}^{1}y + ⋯ + C_{l}^{l}y^{l} = 1, where C_{l}^{i} is the corresponding binomial coefficient. Hence y(l + yy_{1}) = 0, where y_{1} = C_{l}^{2}y + ⋯ + C_{l}^{l}y^{l−1}. Since y ≠ 0, then we obtain that l ∈ 𝔭_{2} ∩ ℤ = (p). But l = |k_{1}^{∗}| = p^{t} − 1 for some t which is a contradiction. So there exists a root ε of 1 of degree l such that \overline{ε} = \overline{x}_{0}. This means that γ − (ε + ε^{−1}) ∈ 𝔭_{2} and hence γ − (ε + ε^{−1}) is not a unit in the ring O. Lemma 8 is proved.

Lemma 9 Let F_{2} = ⟨g, h⟩ be the free group with generators g and h. Set x = τ_{g}, y = τ_{h}, z = τ_{gh}, t = τ_{ghg^{−1}h^{−1}}. Then the following assertions hold.

1) t = x^{2} + y^{2} + z^{2} − xyz − 2.

2) Let R = gh(ghg^{−1}h^{−1})^{s}. Then

τ_{R} = (P_{s}(t) − P_{s−1}(t))z.

3) Let T = (gh)^{−1}(ghg^{−1}h^{−1})^{s}(gh)^{2}(ghg^{−1}h^{−1})^{s}. Then

τ_{T} = (t − 2)P_{s−1}(t)^{2}z^{3} + (2 − P_{2s−1}(t) + P_{2s−2}(t))z.

Proof. 1) One can prove the equality by straightforward computations using relations (3) (see [16]).

2) Let u, v be arbitrary elements of F_{2}. Then using induction and relations (3), it is easy to show that for arbitrary integers p and q the following equality is valid

τ_{u^{p}v^{q}} = P_{p−1}(τ_{u})P_{q−1}(τ_{v})τ_{uv} − P_{p−2}(τ_{u})P_{q}(τ_{v}) − P_{p}(τ_{u})P_{q−2}(τ_{v}).  (10)

Set now u = gh, v = ghg^{−1}h^{−1}. Then τ_{u} = z, τ_{v} = t, τ_{uv} = τ_{gh(ghg^{−1}h^{−1})} = zt − τ_{g^{−1}h^{−1}} = z(t − 1). Hence,

τ_{uv^{s}} = P_{s−1}(τ_{v})τ_{uv} − P_{s−2}(τ_{v})τ_{u} = P_{s−1}(t)(t − 1)z − P_{s−2}(t)z
= z(tP_{s−1}(t) − P_{s−1}(t) − P_{s−2}(t)) = z(P_{s}(t) + P_{s−2}(t) − P_{s−1}(t) − P_{s−2}(t))
= z(P_{s}(t) − P_{s−1}(t)).

3) Let u and v be such as above. Then using relations (3), (10), we have

τ_{u^{−1}v^{s}} = τ_{u^{−1}}τ_{v^{s}} − τ_{uv^{s}} = z(P_{s}(t) − P_{s−2}(t)) − z(P_{s}(t) − P_{s−1}(t))
= z(P_{s−1}(t) − P_{s−2}(t));
τ_{u^{2}v^{s}} = τ_{u}τ_{uv^{s}} − τ_{v^{s}} = z^{2}(P_{s}(t) − P_{s−1}(t)) − P_{s}(t) + P_{s−2}(t);
τ_{u^{3}} = z^{3} − 3z.

Hence,

τ_{u^{−1}v^{s}u^{2}v^{s}} = τ_{u^{−1}v^{s}}τ_{u^{2}v^{s}} − τ_{u^{3}} = z^{3}((P_{s}(t) − P_{s−1}(t))(P_{s−1}(t) − P_{s−2}(t)) − 1)
+ z(3 − (P_{s−1}(t) − P_{s−2}(t))(P_{s}(t) − P_{s−2}(t))).

Using (9), we simplify the last expression. First consider the coefficient by z^{3}.

(P_{s}(t) − P_{s−1}(t))(P_{s−1}(t) − P_{s−2}(t)) − 1 = P_{s}(t)P_{s−1}(t) + P_{s−1}(t)P_{s−2}(t)

− P_{s}(t)P_{s−2}(t) − P_{s−1}(t)^{2} − 1 = P_{s−1}(t)(P_{s}(t) + P_{s−2}(t)) − ∑_{i=1}^{s−1} P_{2i}(t) − P_{0}(t)

− P_{s−1}(t)^{2} = tP_{s−1}(t)^{2} − 2P_{s−1}(t)^{2} = (t − 2)P_{s−1}(t)^{2}.

Now consider the coefficient by z.

3 − (P_{s−1}(t) − P_{s−2}(t))(P_{s}(t) − P_{s−2}(t)) = 3 − P_{s}(t)P_{s−1}(t) + P_{s−1}(t)P_{s−2}(t)

+ P_{s}(t)P_{s−2}(t) − P_{s−2}(t)^{2} = 3 − ∑_{i=1}^{s} P_{2i−1}(t) + ∑_{i=1}^{s−1} P_{2i−1}(t) + ∑_{i=1}^{s−1} P_{2i}(t) − ∑_{i=0}^{s−2} P_{2i}(t)

= 2 − P_{2s−1}(t) + P_{2s−2}(t).

Lemma 9 is proved.

At the end of Section 2 we shall show how one can deduce Corollary 1 from Theorem 1. This gives us another proof of Conjecture 1. Let Γ = ⟨a, b | R^{m}(a, b) = 1⟩, where m ≥ 2, R(a, b) = a^{u_{1}}b^{v_{1}} … a^{u_{s}}b^{v_{s}}, u_{i}, v_{i} ≠ 0, s ≥ 1 and R(a, b) is not a proper power.

First consider the case m ≥ 3. Prove that dim X_{2}^{s}(Γ) ≥ 2. Then Theorem 1 immediately implies that Γ is a non-trivial free product with amalgamation. Consider in the character variety X_{2}(F_{2}) = 𝔸^{3} of the free group F_{2} = ⟨g, h⟩ the hypersurface V defined by the equation

τ_{R(g,h)}(x, y, z) = 2 cos(\frac{2π}{m}),  (11)

where x = τ_{g}, y = τ_{h}, z = τ_{gh}. By Lemma 5 one can write the equation (11) in the form

f(x, y, z) = M_{s}(x, y)z^{s} + ⋯ + M_{0}(x, y) − 2 cos(\frac{2π}{m}) = 0.  (12)

We claim that V ⊂ X_{2}(Γ). Indeed, let v = (x_{0}, y_{0}, z_{0}) ∈ V and let A, B ∈ SL_{2}(ℂ) be matrices such that tr A = x_{0}, tr B = y_{0}, tr AB = z_{0}. Then by Lemma 3 we have R^{m}(A, B) = E. Hence the pair of matrices (A, B) defines a representation ρ of the group Γ into SL_{2}(ℂ). Moreover, the image of ρ in X_{2}(Γ) coincides with v. Hence, v ∈ X_{2}(Γ). Furthermore, let V_{1}, …, V_{r} be the irreducible components of V. It is easy to see (see [21]) that dim V_{i} = 2 for each i. It remains to show that V ∩ X_{2}^{s}(Γ) ≠ ø . Let us assume the contrary. Then all representations corresponding to points from V are reducible. This means that the regular function τ_{ghg^{−1}h^{−1}} − 2 is identically equal to 0 on V. Hence by item 1 of Lemma 9 we have

g(x, y, z) = x^{2} + y^{2} + z^{2} − xyz − 4 ≡ 0

on V. Thus,

f(x, y, z) = g(x, y, z)^{d}  (13)

for some d ≥ 1. If for some i we have |u_{i}| ≥ 2 or |v_{i}| ≥ 2, then by Lemma 5 the leading coefficient M_{s}(x, y) in (12) is not a constant and the equality (13) is impossible.

Let now |u_{i}| = |v_{i}| = 1 for i = 1, …, s. First, if for some i we have u_{i} = u_{i+1} or v_{i} = v_{i+1} (u_{1} = u_{s} or v_{1} = v_{s} for i = s), then we can consider new generators of the group Γ. Assume for definiteness that u_{1} = u_{2}. Set a_{1} = a^{u_{1}}b^{v_{1}}, b_{1} = b. Then Γ = ⟨a_{1}, b_{1} | R_{1}^{m}(a_{1}, b_{1}) = 1⟩, where R_{1}^{m}(a_{1}, b_{1}) = a_{1}^{u′_{1}}b_{1}^{v′_{1}} … a_{1}^{u′_{r}}b_{1}^{v′_{r}}, u′_{i}, v′_{i} ≠ 0, r ≥ 1 and u′_{1} ≥ 2. This case has been considered above.

So we can assume that u_{i+1} = −u_{i}, v_{i+1} = −v_{i}. Since by assumption R(a, b) is not a proper power, then we have for R(a, b) the unique possibility up to cyclic rearrangement R(a, b) = aba^{−1}b^{−1}. Hence

f(x, y, z) = x^{2} + y^{2} + z^{2} − xyz − 2 − 2 cos(\frac{2π}{m}) = g(x, y, z) + 2 − 2 cos(\frac{2π}{m}).

Since 2 − 2 cos(\frac{2π}{m}) ≠ 0, then it is obvious that in this case g(x, y, z) has no zeros on V. So that for m ≥ 3 we have proved that the group Γ is a non-trivial free product with amalgamation.

Let now m = 2. In this case one can consider a group Γ_{1} = ⟨a, b | R^{4}(a, b) = 1⟩. We have proved above that dim X_{2}^{s}(Γ_{1}) ≥ 2. Then the proof of Theorem 1 implies that there exists a representation ρ : Γ_{1} → SL_{2}(ℚ_{p}) for some prime p such that ρ(Γ_{1}) is dense in SL_{2}(ℚ_{p}) in p-adic topology. Hence ρ(Γ_{1}) is a non-trivial free product with amalgamation. Let G be the image of ρ(Γ_{1}) in PSL_{2}(ℚ_{p}). Then it is easy to see that G is a non-trivial free product with amalgamation too. But G is an epimorphic image of Γ whence Γ is a non-trivial free product with amalgamation.

3. The proof of Theorem 2

1) Let Γ_{n} = ⟨a, b | a^{n} = b^{k} = R^{2}(a, b) = 1⟩ and let F_{2} = ⟨g, h⟩ be the free group with generators g and h. Set x = τ_{g}, β = τ_{h} = 2 cos(π/k), z = τ_{gh}. Consider the equation

Q_{R(g,h)}(x, β, z) = 0,  (14)

where Q_{R(g,h)} is the Fricke polynomial of the element R(g, h) ∈ F_{2}. By Lemma 5 we can write (14) in the form

A_{0}(x)z^{s} + ⋯ + A_{s}(x) = 0,  (15)

where A_{0}(x) = ∏_{i=1}^{s} P_{u_{i}−1}(x)P_{v_{i}−1}(β). Since by assumption there exists i such that |u_{i}| ≥ 2, then deg P_{u_{i}−1}(x) ≥ 1. Let x_{0} = −2 cos(π/u_{i}) be one of the roots of P_{u_{i}−1}(x). Then x − x_{0} divides A_{0}(x). Let A_{0}(x) = (x − x_{0})B_{0}(x), where B_{0}(x) ∈ O[x]. Write (15) in the form

(x − x_{0})B_{0}(x)z^{s} + ⋯ + A_{s}(x) = 0.  (16)

First we assume that all polynomials A_{1}(x), …, A_{s}(x) are divided on x − x_{0}. Then one can write (16) in the form

(x − x_{0})f(x, z) = 0,  (17)

where f(x, z) is some polynomial of x, z. Let z_{0} be an arbitrary non-integer element of ℂ, that is, z_{0} ∉ O, and let A, B ∈ SL_{2}(ℂ) be matrices such that tr A = x_{0}, tr B = β, tr AB = z_{0}. By construction the pair of matrices (A, B) defines a representation of the group Γ_{n} into PSL_{2}(ℂ). Applying Lemma 6, we obtain that Γ_{n} is a non-trivial free product with amalgamation.

Assume now that not all polynomials A_{1}(x), …, A_{s}(x) are multiples of x − x_{0}. Let, for example, A_{1}(x) is not a multiple of x−x_{0} and let 0 ≠ δ = A_{1}(x_{0}) ∈ O be the residue of A_{1}(x) modulo x − x_{0}. Let c = N_{ℚ(δ)/ℚ}(δ). For a finite set of prime numbers S from assertion of the theorem we take S = {p ∈ ℤ | p divides c}. Assume that n = u_{i}pf for some integer f and prime p ∉ S such that u_{i}p ∤ u_{j} for j ≠ i. Let x_{r} = 2 cos(rπ/(pu_{i})) for some r ≢ 0(mod p), K_{r} = ℚ(δ, x_{r} − x_{0}). By item 3) of Lemma 7 one can choose r such that p divides N_{K_{r}/ℚ}(x_{r} − x_{0}). Moreover, by construction p does not divide c. Then N_{K_{r}/ℚ}(δ/(x_{r} − x_{0})) ∉ ℤ hence δ/(x_{r} − x_{0}) ∉ O. Thus, we have

\frac{A_{1}(x_{r})}{x_{r} − x_{0}} ∉ O.

Furthermore, since p ∤ r and pu_{i} ∤ u_{j} for each j ≠ i, then B_{0}(x_{r}) ≠ 0. Set now x = x_{r} and write the equation (16) in the form

z^{s} + \frac{A_{1}(x_{r})}{(x_{r}−x_{0})B_{0}(x_{r})}z^{s−1} + ⋯ + \frac{A_{s}(x_{r})}{(x_{r}−x_{0})B_{0}(x_{r})} = 0  (18)

Clearly, we have A_{1}(x_{r})/((x_{r} −x_{0})B_{0}(x_{r})) ∉ O because B_{0}(x_{r}) ∈ O. Hence the equation (18) has a root z_{0} ∉ O. Consider matrices A, B ∈ SL_{2}(ℂ) such that

tr A = x_{r},  tr B = β,  tr AB = z_{0}.

By construction the pair of matrices (A, B) defines a representation of the group Γ_{n} into PSL_{2}(ℂ). Applying Lemma 6, we obtain that the group Γ_{n} is a non-trivial free product with amalgamation.

2) We keep notations of item 1). Consider the equation

Q_{R(g,h)}(x, β, z) = γ_{t},  (19)

where γ_{t} = 2 cos(tπ/m), m ∤ t. By Lemma 5 one can write (19) in the form

(x − x_{0})B_{0}(x)z^{s} + ⋯ + A_{s}(x) − γ_{t} = 0.  (20)

Let x_{r} = 2 cos(rπ/(pu_{i})), where r ≢ 0(mod p). Prove that there exist t and r such that (A_{s}(x_{r}) − γ_{t})/(x_{r} − x_{0}) ∉ O. Indeed, let us assume the contrary.

First, consider the case m = 3. Then γ_{1} = 1, γ_{2} = −1. Since both of the numbers (A_{s}(x_{r}) − 1)/(x_{r} − x_{0}) and (A_{s}(x_{r}) + 1)/(x_{r} − x_{0}) belong to O, then their difference 2/(x_{r} − x_{0}) ∈ O for each r ≢ 0(mod p). Since by assumption p ≠ 2, then we obtain a contradiction to item 4) of Lemma 7.

Now let m = 2^{l}. Then γ_{2^{l−1}} = 0, γ_{2^{l−2}} = \sqrt{2}. Since both of the numbers A_{s}(x_{r})/(x_{r}− x_{0}) and (A_{s}(x_{r}) − \sqrt{2})/(x_{r} − x_{0}) belong to O, then their difference \sqrt{2}/(x_{r} − x_{0}) ∈ O and therefore 2/(x_{r} − x_{0}) ∈ O. Again we obtain a contradiction to item 4) of Lemma 7.

So let us choose t and r such that (A_{s}(x_{r}) − γ_{t})/(x_{r} − x_{0}) ∉ O. Since p ∤ r and pu_{i} ∤ u_{j} for each j ≠ i, then B_{0}(x_{r}) ≠ 0. Set x = x_{r} and write (20) in the form:

z^{s} + ⋯ + \frac{A_{s}(x_{r})−γ_{t}}{(x_{r}−x_{0})B_{0}(x_{r})} = 0.  (21)

By construction (A_{s}(x_{r}) − γ_{t}))/((x_{r} − x_{0})B_{0}(x_{r})) ∉ O hence (21) has a root z_{0} ∉ O. Consider matrices A, B ∈ SL_{2}(ℂ) such that

tr A = x_{r},  tr B = β,  tr AB = z_{0}.

By construction the pair of matrices (A, B) defines a representation of the group Γ_{n} into PSL_{2}(ℂ). Applying Lemma 6, we obtain that the group Γ_{n} is a non-trivial free product with amalgamation.

3) Let m > 3 and m ≠ 2^{l}. We keep notations of item 2). Show that there exist t ≢ 0(mod m) and r ≢ 0(mod p) such that (A_{s}(x_{r}) − γ_{t})/(x_{r} − x_{0}) ∉ O. Indeed, let us assume the contrary. Let for each t ≢ 0(mod m) and each r ≢ 0(mod p) we have (A_{s}(x_{r}) − γ_{t})/(x_{r} − x_{0}) ∈ O.

First, consider the case, where m is odd. Then 1 + ∑_{i=1}^{2g} γ_{2i} = 0 as the sum of all roots of 1 of degree m. Note that −γ_{i} = γ_{m−i}. Set C_{i} = (A_{s}(x_{r}) − (−1)^{i}γ_{2i})/(x_{r} − x_{0}). Then we have

∑_{i=1}^{m−1}(−1)^{i}C_{i} = − ∑_{i=1}^{m−1} \frac{γ_{2i}}{x_{r} − x_{0}} = \frac{1}{x_{r} − x_{0}} ∈ O

for each r ≢ 0(mod p) which contradicts to item 4) of Lemma 7.

Now let m = m_{1}2^{g}, where g ≥ 1, m_{1} > 1 is odd. Consider the numbers γ_{i2^{g}} = 2 cos(iπ/m_{1}). Arguing just as in the case of odd m above, we obtain a contradiction to item 4) of Lemma 7.

So let us choose t and r such that (A_{s}(x_{r}) − γ_{t})/(x_{r} − x_{0}) ∉ O. Then the constant term in the equation (21) does not belong to O and (21) has a root z_{0} ∉ O. Let A, B ∈ SL_{2}(ℂ) be matrices such that

tr A = x_{r},  tr B = β,  tr AB = z_{0}.

By construction the pair of matrices (A, B) defines a representation of the group Γ_{n} into PSL_{2}(ℂ). Applying Lemma 6, we complete the proof of Theorem 2.

Remark. In several cases one can obtain the more precise information when a generalized triangle group Γ is a non-trivial free product with amalgamation. For example, consider the following group Γ_{k} = ⟨a, b | a^{2} = b^{k} = (ab^{2})^{3} = 1⟩. Then Theorem 2 implies that if k = 2k_{1}, where 1 < k_{1} ≠ 2^{l}, then Γ_{k} is a non-trivial free product with amalgamation. However it is easy to see that for k_{1} = 2^{l}, l ≥ 1, the group Γ_{k} is a non-trivial free product with amalgamation too. Indeed, let F_{2} = ⟨g, h⟩ be the free group, let x = τ_{g} = 0, y = τ_{h}, z = τ_{gh}. Consider the equation

Q_{gh^{2}}(0, y, z) = yz = 2 cos(π/3) = 1.

Let y = y_{r} = 2 cos(rπ/2^{l+1}). Then by item 6) of Lemma 7 for each odd r, we have that z_{r} = 1/y_{r} ∉ O. Now Lemma 6 implies that Γ_{k} is a non-trivial free product with amalgamation.

4. Proof of Theorem 3

First, assume that for a word R(a, b) = a^{u_{1}}b^{v_{1}} … A^{u_{s}}b^{v_{s}}, we have v = max_{1≤i≤s} |v_{i}| ≥ 2. Then by Theorem 2 there exists a prime p such that the group Γ_{1} = ⟨a, b | a^{n} = b^{pv} = R^{m}(a, b) = 1⟩ is a non-trivial free product with amalgamation. Since Γ_{1} is an epimorphic image of Γ, then Γ is a non-trivial free product with amalgamation too.

Thus one can assume that

R(a, b) = a^{u_{1}}b^{v_{1}} … a^{u_{s}}b^{v_{s}},

where v_{i} ∈ {−1, 1}, i = 1, …, s. Suppose for a moment that there exists i, 1 ≤ i ≤ s, such that either v_{i} = v_{i+1} or v_{1} = v_{s}. For definiteness, let v_{1} = v_{2}. In this case one can consider the new generators of the group Γ: a_{1} = a, b_{1} = a^{u_{2}}b^{v_{1}}. Then it is easy to check that Γ = ⟨a_{1}, b_{1} | a_{1}^{n} = R_{1}^{m}(a_{1}, b_{1}) = 1⟩, where R_{1}(a_{1}, b_{1}) = a_{1}^{u′_{1}}b_{1}^{v′_{1}} … A_{1}^{u′_{l}}b_{1}^{v′_{l}}, 0 < u′_{i} < n, v′_{i} ≠ 0 for i = 1, …, l. In addition, we have v′ = max_{1≤i≤l} |v′_{i}| ≥ 2. But we have proved just above that in this case the group Γ is a non-trivial free product with amalgamation.

Thus without loss of generality we can assume that

R(a, b) = a^{u_{1}}ba^{u_{2}}b^{−1} … a^{u_{2k−1}}ba^{u_{2k}}b^{−1},

where k ≥ 1, 0 < u_{i} < n for i = 1, …, 2k. Set c = ba^{−1}b^{−1}. Then

R(a, b) = a^{u_{1}}c^{−u_{2}} … a^{u_{2k−1}}c^{−u_{2k}} = R_{1}(a, c).

Let F_{2} = ⟨g, h⟩ be the free group of rank 2, f = hg^{−1}h^{−1}. Set x = τ_{g}, y = τ_{h}, z = τ_{gh}, t = τ_{gf}. Then τ_{f} = τ_{g} = x, t = τ_{gf} = τ_{ghg^{−1}h^{−1}} = x^{2} + y^{2} + z^{2} − xyz − 2 by item 1 of Lemma 9. Consider the element R_{1}(g, f) ∈ F_{2} as a word of g and f. Let q(x, t) be the Fricke polynomial of R_{1}(g, f), that is,

Q(x, t) = Q_{R_{1}(g,f)}(τ_{g}, τ_{f}, τ_{gf}) = Q_{R_{1}(g,f)}(x, x, t).

Since R_{1}(g, f) contains k blocks of the form g^{u_{j}}f^{−u_{j+1}}, then by Lemma 5 the degree of the polynomial q(x, t) with respect to t is equal to k and the leading coefficient of q(x, t) is equal to (−1)^{k} ∏_{i=1}^{2k} P_{u_{i}−1}(x). Since by construction R(g, h) = R_{1}(g, f), then

Q_{R(g,h)}(x, y, z) = q(x, t) = q(x, x^{2} + y^{2} + z^{2} − xyz − 2).  (22)

Set now x = τ_{g} = α_{r} = 2 cos(rπ/n), γ_{l} = 2 cos(lπ/m), where r ≢ 0(mod n), l ≢ 0 (mod m), and consider the equation

Q_{R(g,h)}(α_{r}, y, z) = γ_{l}.  (23)

By (22) one can write the equation (23) in the form

q(α_{r}, t) = γ_{l}.  (24)

Lemma 10 There exist r, l ∈ ℤ, where r ≢ 0(mod n) and l ≢ 0(mod m), such that P_{u_{i}−1}(α_{r}) ≠ 0 for i = 1, …, 2k and the equation (24) has a root t = t_{0} ≠ 2.

Proof. First let m ≥ 3. In this case γ_{1} ≠ γ_{2}. Set r = 1. Then the degree of the polynomial q(α_{1}, t) is equal to k. It is evident that at least one of the equations q(α_{1}, t) = γ_{1}, q(α_{1}, t) = γ_{2} has a root t_{0} ≠ 2.

Now we assume that m = 2. Suppose that the equation q(α_{r}, t) = 0 has a unique root t = 2. This means that for arbitrary matrices A, B ∈ SL_{2}(ℂ) such that tr A = tr B = α_{r}, the condition tr R_{1}(A, B) = tr A^{u_{1}}B^{−u_{2}} … A^{u_{2k−1}}B^{−u_{2k}} = 0 implies that tr AB = 2. Prove that this is not the case. To obtain a contradiction, it is sufficient to find matrices A, B ∈ SL_{2}(ℂ) satisfying to the conditions:

1) tr A = tr B = α_{r};

2) tr AB ≠ 2;

3) tr R_{1}(A, B) = tr A^{u_{1}}B^{−u_{2}} … A^{u_{2k−1}}B^{−u_{2k}} = 0.

We shall find A, B in the form

A = ( \begin{matrix}ε_{r} & w \\ 0 & ε_{r}^{−1} \end{matrix} ) , B = ( \begin{matrix}ε_{r} & 0 \\ w & ε_{r}^{−1} \end{matrix} ) ,

where ε_{r} + ε_{r}^{−1} = α_{r} = 2 cos(rπ/n) and w is a variable. It is easy to see that tr AB = w^{2} + ε_{r}^{2} + ε_{r}^{−2}. Hence the condition tr AB ≠ 2 is equivalent to w^{2} + ε_{r}^{2} + ε_{r}^{−2} ≠ 2, that is,

w^{2} ≠ 2 − (ε_{r}^{2} + ε_{r}^{−2}) = 2 − 2 cos(\frac{2rπ}{n}) = 4 sin^{2}(\frac{rπ}{n}).

It is easy to verify by induction that

A^{i} = ( \begin{matrix}ε_{r}^{i} & P_{i−1}(α_{r})w \\ 0 & ε_{r}^{−i} \end{matrix} ) , B^{i} = ( \begin{matrix}ε_{r}^{i} & 0 \\ P_{i−1}(α_{r})w & ε_{r}^{−i} \end{matrix} ) .

Furthermore, it is easy to show that

R_{1}(A, B) = ( \begin{matrix}ε_{r}^{d} + C_{1}(α_{r})w^{2} + ⋯ + C_{k}(α_{r})w^{2k} & wf_{1}(w) \\ wf_{2}(w) & ε_{r}^{−d} + D_{1}(α_{r})w^{2} + ⋯ + D_{k−1}(α_{r})w^{2k−2} \end{matrix})

where d = ∑_{i=1}^{2k} u_{i}, C_{k}(α_{r}) = (−1)^{k} ∏_{i=1}^{2k} P_{u_{i}−1}(α_{r}), f_{1}(w), f_{2}(w) are some polynomials of w. Hence,

tr R_{1}(A, B) = C_{k}(α_{r})w^{2k} + ⋯ + (C_{1}(α_{r}) + D_{1}(α_{r}))w^{2} + (ε_{r}^{d} + ε_{r}^{−d}) = g(w^{2}).

Prove that there exists r, 1 ≤ r < n, such that C_{k}(α_{r}) ≠ 0 and the polynomial g(w^{2}) has a root w_{0} with w_{0}^{2} ≠ 4 sin^{2}(rπ/n). Let’s assume the contrary. Suppose that for each r such that C_{k}(α_{r}) ≠ 0 we have

g(w) = C_{k}(α_{r})(w − 4 sin^{2}(\frac{rπ}{n}))^{k}.  (25)

Comparing the constant terms in the left and right part of (25) and taking into account the expression for C_{k}(α_{r}), we obtain

(∏_{i=1}^{2k} P_{u_{i}−1}(2 cos(\frac{rπ}{n}))) 4^{k} (sin(\frac{rπ}{n}))^{2k} = 2 cos(\frac{drπ}{n}).  (26)

By (4) we have P_{u_{i}−1}(2 cos(rπ/n)) = sin(u_{i}rπ/n)/ sin(rπ/n). Write u_{i}/n in the form u′_{i}/n_{i}, where (u′_{i}, n_{i}) = 1. Then (26) has the form

∏_{i=1}^{2k} (2 sin (\frac{u′_{i}rπ}{n_{i}})) = 2 cos(\frac{drπ}{n}).  (27)

Thus, to complete the proof of the lemma it is sufficient to prove that one gets a contradiction assuming that (27) holds for each r such that the left part in (27) is non-zero.

First consider the case, where n is odd. Let n_{0} = min_{j} n_{j} and let for definiteness n_{0} = n_{1}. Then n_{1} is odd and let p > 2 be an arbitrary prime divisor of n_{1}. Set r = n_{1}/p. Then 2 sin(u′_{1}rπ/n_{1}) = 2 sin(u′1π/p). If j > 1, then 2 sin(u′_{j}rπ/n_{j}) = 2 sin(u′_{j}n_{1}π/(pn_{j})) ≠ 0 because by construction pn_{j} does not divide u′_{j}n_{1}. It follows from (27) that

∏_{i=2}^{2k} (2 sin (\frac{u′_{i}n_{1}π}{pn_{i}})) = \frac{2 cos(dn_{1}π/(pn))}{sin(u′_{1}π/p)} ∈ O.  (28)

If pn divides dn_{1}, then the equality (28) implies that 1/(sin(u′_{1}π/p)) ∈ O which contradicts to item 5 of Lemma 7. Further if pn does not divide dn_{1}, then 2 cos(dn_{1}π/(pn)) ∈ O^{∗} by item 2 of Lemma 7. Thus it follows from (28) that 1/(2 sin(u′_{1}π/p)) ∈ O which again contradicts to item 5 of Lemma 7.

Let now n = 2^{l}n′, where l ≥ 1 and n′ is odd. Let n_{i} = 2^{l_{i}}n′_{i}, where l_{i} ≥ 0, n′_{i} is odd, and let n′_{0} = min_{j} n′_{j}.

If n′_{0} > 1, then we set r = 2^{l}r′, where r′ ≢ 0(mod n′). Then (27) has the form

∏_{i=1}^{2k} (2 sin (\frac{u′_{i}2^{l−l_{i}}r′π}{n′_{i}})) = 2 cos(\frac{dr′π}{n′}),  (29)

where n′ is odd. We have proved above that there exists an r′ such that the left part of (29) is non-zero and the equality (29) is not valid.

Let now n′_{0} = 1. Set

I = {i | n′_{i} = 1},  l_{0} = min_{i∈I} l_{i},  I_{0} = {i ∈ I | l_{i} = l_{0}}.

Furthermore, we set r = 2^{l_{0}−1}r′, where r′ is odd. Then for i ∈ I_{0} we have

2 sin (\frac{u′_{i}rπ}{n_{i}}) = 2 sin (\frac{u′_{i}2^{l_{0}−1}r′π}{2^{l_{0}}}) = 2 sin (\frac{u′_{i}r′π}{2}) = ±2.

Now we can write the equality (27) in the form:

∏_{i∉I_{0}} (2 sin (\frac{u′_{i}r′π}{2^{l_{i}−l_{0}+1}n′_{i}} )) = ±\frac{1}{2^{|I_{0}|−1}} cos(\frac{dr′π}{2^{l−l_{0}+1}n′}).  (30)

Choose r′ such that the left part of (30) is non-zero. Then the right part of (30) should be non-zero too. If |I_{0}| > 1 or |I_{0}| = 1 and cos(dr′π/(2^{l−l_{0}+1}n′)) ≠ ±1, then the left part of (30) belongs to O. But by item 1) of Lemma 7 the right part of (30) does not belong to O which is a contradiction.

Thus, it remains to consider the case |I_{0}| = 1 and cos(dr′π/(2^{l−l_{0}+1}n′)) = ±1. In this case (30) has the form

∏_{i≠i_{0}} (2 sin (\frac{u′_{i}r′π}{2^{l_{i}−l_{0}+1}n′_{i}} )) = ±1.  (31)

If |I| > 1 and i_{0} ≠ i ∈ I, then l_{i} > l_{0}, n_{i} = 1. Hence for each odd r′ the left part of (31) is non-zero and (31) implies that 1/(2 sin(u′_{i}r′π/(2^{l_{i}−l_{0}+1}))) ∈ O. We obtain a contradiction to item 5 of Lemma 7.

Let now I = I_{0} = {i_{0}}. Set

n_{j_{0}} = min_{j≠i_{0}} n_{j} ≥ 3,  J = {j | n_{j} = n_{j_{0}}},  l_{j_{0}} = min_{j∈J} l_{j}.

If l_{j_{0}} − l_{0} + 1 > 0, then we set r′ = n_{j_{0}}. It is easy to verify that in this case the left part of (31) is not equal to 0 and (31) implies that 1/(2 sin(u′_{j_{0}}π/2^{l_{j_{0}}−l_{0}+1})) ∈ O. We obtain a contradiction to item 6 of Lemma 7.

At last, if l_{j_{0}} − l_{0} + 1 ≤ 0, then we take an arbitrary prime divisor p ≥ 3 of n_{j_{0}} and set r′ = n_{j_{0}}/p. Then, as above, the left part of (31) is not equal to 0 and we obtain from (31) that

2 sin (\frac{u′_{j_{0}}r′π}{2^{l_{j_{0}} −l_{0}+1}n′_{j_{0}}} ) = 2 sin (^{\frac{u′_{j_{0}}2^{−l_{j_{0}} +l_{0}−1}π}{p}}) ∈ O^{∗}.

We have a contradiction to item 5 of Lemma 7. Lemma 10 is proved.

Now we can complete the proof of Theorem 3. By Lemma 10 one can choose r, l such that the equation (24) has a root t_{0} ≠ 2. Since by construction t = x^{2} +y^{2} +z^{2} −xyz −2, x = α_{r}, then we have that y, z satisfy to the equation

y^{2} + z^{2} − α_{r}yz + α_{r}^{2} − 2 − t_{0} = 0.  (32)

Let (y_{0}, z_{0}) be some solution of (32) and let A, B ∈ SL_{2}(ℂ) be matrices such that tr A = α_{r}, tr B = y_{0}, tr AB = z_{0}. Then by construction tr ABA^{−1}B^{−1} = t_{0}, tr R(A, B) = γ_{l}, and a pare of matrices (A, B) defines a representation of the group Γ into PSL_{2}(ℂ). Note that this representation is irreducible because t_{0} ≠ 2. Our goal is to apply Lemma 6. To this end, it is sufficient to find a solution (y_{0}, z_{0}) of the equation (32) such that:

1) there exists an element of finite order W_{1}(A, B) = A^{α_{1}}B^{β_{1}} … A^{α_{g}}B^{β_{g}} such that α_{i}, β_{i} ≠ 0 for i = 1, …, g and ∑_{i=1}^{g} β_{g} ≠ 0;

2) z_{0} = tr AB ∉ O. The rest of the proof depends on the form of t_{0}. We consider the following cases:

1) t_{0} ∉ O;

2) t_{0} = 2 cos((2k + 1)π/(2s + 1)), where s ≥ 1, (2k + 1, 2s + 1) = 1;

3) t_{0} = 2 cos(2kπ/(2s + 1)), where s ≥ 1, (k, 2s + 1) = 1;

4) t_{0} = 2 cos((2k + 1)π/(2s)), where s ≥ 1, (2k + 1, s) = 1;

5) t_{0} ∈ O, t_{0} ≠ 2 cos(kπ/s) for arbitrary integers k and s.

1) Set y_{0} = 0 and W_{1}(A, B) = B. Then W_{1}(A, B) = B has order 4. Since t_{0} ∉ O, then the equation (32) has a solution (0, z_{0}) such that z_{0} ∉ O. Applying Lemma 6, we obtain the assertion of Theorem 3.

2) Set W_{1}(A, B) = AB(ABA^{−1}B^{−1})^{s}. Combining Lemmas 8 and 9, we obtain

tr W_{1}(A, B) = (P_{s+1}(t_{0}) − P_{s}(t_{0}))z_{0} = 0 · z_{0} = 0.

Hence W_{1}(A, B) has order 4. Take an arbitrary solution (y_{0}, z_{0}) of the equation (32) such that z_{0} ∉ O. Again we can apply Lemma 6.

3) Set W_{1}(A, B) = AB(ABA^{−1}B^{−1})^{s} and assume that

tr W_{1}(A, B) = 2 cos(π/3) = 1.

Then W_{1}(A, B) has order 6 and item 2 of Lemma 9 implies that

tr W_{1}(A, B) = (P_{s+1}(t_{0}) − P_{s}(t_{0}))z_{0} = 1.

Hence by item 3 of Lemma 8 we have z_{0} = 1/(P_{s+1}(t_{0}) − P_{s}(t_{0})) ∉ O. Let (y_{0}, z_{0}) be a solution of (32). Now we can apply Lemma 6.

4) Set W_{1}(A, B) = (AB)^{−1}(ABA^{−1}B^{−1})^{s}(AB)^{2}(ABA^{−1}B^{−1})^{s} and assume that

tr W_{1}(A, B) = 2 cos(π/3) = 1.  (33)

Then W_{1}(A, B) has the order 6 and by item 3 of Lemma 9 we can write (33) in the form

(t_{0} − 2)P_{s−1}(t_{0})^{2}z_{0}^{3} + (2 − P_{2s−1}(t_{0}) + P_{2s−2}(t_{0}))z_{0} − 1 = 0.  (34)

By item 4 of Lemma 8 we have 0 ≠ P_{s−1}(t_{0}) ∉ O^{∗} whence 1/((t_{0} − 2)P_{s−1}(t_{0})^{2}) ∉ O. Thus, (34) has a root z_{0} ∉ O. Let (y_{0}, z_{0}) be a solution of (32). Applying Lemma 6, we obtain the assertion of Theorem 3.

5) Since t_{0} ∈ O, t_{0} ≠ 2 cos(kπ/s) for arbitrary integers k and s, then by item 5 of Lemma 8 there exists an integer l > 0 such that 0 ≠ P_{l}(t_{0}) ∉ O^{∗}. Set W_{1}(A, B) = (AB)^{−1}(ABA^{−1}B^{−1})^{l+1}(AB)^{2}(ABA^{−1}B^{−1})^{l+1} and assume that the condition (33) holds. Then W_{1}(A, B) has the order 6 and by item 3 of Lemma 9 we can write (33) in the form

(t_{0} − 2)P_{l}(t_{0})^{2}z_{0}^{3} + (2 − P_{2l+1}(t_{0}) + P_{2l}(t_{0}))z_{0} − 1 = 0.  (35)

Since by construction 1/((t_{0} − 2)P_{l}(t_{0})^{2}) ∉ O, then (35) has a root z_{0} ∉ O. Let (y_{0}, z_{0}) be a solution of (32). Applying Lemma 6, we complete the proof of Theorem 3 in the last case.

References

[1] R.C. Lyndon and P.E. Schupp, “Combinatorial group theory”, Springer-Verlag, Berlin–Heidelberg–New York, 1977.

[2] C.T.C. Wall (editor), “Homological group theory” (Proceedings of a conference in Durham), London Math. Soc. Lecture Notes. 36 (1977).

[3] G. Baumslag, and P.B. Shalen,“ Amalgamated products and finitely presented groups”, Comment. Math. Helv. 65 (1990), 243–254.

[4] B. Fine, F. Levin, and G. Rosenberger, “Free subgroups and decompositions of one-relator products of cyclics; Part 2: Normal torsion free subgroups and FPA decompositions”, J. of Indian Math. Soc. 49 (1985), 237–247.

[5] H. Zieschang, “On decompositions of discontinuous groups of the plane”, Math. Zeit. 151 (1976), 165–188.

[6] G. Rosenberger, “Bemerkungen zu einer Arbeit von H. Zieschang”, Arch. der Math. 29 (1977), 623–627.

[7] D.D. Long, C. Maclachlan, and A.W. Reid, “Splitting groups of signature (1, n)”, J. of Algebra. 185 (1996), 329–341.

[8] M.J. Dunwoody, and M. Sageev, “Splittings of certain Fuchsian groups”, Proc. of the Amer. Math. Soc. 125:7 (1997), 1953–1954.

[9] V.V. Benyash-Krivets, “Decomposing one-relator products of cyclic groups into free products with amalgamation”, Sbornik: Mathematics, 189:8 (1998), 1125–1137.

[10] V.V. Benyash-Krivets, “On decomposing some F -groups into free products with amalgamation”, Dokl. Akad. Nauk Belarus , 41:6 (1997), 1–4.

[11] A. Lubotzky, and A. Magid, “Varieties of representations of finitely generated groups”, Memoirs AMS. 58 (1985), 1–116.

[12] M. Culler, and P. Shalen, “Varieties of group representations and splittings of 3 manifolds”, Ann. of Math. 117 (1983), 109–147.

[13] A.S. Rapinchuk, V.V. Benyash-Krivetz, and V.I. Chernousov, “Representation varieties of the fundamental groups of compact orientable surfaces”, Israel J. Math. 93 (1996), 29–71.

[14] V.V. Benyash-Krivetz and V.I. Chernousov, “Representation varieties of the fundamental groups of compact non-orientable surfaces”, Sbornik: Mathematics, 188:7 (1997), 997–1039.

[15] D. Mumford, “Geometric invariant theory”, Springer-Verlag, Berlin–Heidelberg– New York, 1965.

[16] R. Horowitz, “Characters of free groups represented in the two dimensional linear group”, Comm. Pure. Appl. Math. 25 (1972), 635–649.

[17] H. Helling, “Diskrete Untergruppen von SL_{2}(ℝ)”, Invent. Math. 17 (1972), 217– 229.

[18] W. Magnus, “The uses of 2 by 2 matrices in combinatorial group theory”, Result. der Math. 4:2 (1981), 171–192.

[19] J.-P. Serr, “Arbres, amalgames, SL_{2}”, Astérisque , 42 (1977).

[20] E.B. Vinberg, “Rings of definotionof dense subgroups of semisimple linear groups”, Math. USSR Izvestiya, 5:1 (1971), 45–55.

[21] I.R. Shafarevich, “Basic algebraic geometry”, Springer-Verlag, Berlin–Heidelberg– New York, 1974.

[22] C. Traina, “Trace polynomial for two generated subgroups of SL_{2}(ℂ)”, Proc. Amer. Math. Soc. 79 (1980), 369–372.

[23] H. Bass, “Finitely generated subgroups of GL_{2}(ℂ)”, in “The Smith Conjecture”, Wiley, New York, 1984, pp. 127–136.