{'fraction_non_alphanumeric': 0.05675015564388679, 'fraction_numerical': 0.014846032278147599, 'mean_word_length': 4.018505167027157, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 18, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Conical averagedness and convergence analysis of fixed point algorithms

In this paper, we consider a conical extension of averaged nonexpansive operators and its usefulness in the convergence analysis of fixed point algorithms. Various properties of conically averaged operators are systematically studied, in particular, the stability under relaxations, convex combinations, and compositions. We then derive the conical averagedness of the resolvents of operators that possess certain types of generalized monotonicity. These properties are used to analyze the convergence of fixed point algorithms including the proximal point algorithm, the forward-backward algorithm, and the adaptive Douglas-Rachford algorithm. Our results not only unify and improve recent studies but also provide a different perspective on the topic.


Introduction
Averaged nonexpansive operators were originally introduced by [1] and they are well known to be useful in convergence theories of various fixed point algorithms, see [2,4,6,8,20] and the references therein. Particularly, iterative sequences generated by many fixed point algorithms can be expressed in terms of Krasnosel'skiȋ-Mann iterations whose convergence relies on averagedness property. Although frequently understood in the single-valued setting, some averaged nonexpansive structure can also be explored in the set-valued framework. For instance, the notion of union averaged nonexpansive operators has been recently studied in [11] with applications to the local convergence of proximal algorithms.
Apparently, each averaged operator is an under-relaxation of some nonexpansive operator. As we will see later on, over-relaxations of nonexpansive operators also arise in several situations. In this paper, we consider the conically averaged operators that unify both types of relaxations for nonexpansive operators. We then show that this class of operators plays a significant role in the convergence of several fixed point algorithms, in particular, the proximal point algorithm, the forward-backward algorithm, and the adaptive Douglas-Rachford (DR) algorithm.
The proximal point algorithm was first introduced by Martinet [18] and further studied by Rockafellar [22] for finding a zero of a maximally monotone operator. Since then, it has become an indispensable component of optimization theory and applications. In fact, several other iterative optimization algorithms can be reformulated as special cases of the proximal point algorithm, see [13] and the references therein. The forward-backward algorithm, on the other hand, was first proposed by Lions and Mercier [17] and Passty [21] for finding a zero of the sum of two maximally monotone operators. Indeed, this splitting idea can be traced back to the projected gradient method [15]. Another famous splitting algorithm is the DR algorithm, which was initially studied by Douglas and Rachford [12] in the setting of linear operators, and was later generalized for maximally monotone operators by Lions and Mercier also in [17]. It is worth to mention that both forward-backward and DR algorithms reduce to the proximal point algorithm when one operator is zero. Recently, the so-called adaptive DR algorithm has been proposed in [9] to deal with the lack of the classical monotonicity assumption.
The main contributions of the paper are two folds. On the one hand, we systematically study the conical averagedness property, which is stable under relaxations, convex combinations, and compositions. It is also showed that the relaxed resolvents of generalized monotone operators either belong to or directly link to the class of conically averaged operators. On the other hand, we analyze the convergence of all the aforementioned algorithms by means of the conical averagedness. This approach not only provides simple convergence proofs but also brings more flexibility for the algorithm parameters. In particular, we prove the global convergence and the rate of asymptotic regularity of the relaxed proximal point algorithm, the relaxed forward-backward algorithm, and the adaptive DR algorithm when one operator is no longer monotone. An application to the strongly and weakly convex optimization problem is also discussed. Our analysis also improves several contemporary results on this topic.
The remainder of the paper is organized as follows. Section 2 presents the conically averaged operators with numerous interesting properties that are beneficial to our analysis. In Section 3, we study the relaxed resolvents of generalized monotone operators in connection with conical averagedness properties. Based on these developments, in Sections 4 and 5, we provide our main results on the averagedness of operators associated with those algorithms, which leads to the convergence and the rate of asymptotic regularity. Finally, we warp up the paper in Section 6.
The notations in the paper are fairly standard and follow largely [2]. Throughout, X is a real Hilbert space with inner product ·, · and induced norm · . The set of nonnegative integers is denoted by N, the set of real numbers by R, the set of nonnegative real numbers by R + := {x ∈ R x ≥ 0}, and the set of the positive real numbers by R ++ := {x ∈ R x > 0}. We use the notation A : X ⇒ X to indicate that A is a set-valued operator on X and the notation A : X → X to indicate that A is a single-valued operator on X. Given an operator A on X, its domain is denoted by dom A := {x ∈ X Ax = ∅}, its set of zeros by zer A := {x ∈ X 0 ∈ Ax}, and its set of fixed points by Fix A := {x ∈ X x ∈ Ax}. As usual, Id denotes the identity operator.
We will frequently use the following identity, for every σ, τ ∈ R and s, t ∈ X, and if σ + τ = 0, then

Conically averaged operators
We recall that T : X → X is nonexpansive if it is Lipschitz continuous with constant 1 on its domain, i.e., ∀x, y ∈ dom T, The operator T is said to be θ-averaged if θ ∈ ]0, 1[ and T = (1 − θ) Id +θN for some nonexpansive operator N : X → X, see, e.g., [2,Definition 4.33]. We now extend this concept to allow for θ ∈ R ++ . Definition 2.1 (conically averaged operator). We say that an operator T : X → X is conically averaged with constant θ ∈ R ++ , or conically θ-averaged, if there exists a nonexpansive operator N : Let T be conically θ-averaged. Then T is nonexpansive when θ = 1, and T is θ-averaged when θ ∈ ]0, 1[. As one would expect, conically averaged operators also possess properties similar to averaged operators. Indeed, we now present numerous properties which generalize and extend the corresponding ones in [2,Chapter 4], see also [3] for a related development where conically averaged operators were referred to as conically nonexpansive operators.

Proposition 2.2.
Let T : X → X, θ ∈ R ++ , and λ ∈ R ++ . Then the following are equivalent: (iv) For all x, y ∈ dom T , Proof. Set N : which implies the equivalence between (i) and (ii).
Next, we note that Id −N = (Id −T )/θ and dom T = dom N =: D. Now using (1), we have for all x, y ∈ D that Therefore, and we get the equivalence between (i), (iii), and (iv). The proof is complete.
In view of Proposition 2.2, T is 1/2-averaged if and only if, for all x, y ∈ dom T , in which case, T is also said to be firmly nonexpansive.
The following results reiterate and extend the corresponding results for nonexpansive operators and averaged operators.

Proposition 2.4 (convex combination).
Let I be a finite index set, let T i : X → X be conically θ i -averaged for each i ∈ I, and let {w i } i∈I ⊆ R ++ be such that i∈I ω i = 1.
Proof. For each i ∈ I, there exists a nonexpansive operator N i such that As i∈I ω i θ i θ N i is a nonexpansive operator due to [2, Proposition 4.9(i)], the proof is complete.
(ii) T is nonexpansive if and only if T 1 and T 2 are nonexpansive.
(iii) T is averaged if and only if T 1 and T 2 are averaged.
For convenience in later applications, we derive an equivalent presentation of Proposition 2.5(i). This result generalizes [2,Propsition 4.44] and interestingly, also recaptures [14,Proposition 3.12], in which the composition of an averaged operator and a so-called negatively averaged operator was considered.
Proof. (i): The proof is straightforward by using Proposition 2.5 repeatedly.
Since the statement is true for m − 1 conically averaged operators, we have is θ * -averaged with Using (24) with k = m, we have Now apply Proposition 2.5 to two operators T * and T m , It follows that So, the statement is true for m. By the mathematical induction principle, the statement is true for all m ≥ 2.
which fulfills (22). The conclusion then follows by applying (ii). On the other hand, it expands these results beyond the usual composition of conically averaged operators by applying scalar multiplications on each of these operators, see (21).
We conclude this section by establishing the convergence of sequences generated by an averaged/conically averaged operator using Fejér monotonicity. Recall that a sequence (x n ) n∈N is said to be Fejér monotone with respect to a nonempty subset of C of X if The following result whose proof is included for completeness is a slight extension of [2, Theorem 5.15].

Proposition 2.9 (Krasnosel'skiȋ-Mann iterations). Let T be a conically θ-averaged operator with full domain and Fix
where (λ n ) n∈N is a sequence in [0, 1/θ] such that +∞ n=0 λ n (1 − θλ n ) = +∞. Then the following hold: Proof. By definition, T = (1 − θ) Id +θN for some nonexpansive operator N : (i): For all y ∈ Fix N and all n ∈ N, using (1) yields Moreover, for all n ∈ N, since x n+1 = (1 − θλ n )x n + θλ n N x n and since N is nonexpansive, We deduce that x n − N x n n∈N is decreasing and bounded below by 0, hence it converges. Combining with (38) gives (iii): Let x * be a weak cluster point of (x n ) n∈N . Then there exists a subsequence (x kn ) n∈N such that x kn ⇀ x * . By (40) and [2,Corollary 4.28], x * ∈ Fix N . In turn, [2, Theorem 5.5] implies that (x n ) n∈N converges weakly to a point in Fix N = Fix T .

Corollary 2.10 (convergence of averaged operators). Let T be a θ-averaged operator with full domain and
Fix T = ∅. Let (x n ) n∈N be a sequence generated by T . Then (x n ) n∈N converges weakly to a point in Fix T and the rate of asymptotic regularity of as n → +∞.

Generalized monotonicity
and the relaxed resolvent of A with parameter λ ∈ R + is defined by and α-comonotone [3] if We say that A is maximally α-monotone (respectively, maximally α-comonotone) if it is αmonotone (respectively, α-monotone) and there is no α-monotone (respectively, α-monotone) operator B : X ⇒ X such that gra B strictly contains gra A.

(i) A is maximally α-monotone if and only if A is α-monotone and maximally monotone.
(ii) A is maximally α-comonotone if and only if A is α-comonotone and maximally monotone.
We now collect important and useful properties of relaxed resolvents of α-monotone and αcomonotone operators. Although part of the results appeared in [2,3,9,10], we include here for the readers' convenience. In particular, we will show that if an operator is either α-monotone or αcomonotone, then its relaxed resolvents are, to some extent, related to conically averaged operators. These results are crucial in the convergence analysis of several iterative algorithms that make use of the resolvents. We begin with some auxiliary properties.

Lemma 3.6 (metric inequalities for resolvents of α-comonotone operators). Let
A : X ⇒ X and let γ ∈ R ++ . Then A is α-comonotone if and only if, for all (x, a), (y, b) ∈ gra J γA , Consequently, if A is α-comonotone and J γA is single-valued, then, for all x, y ∈ dom J γA , Proof. Let (a, u), (b, v) ∈ gra A and set x := a + γu and y := b + γv. We have the equivalence The conclusion then follows.

Proposition 3.7 (single-valuedness and full domain). Let
A : X ⇒ X be α-comonotone and let γ ∈ R ++ be such that γ + α > 0. Then the following hold: and that Therefore, We deduce that dom J γA = X if and only if dom J 1 γ+α A ′ , which, due to [2, Theorem 21.1 and Proposition 20.22], happens if and only if A ′ is maximally monotone. In turn, it is clear that the maximal monotonicity of A ′ is equivalent to the maximal α-comonotonicity of A.

2(i)&(ii).
In the sequel, we will present the connection between the conical averagedness and several fixed point algorithms including the forward-backward algorithm and the adaptive Douglas-Rachford algorithm.

Relaxed forward-backward algorithm
In this section, we let A : X ⇒ X, B : X → X, and consider the problem find x ∈ X such that 0 ∈ Ax + Bx.
When an iterative fixed point algorithm is used to solve a problem, an important concern is the connection between the fixed points of that algorithm and the solutions of the given problem. For completeness, we include the proof of the following known result, see, e.g., [2, Proposition 26.1(iv)(a)], which states that the fixed points of the forward-backward algorithm actually solve (55).

Remark 4.2.
In the light of Lemma 4.1 and Corollary 2.10, if T FB is averaged, then, as soon as zer(A + B) = ∅, we deduce that every sequence (x n ) n∈N generated by T FB converges weakly to a point in Fix T FB = zer(A + B) and that the rate of asymptotic regularity of T FB is o(1/ √ n).
Therefore, we will only investigate the averagedness of the operator T FB .
When B = 0, problem (55) reduces to finding a zero of operator A : X ⇒ X, i.e., find x ∈ X such that 0 ∈ Ax (58) and the corresponding relaxed forward-backward algorithm reduces to the relaxed proximal point algorithm of the form In this case, we have from Lemma 4.1 that We arrive at the main results of this section, in which the averagedness of T PP and T FB is obtained when the operator A is not necessarily monotone. For classical results, we refer the reader to, e.g., [2, Example 23.40 and Proposition 26.1(iv)(d)].
Then T FB is κ-averaged and has full domain.
Then T FB is κ κ * -averaged and has full domain. Proof. Note that A is maximally 0-comonotone and then apply Theorem 4.5 with α = 0.
Given x 0 ∈ X, the adaptive DR algorithm (aDR) for (55) generates a sequence (x n ) n∈N , also called a DR sequence, according to ∀n ∈ N, x n+1 ∈ T x n .
As usual, we will refer to the case when δ = γ > 0 and λ = µ = 2 as the classical DR algorithm (or simply DR).
Unlike the forward-backward counterpart, the fixed points of the adaptive DR algorithm, in general, do not directly solve (55). Nevertheless, by choosing appropriate parameters, we are able to show that the images of the fixed points under the resolvent will do the job. For this purpose, as in [9], we make the standing assumption which is equivalent to and which clearly holds for classical DR algorithm. The following fact justifies our setting. Therefore, we only study the convergence of the adaptive DR algorithm to the fixed point set of the operator T under condition (74). In turn, such a convergence can be guaranteed by the averagedness property as shown in the next result. Based on these observations, we will focus on the averagedness of the adaptive DR operator. To achieve such goal, we need to find appropriate parameters γ, δ, λ, µ, κ. In fact, it suffices to determine only γ, δ, κ > 0, since the parameters λ, µ will be automatically defined by (74).

The case of αand β-monotone operators revisited
The convergence of the adaptive DR algorithm for α-and β-monotone operators was originally provided in [9]. In this section, we revisit part of the results using the conical averagedness. In comparison to [9], the new results (see Theorems 5.3 and 5.5) enlarge the acceptable range for the parameters γ, δ, λ, µ, and κ, which guarantees the averagedness of T , and hence, the convergence of the adaptive DR algorithm. In addition, the assumptions on the parameters are unified into a single condition, see, for example, (82). For clarity, we will split the main results of this section into two cases: α + β = 0 and α + β > 0. Theorem 5.3 (aDR for αand β-monotone operators with α + β = 0). Suppose that A and B are respectively maximally α-and β-monotone with α + β = 0, that γ, δ, κ ∈ R ++ satisfy and that λ, µ are given by (74). Then the adaptive DR operators T A,B and T B,A are both κ-averaged and have full domain.
For the case α + β > 0, we will need the following technical lemma.
Proof. We have that which implies that the three statements are equivalent.
To see the existence of γ and δ, we choose γ > 0 such that 1/γ > −α and then choose δ > 0 that satisfies the second condition in (iii). Theorem 5.5 (aDR for αand β-monotone operators with α + β > 0). Suppose that A and B are respectively maximally α-and β-monotone with α + β > 0, that γ, δ, κ ∈ R ++ satisfy and that λ, µ are given by (74). Then the adaptive DR operators T A,B and T B,A are both κ κ * -averaged and have full domain.
Next, (82) implies (83), and again by Lemma 5.4, we have 1 + γα > 0 and then also 1 + δβ > 0. On the one hand, by Proposition 3.4, J γA , J δB and hence T A,B and T B,A are single-valued and have full domain. On the other hand, Proposition 3.5 implies that 1 1−λ R 1 and 1 1−µ R 2 are respectively conically θ 1 -and θ 2 -averaged with Now, it follows from (83) that θ 1 θ 2 < 1. Thus, Proposition 2.6 implies that R 1 R 2 and R 2 R 1 are both conically θ-averaged with Finally, we derive from Proposition 2.2(i)&(ii) that T A,B = (1 − κ) Id +κR 2 R 1 and T B,A = (1 − κ) Id +κR 1 R 2 are both κ κ * -averaged. It is worth noting that Theorems 5.3 and 5.5 indeed present a different perspective on the convergence analysis of the adaptive DR algorithm for two generalized monotone operators. Moreover, they also improve recent results as presented in the next remark.
Then the DR operators T A,B and T B,A are both κ κ * -averaged and have full domain.
Remark 5.8 (DR for the case α + β > 0). Corollary 5.7 is essentially [9,Theorem 4.5(ii)]. In the standard setup, the DR operator is defined as a strict convex combination of Id and R 2 R 1 (or R 1 R 2 ), i.e., κ ∈ ]0, 1[. In Corollary 5.7, on the one hand, we note that when both operators A and B are strongly monotone (α > 0 and β > 0), the upper bound for κ is κ * = 1 + γ αβ α+β > 1, that means, κ can be chosen to be larger than 1 while still guarantees the convergence. On the other hand, if either A or B is weakly monotone (i.e., α < 0 or β < 0), then κ * < 1, that means, one needs to further restrict κ from the standard range ]0, 1[ in order to maintain the convergence.
In the rest of this section, we consider the adaptive DR algorithm for the problem of minimizing the sum of two functions. Let f : We say that f is convex if α = 0, strongly convex if α > 0, and weakly convex if α < 0. The proximity operator of a proper function f with parameter γ ∈ R ++ is the mapping Prox γf : X ⇒ X given by Now, we consider the (α, β)-convex minimization problem of the form where f and g are respectively α-and β-convex functions. This problem arises in several important applications, see [16]. The adaptive DR algorithm for (93) is based on the operators where The following analysis shows the connection between the proximal operators and resolvents of the subdifferentials and how the problem of finding zeros of sum of the operators can help in solving the minimization problems. Recall that the Fréchet subdifferential of f at x is given by For more discussion on various subdifferentials and related properties, we refer interested readers to the monograph [19].
(ii) Prox γf = J γ ∂f is single-valued and has full domain.
Then the DR operators T f,g and T g,f are both κ κ * -averaged and have full domain. Additionally, if zer( ∂f + ∂g) = ∅, then, for any T ∈ {T f,g , T g,f }, every sequence (x n ) n∈N generated by T converges weakly to a point x ∈ Fix T with Prox γf (x) ∈ zer( ∂f + ∂g) ⊆ argmin(f + g) and the rate of asymptotic regularity of T is o(1/ √ n).

The case of αand β-comonotone operators
In this section, we consider the adaptive DR operators for two comonotone operators. Particularly, we derive the convergence results by using the conical averagedness.
That is, R 1 and R 2 are nonexpansive, so are R 2 R 1 and R 1 R 2 . The conclusion then follows.
Analogous to Lemma 5.4, we also present the existence result for the parameters.
Consequently, there always exist γ, δ ∈ R ++ that satisfy the above two statements.
Then the adaptive DR operators T A,B and T B,A are both κ κ * -averaged and have full domain. Proof. Apply Theorem 5.14 with γ = δ and λ = µ = 2.

Conclusion
We have studied the conical averagedness and its characterizations, especially, the stability under relaxations, convex combinations, and compositions. We have also explored its connection to relaxed resolvents of generalized monotone operators. This property facilitates a new perspective on the convergence analysis of several fixed point algorithms including the proximal point, the forward-backward, and the adaptive DR algorithms.