{'fraction_non_alphanumeric': 0.040690627550894205, 'fraction_numerical': 0.011848517080169194, 'mean_word_length': 4.601773590134405, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 17, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Distributed Riemannian Optimization with Lazy Communication for Collaborative Geometric Estimation

We present the first distributed optimization algorithm with lazy communication for collaborative geometric estimation, the backbone of modern collaborative simultaneous localization and mapping (SLAM) and structure-from-motion (SfM) applications. Our method allows agents to cooperatively reconstruct a shared geometric model on a central server by fusing individual observations, but without the need to transmit potentially sensitive information about the agents themselves (such as their locations). Furthermore, to alleviate the burden of communication during iterative optimization, we design a set of communication triggering conditions that enable agents to selectively upload a targeted subset of local information that is useful to global optimization. Our approach thus achieves significant communication reduction with minimal impact on optimization performance. As our main theoretical contribution, we prove that our method converges to first-order critical points with a global sublinear convergence rate. Numerical evaluations on bundle adjustment problems from collaborative SLAM and SfM datasets show that our method performs competitively against existing distributed techniques, while achieving up to 78% total communication reduction.


I. INTRODUCTION
Geometric estimation, which refers to the task of estimating geometric models (e.g., poses and 3D structure) from multiple views, is a fundamental technology that underlies important robotic applications such as simultaneous localization and mapping (SLAM) and Structure-from-Motion (SfM). For emerging applications in multi-robot systems and mixed reality, collaborative geometric estimation enables multiple agents to build and use a shared geometric model (e.g., a large-scale 3D map). At the core of this process is a large-scale optimization that fuses measurements collected by all agents to produce a global geometric model.
Existing multi-agent systems often offload the aforementioned global optimization, such as bundle adjustment (BA), to a central server or base station [1]- [4]. However, as the number of robots or mission time increases, centralized optimization suffers from increasing problem size that eventually makes the server a computational bottleneck. Furthermore, centralized optimization usually requires agents to communicate private data (e.g., images or locations) to the server, which compromises privacy requirements in applications such as autonomous driving and mixed reality.
Distributed optimization provides a promising solution that addresses both scalability and privacy concerns by leveraging the local computational power of the agents. In a distributed architecture, agents collaboratively solve the underlying optimization problem by coordinating with the server or with each other directly. However, distributed systems often require more frequent communication than their centralized counterparts due to the iterative nature of most optimization algorithms. Furthermore, the amount of data communicated at each iteration often grows proportionally with the dimension of the shared model. For large models, this type of iterative communication can result in long delays under real-world communication networks. Consequently, existing distributed systems often use simpler formulations that require less communication (e.g., pose graph optimization [5]- [7]) or operate on computer clusters with highperformance communication [8].
In this work, we develop a communication-efficient algorithm for collaborative geometric estimation, which significantly reduces the burden of communication when performing distributed optimization on high-dimensional problems. The core idea behind our approach is lazy communication: instead of uploading all information at every iteration, agents selectively upload parts of their local information that have changed significantly from the past. While the main idea is intuitive, incorporating lazy communication in our applications raises a series of technical questions ranging from algorithm design to theoretical analysis of convergence that we address in this work.
Contributions. We propose a communication-efficient distributed Riemannian optimization algorithm for collaborative geometric estimation. To tackle the numerical poor conditioning associated with most real-world problems, we design a distributed method that performs approximate secondorder updates while simultaneously protecting the privacy of participating agents. Furthermore, we augment our basic method with lazy communication, which enables agents to only transmit the parts of their local information that satisfy certain communication triggering conditions, and hence significantly reduces overall communication. We prove that our final algorithm converges globally to first-order critical points with a global sublinear rate. Compared to related works that study lazy communication in distributed firstorder methods (e.g., [9]), our algorithm design and convergence analysis are significantly different and account for the employed second-order updates, the treatment of nonconvex manifold constraints, among other details (see Remark 1). We perform extensive evaluations on large-scale BA problems in collaborative SLAM and SfM scenarios, which are central to emerging multi-robot navigation and mixed reality applications. Results show that our algorithm achieves competitive performance compared to other state-of-the-art methods under the same communication architecture, while achieving up to 78% total communication reduction.

Preliminaries on Riemannian Optimization
For a smooth Riemannian manifold X , we denote the tangent space at x ∈ X as T x X . For two tangent vectors u 1 , u 2 ∈ T x X , the inner product is denoted as u 1 , u 2 x , and the corresponding norm is u x u, u x . In the rest of the paper, we drop the subscript x as it will be clear from context. Let M : T x1 X → T x2 X be a linear map between two tangent spaces. With a slight abuse of notation, we also use M to denote the matrix representation of this linear map under chosen bases of T x1 X and T x2 X . For u ∈ T x1 X , M u ∈ T x2 X denotes the result of applying M on u. Further, M denotes the operator norm of M with respect to the Riemannian metric. When M : T x X → T x X maps a tangent space to itself and is symmetric and positive definite, we define its associated inner product as u 1 , u 2 M u 1 , M u 2 with the corresponding norm u M u, u M . A retraction at x is a smooth map Retr x : T x X → X that preserves the first-order geometry of X . For a scalar function defined on the manifold f : X → R, we use grad f (x) ∈ T x X to denote its Riemannian gradient at x ∈ X . Intuitively, grad f (x) provides the direction of steepest ascent in the tangent space at x. The reader is referred to [10], [11] for a more rigorous treatment of Riemannian optimization.

II. RELATED WORK
Centralized geometric estimation is a well studied subject with off-the-shelf high-performance solvers available [12]- [14]. Recently, Zhang et al. [15] develop a centralized incremental solver for multi-robot SLAM. Meanwhile, distributed methods have gained increasing attention; see [16] for a recent survey. Cunningham et al. develop the pioneering work of DDF-SAM [17], [18], where agents use Gaussian elimination to exchange marginals over commonly observed landmarks. Later work build on similar idea and propose extensions such as consistent sparsification [19] and realtime operation on devices with limited resources [20]. Our proposed method employs a similar elimination technique, and furthermore supports lazy communication to achieve significant communication reduction. Another recent line of research investigates distributed pose graph SLAM [21]- [25]. However, in most cases these methods only estimate robots' poses and not the global map. Related work in computer vision considers solving large-scale SfM using distributed architectures. Earlier work proposes to use distributed conjugate gradients for multi-core BA [26]. More recently, researchers have proposed alternative algorithms based on Douglas-Rachford splitting [27] or alternating direction method of multipliers (ADMM) [8].
Communication efficiency has been a central theme in distributed optimization. Recently, this topic has gained increasing attention due to the success of federated learning [28]. Multiple techniques to achieve communication efficiency have been proposed, including the use of quantization [29] and distributed second-order methods [30]. In this work, we explore an alternative strategy based on lazy or eventtriggered communication, which has demonstrated impressive results [9]. The same idea has found successful applications in related areas such as decentralized control [31]. We develop lazy communication schemes for collaborative geometric estimation, which requires substantial innovations in algorithm design and theoretical analysis compared to existing work [9]; see Remark 1.

III. COLLABORATIVE GEOMETRIC ESTIMATION
We consider a scenario where N agents navigate in a common environment, and seek to collaboratively estimate a shared geometric model y ∈ Y. For this purpose, agents communicate with a central server, who is responsible for coordinating updates across the team. In practice, the server can be a lead agent or a base station. Motivated by most real-world applications, we assume that the shared model y consists of m smaller elements y = {y 1 , . . . , y m }, where each element y l ∈ Y l corresponds to a single geometric primitive. For instance, when y represents a point cloud map, each y l corresponds to a single 3D point. During navigation, each agent i ∈ [N ] {1, . . . , N } observes a subset of the shared model. In addition, agent i also maintains private variable x i ∈ X i , which can contain sensitive information such as the trajectory of this agent.
In this work, we focus on solving the maximum likelihood estimation (MLE) problem in the multi-agent scenario described above. The MLE formulation is very general and encompasses a wide range of robot perception problems [32]. Under the MLE formulation, local measurements collected by agent i induce a local cost function 1 f i : X i ×Y → R that is usually non-convex. Under the standard assumption that agents' measurements are corrupted by independent noise, the global MLE problem takes the following form.
In (1), we use x ∈ X to denote the concatenation of all private variables x i , i ∈ [N ]. Next, we present several motivational examples related to multi-robot navigation.
Example 1 (Collaborative Bundle Adjustment). Bundle adjustment (BA) [33] is a crucial building block of modern visual SLAM and SfM systems. In collaborative BA, agents jointly estimate a global map using local measurements collected by monocular cameras. Assuming known camera intrinsics, the private variable x i contains camera poses of agent i, i.e., The shared variable y consists of points in the global map, i.e., y = {y 1 , y 2 , . . . , y m } ∈ R 3×m . Under the standard Gaussian noise model, the local cost function is given by the sum of local squared reprojection errors, In (2), L ij ⊆ [m] denotes the set of points observed by agent i at pose T (i) j , π(·, ·) is the camera projection model, q (i) jl ∈ R 2 denotes noisy observation on the image plane, and w (i) jl > 0 is the corresponding measurement weight. Example 2 (Collaborative Point Cloud Registration). Multiple point cloud registration (e.g., [34]) is an important problem with robotic applications such as merging multiple point cloud maps or collaborative SLAM with range sensors. In this case, the private and shared variables are the same as Example 1. The local cost function is given by, where T j ) denote the rotation matrix and translation vector of the jth pose of agent i, and q (i) jl ∈ R 3 denotes noisy 3D observation in the local frame.
Example 3 (Collaborative Object-Based Pose Graph Optimization). In some applications, it suffices to produce an object-level map of the environment (e.g., [22]). In this case, the set of shared variables becomes y = {T 1 , T 2 , . . . , T m } ∈ SE(3) m , where T l is the pose of object l. The local cost function (using chordal distance) is given by, where T In this work, we focus on collaborative BA (Example 1) in our experimental validation (Sec. VI), due to its fundamental role in multi-robot visual SLAM [1]- [4]. However, we note that our approach extends beyond the above examples to many other multi-agent estimation problems that can be described with a factor graph [32]. In particular, these include other multi-view reconstruction problems that use alternative sensors or estimate other types of geometric primitives (e.g., planes, quadrics, and cuboids).

IV. PROPOSED ALGORITHM
In this section, we present our communication-efficient distributed algorithm for solving Problem 1. In Sec. IV-A, we develop the basic form of our method based on distributed approximate second-order updates. Similar to DDF-SAM [17], [18], in each iteration our method analytically eliminates the updates to private variables, which leads to more effective updates and also protects the privacy of participating agents. However, unlike DDF-SAM, our method avoids the transmission of dense matrices resulting from elimination, which makes it applicable to larger scale problems. Furthermore, in Sec. IV-B, we augment our basic method with lazy communication, which achieves significant communication reduction. Lastly, Sec. IV-C summarizes the discussion and presents the complete algorithm.

A. Distributed Update with Analytic Elimination
At each iteration, agents collaboratively compute an updated solution that decreases the global cost in Problem 1. To start, each agent i constructs a second-order approximation m i for its local cost f i , which is defined at the tangent space of the current iterate (x i , y). Intuitively, m i approximates the true local cost f i when perturbing x i and y on the tangent space. Formally, given tangent vectors In (5), g i grad f i (x i , y) is the local Riemannian gradient. The user-specified linear map M i 0 serves as an approximation of the local Riemannian Hessian, and is assumed to be symmetric and positive definite. For geometric estimation problems such as BA (2), we obtain the second-order approximation via the Riemannian Levenberg-Marquardt (LM) method [10,Chapter 8]. In this case, we have M i = J i J i + λI, where J i is the Jacobian of agent i's measurement residuals, and λ > 0 is a regularization parameter that ensures M i to be positive definite.
Given the local approximations m i , a second-order approximation of the global cost f is given by where we use u to denote the concatenation of local tangent vectors u i . Note that m can be expanded as, It can be verified that g = grad f (x, y) is the Riemannian gradient of the global objective. The linear map M in (6) is now an approximation of the global Riemannian Hessian.
More importantly, M is a block matrix with an arrowhead sparsity pattern, and its blocks are related to the blocks of M i in (5) as follows, In the proposed method, we seek to compute an update for all variables by approximately minimizing m. To proceed, we analytically eliminate private vector u from (6). Formally, define u (v) arg min u m(u, v) as the optimal private vector conditioned on the shared vector. Furthermore, define the reduced second-order approximation as h(v) m(u (v), v), which only involves the shared vector v. Both u (v) and h(v) admit closed-form expressions.
Lemma 1 (Reduced second-order approximation). For each agent i ∈ [N ], the corresponding optimal private vector is, Furthermore, h(v) has the closed-form expression, where vector w and matrix S are defined as, In the following, we refer to w in (10) and S in (11) as the reduced gradient and reduced Hessian, respectively. The analytic elimination technique presented above has been widely used to solve SLAM and BA [33], and is a special case of the variable projection approach to solve nonlinear least squares problem [35]. In the distributed setting, Lemma 1 suggests that the server can first aggregate w i and S i from all agents, and then minimize h(v) by computing v = −S −1 w. This type of approach has been proposed by DDF-SAM [17], [18]. Nevertheless, for large-scale problems such as BA, this approach is less suitable as it requires the communication of the S i matrices, which are generally dense and thus expensive to evaluate, store, and transmit.
To design a communication-efficient update, we instead resort to finding an approximate minimizer of h(v). In the following, let k denote the iteration number. We let our approximate minimizer of h(v) take the following form, where γ > 0 is a constant stepsize, and P k is a sparse matrix that approximates the inverse of the reduced Hessian S k . Viewing w k as the reduced gradient and P k as a preconditioner, we may interpret (12) as a single step of preconditioned Riemannian gradient descent. Note that our method is agnostic to the specific choice of preconditioners.
In the following, we use the block Jacobi preconditioner [36] due to its simplicity, where S k i,l is the l-th diagonal block of S k i . Note that each block l corresponds to a single element y l in the shared variable y (see Sec. III). With (12) and (13), agent i only needs to upload w k i and the diagonal blocks of S k i to the server. Furthermore, the server can easily compute P k as it only requires inverting a block-diagonal matrix.
Once the shared update v k is computed, we leverage Lemma 1 to compute the corresponding optimal secondorder update for each agent's private variable, Finally, we compute the updated estimates using retraction, (15) and the algorithm proceeds to the next iteration.

B. Incorporating Lazy Communication
In the method developed so far, at each iteration, agent i needs to upload w i and D i to the server. For larger problems, the resulting transmission can still become too expensive. In this subsection, we present a technique to further reduce communication. The core idea behind our approach is lazy communication: when some blocks of w i and D i do not change significantly from previous iterations, agent i simply skips the transmission of those blocks, and the server reuses values received at previous iterations for its computation. In the following, we describe this process in detail for the computation of preconditioner and the reduced gradient, respectively. Without loss of generality, we present our method from the perspective of agent i.
Lazy communication of preconditioner. Let k be the current iteration number. For each block l, let k < k be the last iteration when agent i uploads S k i,l to the server. 3 Using S k i,l , we can compute an approximation of S k i,l as, where T k ←k l is the matrix that represents a transporter [11, Sec. 10.5] from the tangent space at iteration k to iteration k , and T k←k l is its adjoint. Intuitively, transporters are needed to ensure that the approximation defined in (16) represents a valid linear map on the tangent space at iteration k. For matrix manifolds, a simple and computationally efficient transporter can be obtained from orthogonal projections to tangent spaces [11,Proposition 10.60]. Note that since (16) only uses past information, both the server and agent i can compute S k i,l without any communication. The above approximation leads to the following lazy communication scheme. First, agent i compares S k i,l and its approximate version S k i,l locally. Then, agent i only uploads S k i,l to the server if the approximation error is large, where δ p ≥ 0 is a user defined threshold. On the other hand, if (17) does not hold (i.e., approximation error is small), agent i skips the communication of S k i,l , and the server uses the approximation S k i,l instead. In summary, for each agent i, instead of using D k i as defined in (13), the server now uses an approximation D k i that consists of a mixture of exact and approximate blocks. Specifically, the l-th diagonal block of D k i is given by, Finally, the preconditioner becomes P k = ( Lazy communication of reduced gradient. We can employ a similar strategy to design a lazy communication rule for the transmission of the reduced gradient w i , which is needed by the server to compute the update step in (12). For this purpose, let us view w i as a block vector, where each block w i,l corresponds to a single element y l in the shared variable. For each block l, let k < k be the last iteration when agent i uploads w k i,l to the server. Using w k i,l , we can compute an approximation of w k i,l as follows, Once again, a transporter is needed to ensure (19) defines a valid tangent vector on the tangent space at the current iteration k. Similar to (16), computing (19) does not require communication between the server and agent i. Similar to the previous development, at each iteration, agent i only uploads w k i,l to the server if it differs significantly from w k i,l . Specifically, we define the following communication triggering condition, The left hand side of (20) measures the approximation error, using the norm associated with the current preconditioner of this block. The right hand side defines a threshold on the approximation error using information from the past d iterations. Specifically, w k−d is the approximate reduced gradient used by the server at iteration k − d, and its exact definition is provided in (22). Bothd and weights { d ≥ 0, d = 1, . . .d} are user specified constants. Note that setting d = 0 forces the agent to always upload. Consequently, on the server's side, instead of using the upto-date w k i vector for agent i, it uses an approximate version w k i that consists of both exact and approximate blocks, In parallel, agent i computes the local second-order approximation mi (5), and evaluates w k i (10) and D k i (13) 4: // Lazy communication of preconditioner 5: for each agent i in parallel do 6: for each block l do 7: Upload S k i,l to server if (17)  Server computes preconditioner P k = ( N i=1 D k i ) −1 and broadcasts to agents 12: // Lazy communication of reduced gradient 13: for each agent i in parallel do 14: for each block l do Server computes v k = −γP k w k and broadcasts to agents 21: In parallel, each agent computes u k i = u i (v k ); c.f. (8) 22: Both server and agents update iterates Finally, instead of using w k to compute the update in (12), the server uses the approximation defined as, We conclude this subsection by noting that the lazy communication condition for reduced gradient (20) is more complex than the condition for preconditioner (17). The more complex rule (20) is needed for our convergence analysis, and we provide more discussions in Sec. V.

C. The Complete Algorithm
We collect the steps discussed above and present the Lazily Aggregated Reduced Preconditioned Gradient (LARPG) algorithm with the complete pseudocode in Algorithm 1. Each iteration of LARPG has three stages. The first stage (lines 4-11) performs the lazy communication of the preconditioner. The second stage (lines 12-18) performs the lazy communication of the reduced gradients. We note that this stage needs to happen after the first stage, as the triggering rule for the reduced gradient (20) depends on the preconditioner P k . The third stage (lines [19][20][21][22] uses the lazily aggregated information to compute the next iterate of the algorithm. Remark 1 (Novelty with respect to [9]). Our lazy communication scheme is inspired by Chen et al. [9], who study lazily aggregated gradient methods in distributed optimization. However, our algorithm and analysis (Sec. V) consists of the following important innovations to account for the unique challenges of Problem 1: (i) we consider problems with non-convex manifold constraints that are prevalent in robot perception applications, (ii) we incorporate the use of approximate second-order updates that require substantial changes in the convergence analysis, (iii) we handle private variables via analytic elimination, and (iv) we propose lazy communication on individual blocks of the gradient and preconditioner, which leads to further communication reduction.
Remark 2 (Implementation). In many applications, such as collaborative SLAM, each agent i only observes parts of the shared model during navigation. Consequently, the local cost f i only depends on the observed subset of the shared variable y. In our implementation and experiments (Sec. VI), we account for this fact by performing lazy communication only on the observed parts of y for each agent.

V. CONVERGENCE ANALYSIS
Since LARPG allows agents to lazily upload information to the server, it is unclear if the algorithm can converge to a desired solution in general. In this section, we provide a rigorous answer to this important question. In particular, we show that under mild technical conditions, LARPG provably converges to a first-order critical point of Problem 1, despite the use of lazy communication. Assumption 1 below summarizes these technical assumptions. Assumption 1. There exist constants L > µ ≥ c g > 0 and µ p , σ p > 0 such that the following conditions hold at any iteration k ∈ N of Algorithm 1, A1 (Lipschitz-type gradient for pullbacks [38]) Let f k f (x k , y k ) and g k grad f (x k , y k ) denote the objective and Riemannian gradient at iteration k. The pullback function f k (u, v) f (Retr x k (u), Retr y k (v)) satisfies for all (u, v) ∈ T x k X × T y k Y.

A2 (Bounded Hessian approximation) The approximate
Hessian M k at iteration k satisfies µI M k LI. A3 (Preconditioner) The preconditioner P k at iteration k satisfies P k µ p I and S k P k P k ≤ σ p . Above, (A1) is first introduced in [38] as a generalization of the standard Lipschitz smoothness assumption to the Riemannian setting. Intuitively, (A1) bounds the pullback function by its local linearization. Prior work in distributed BA [8], [27] requires similar smoothness conditions for convergence. In comparison, our assumptions and convergence guarantees extend beyond BA and hold for more general problems. (A2) assumes that the employed Hessian approximation M is bounded, which is also a standard assumption. Lastly, (A3) assumes that the preconditioner P k is sufficiently positive definite, and the approximation error of P k as the inverse of the reduced Hessian S k is bounded. We note that the latter two assumptions (A2) and (A3) can be easily satisfied, since the user has freedom to change what Hessian approximation M and preconditioner P to use; for example, M = c g I is always a valid choice that satisfies (A2).
The key to our convergence analysis (inspired by [9]) is to study the iterates of LARPG with respect to a Lyapunov function, At iteration k, V k combines the cost function with weighted squared norms of past approximate reduced gradients. Intuitively, these squared norms account for the approximation errors caused by lazy communication. In Appendix I-B, we show that LARPG is a descent method with respect to V k . This enables us to establish our main theoretical result.
In Appendix I-B, we prove Theorem 1 and provide explicit parameter settings that guarantee convergence. The established O(1/K) convergence rate matches standard global convergence result in Riemannian optimization [38]. While our convergence conditions involve additional parameters, experiments (Sec. VI) show that LARPG is not sensitive to these parameters and converges under a wide range of values.

VI. EXPERIMENTAL RESULTS
In this section, we evaluate LARPG on BA problems from benchmark collaborative SLAM and SfM datasets. All algorithms are implemented in C++ using g2o [13], and experiments are conducted on a computer with Intel i7-7700K CPU and 16 GB RAM. Unless otherwise mentioned, the default parameters we use for LARPG are summarized in Table III in the appendix. Our results show that LARPG converges under a wide range of parameter settings, and compares favorably against existing methods while achieving up to 78% total communication reduction. In the rest of this section, we first perform ablation studies on the proposed lazy communication scheme (Sec. VI-A). Then, we present evaluation and comparison results on large-scale benchmark datasets (Sec. VI-B and VI-C).

A. Evaluating Lazy Communication
We evaluate the proposed lazy communication scheme using the Castle30 dataset [37], which consists of 30 images observing a courtyard (Fig. 1a). We use Theia [42] to generate the input BA problem, which contains 23564 map points in total. We divide the BA problem into 30 agents and run LARPG for 50 iterations. In this experiment, we find that it is sufficient to fix the preconditioner at the initial iteration, which corresponds to letting δ p → +∞ in (17). This is because for this relatively simple problem, the initial preconditioner already gives a good approximation of curvature information at all subsequent iterates. Consequently, we   [39], [40]. Columns N, #IM, #MP, #Obs denote the total number of agents, images (keyframes), map points, and observations, respectively. Init: input to all algorithms. Ref: reference solution from centralized optimization using g2o [13]. PCG: baseline distributed preconditioned conjugate gradient method [26]. DR: baseline Douglas-Rachford splitting method [27]. LARPG: proposed method ( = 1). For each metric, the best-performing distributed method is highlighted in bold.  mainly focus on evaluating parameters that affect the lazy communication of gradients (20). We first evaluate the impact of d in (20). Intuitively, larger values of d imply that agents are more tolerant of gradient approximation error, and hence communicate less at each iteration. We set all d (d = 1, . . . , D) to a common value and vary in our experiments. To measure solution accuracy, we record the root-mean-square error (RMSE) of camera positions, computed after aligning with the ground truth via a similarity transformation. Fig. 1b shows the convergence of LARPG under varying values of . For comparison, we also include a reference solution computed by centralized optimization using g2o. Except when using a very loose threshold of = 100 (red curve), lazy communication has minimal impact on the iterations of LARPG. Furthermore, the communication efficiency of our method is clearly seen in Fig. 1c, where we plot convergence as a function of total amount of uploads to the server. To provide more insights, Fig. 1d visualizes the amount of gradient blocks uploaded to the server at each iteration. For each value of , the corresponding solid line denotes the percentage of uploaded gradient blocks averaged across all agents, and the surrounding shaded area represents one standard deviation.
Recall that choosing = 0 forces all agents to upload all blocks at every iteration (blue curve in Fig. 1d). Our result clearly shows that varying provides an effective way to control the amount of uploads during optimization.
In addition, we also evaluate the impact ofd on convergence. Recall from (20) thatd determines the number of past gradients that are used to compute the triggering threshold. Fig. 1e shows the performance of LARPG under different choices ofd with fixed = 5. While the differences are not significant, our result still suggests that using more past gradients (e.g.,d = 9) helps to save more communication.

B. Performance on Collaborative SLAM Datasets
In this subsection, we evaluate LARPG on collaborative BA problems from multi-robot SLAM applications. We use the monocular version of ORB-SLAM3 [43] to extract BA problems from the EuRoc [39] and KITTI [40] datasets. Each EuRoc dataset contains multiple sequences recorded in the same indoor space, and we use the multi-session feature of ORB-SLAM3 to simulate each sequence as a single robot. For each KITTI dataset, we divide the overall trajectory into multiple segments to simulate multiple robots. We generate noisy inputs for each dataset by perturbing the ORB-SLAM3 estimates by zero-mean Gaussian noise. 4 We compare LARPG against two baseline methods that can be implemented under the communication architecture considered in this work. The first baseline is the method in [26] using distributed preconditioned conjugate gradient (PCG). In our case, we use distributed PCG to solve the reduced second-order approximation in Lemma 1, where the problem is re-linearized after every 10 PCG iterations. The second baseline is the Douglas-Rachford (DR) splitting method proposed in [27]. Similar to our method, both baseline methods only require agents to communicate information over the observed parts of the shared model (see Remark 2). Table I shows the performance of all algorithms after 50 iterations. All results are averaged across 10 random runs. We evaluate the RMSE absolute trajectory error (ATE) against ground truth, the mean reprojection error, as well as the total amount of uploads during optimization. For comparison, we also include a reference solution computed by centralized optimization using g2o. We note that the higher ATE in KITTI is due to the larger scale of the datasets. As shown in Table I, LARPG achieves similar or better performance compared to baseline methods, while using significantly less communication. Specifically, when compared to DR, LARPG achieves up to 65% total communication reduction, clearly demonstrating the communication efficiency of our method.

C. Performance on Collaborative SfM Datasets
We also evaluate LARPG on collaborative SfM scenarios using the 1DSfM dataset [41], which contains 15 medium to large scale internet photo collections. We use Theia [42] to generate the input BA problems. Then, we partition each problem randomly to simulate a scenario with 50 agents.
Similar to the previous subsection, we evaluate the performance of all algorithms after 50 iterations. Table II shows the results. Since ground truth is not available, we only record the final mean reprojection error. LARPG outperforms baseline methods in most datasets, and achieves final reprojection errors that are close to the centralized reference solutions. Once again, LARPG demonstrates superior communication efficiency. When compared to DR, our method achieves 68%-78% reduction in terms of total uploads. Lastly, we also evaluate the average local iteration time of all methods (last three columns in Table II). Our method is faster than DR, since the latter requires each agent to solve a smaller nonlinear optimization problem at every iteration. On the other hand, the local iteration time of our method is larger than PCG. However, considering the large size of the SfM datasets, an average iteration time ranging from 8 ms to 293 ms for our method is still reasonable, and can be improved by further optimizing our implementation (e.g., via additional parallelization). 4 Specifically, the noise standard deviation for robot rotation, robot position, and map points are set to 5 deg, 0.1 m, 0.05 m for EuRoc, and 5 deg, 2 m, 0.1 m for KITTI.

VII. CONCLUSION
We presented LARPG, a communication-efficient distributed algorithm for collaborative geometric estimation. Each iteration of LARPG allows agents to analytically eliminating private variables. Furthermore, by incorporating lazy and partial aggregation at the server, LARPG achieves significant communication reduction, which makes it suitable for multi-robot and mixed reality applications subject to limited network bandwidth. Under generic conditions, we proved that LARPG converges globally to first-order critical points with a sublinear convergence rate. Evaluations on large-scale BA problems in collaborative SLAM and SfM scenarios show that LARPG performs competitively against existing techniques while achieving a consistent communication reduction of up to 78%.
While our formulation is general, the current evaluation focuses on BA with the standard least squared costs. In future work, we plan to evaluate on more applications and study the impact of robust cost functions. In addition, it would be interesting to further improve the lazy communication scheme, e.g., by designing a single triggering condition that controls the communication of both preconditioners and gradients.

A. Forming the Reduced Second-Order Approximation
Proof of Lemma 1. Recall from (6) that the global second-order approximation m(u, v) involving both private vectors u ∈ T x X and shared vector v ∈ T y Y is defined as, Setting the gradient of m(u, v) with respect to u to zero yields, Recall the definition of A and C in (7). In particular, since A is a block-diagonal matrix, the i-th component of u (v) (corresponding to agent i) is given by, Next, substitute u (v) defined in (27) into (26). After collecting terms, we obtain, Consider the vector w as defined in (29). Note that the global Riemannian gradient with respect to y satisfies g y = N i=1 g iy where g iy grad y f i (x i , y). In addition, because of the block-diagonal structure of A in (7), (10). Similarly, for the matrix S defined in (29), it can be readily verified that S = N i=1 S i where S i is defined as in (11).

B. Convergence Proofs for Algorithm 1
We start by reviewing several notations that are needed in this section. We use the superscript k to denote the value of a variable at iteration k of Algorithm 1. For example, g k x denotes the value of the Riemannian gradient g x introduced in (6) at iteration k. Recall that · (without subscript) denotes the standard norm associated with the Riemannian metric. We also introduce an additional notation to simplify our presentation. Recall that P k : T y k Y → T y k Y is the preconditioner used at iteration k to update the shared variable. In the following, we use ·, · k and · k as shortcuts for ·, · P k and · P k , i.e., In order to establish the convergence of Algorithm 1, we start by analyzing the change in the global objective (1a) after a single iteration (one step change). The following lemma provides an upper bound on the change in objective value.
Proof. Let m k (·) and h k (·) denote the second-order approximation in (6) and reduced second-order approximation in (9) at iteration k, respectively. Substitute v k = −γP k w k into h k (·): In (33), recall that w k = N i=1 w k i is the true global gradient computed using latest local gradients from all agents. On the other hand, w k = N i=1 w k i is the approximate global gradient computed using the lazily uploaded gradients from agents.
Rearrange the third term in the right hand side of (33) as, In (37), the matrix γP k is positive definite, and we use γP k to denote its matrix square root. Recall the equality a, b = 1 2 a 2 2 + 1 2 b 2 2 − 1 2 a − b 2 2 . Applying this equality on the inner product term in (37), we obtain, Substitute (38) into (37), we obtain, Now, let us focus on the last term in (33). Applying the Cauchy-Schwartz inequality with respect to the norm induced by P k , it holds that, Next, we obtain an upper bound for w k 2 k . First, using the definition of the approximate gradient w k , it holds that, Next, applying Young's inequality, we arrive at the following upper bound, = (1 + ρ) w k 2 k + (1 + ρ −1 ) where ρ > 0 is any constant. Plug (60) into (57). After grouping terms, we arrive at, Next, we obtain an upper bound for the second row of (61). Note that, Recall the communication triggering condition (20). By definition, (20) guarantees that the approximation error for each block l is upper bounded as follows, Summing the above inequality over all m blocks, we can obtain an upper bound on the approximation error for the entire local gradient, Substitute (66) into (64), it holds that C = A 1/2 ZB 1/2 where Z ≤ 1 [44, Lemma 3.5.12]. Therefore we also have that C ≤ L. Applying these results together with the triangle inequality, Lastly, for g k grad f (x k , y k ), it holds that The proof is completed.     [39], [40], generated using ORB-SLAM3 [43]. Map points are shown in black. Poses of each simulated agent are shown in a distinct color.  [41], generated using Theia [42]. Map points and camera poses are shown in black and blue, respectively. Each dataset is partitioned into 50 agents to simulate a collaborative BA problem.