{'fraction_non_alphanumeric': 0.034473684210526316, 'fraction_numerical': 0.015497076023391813, 'mean_word_length': 4.576553073536605, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
PLIKS: A Pseudo-Linear Inverse Kinematic Solver for 3D Human Body Estimation

We consider the problem of reconstructing a 3D mesh of the human body from a single 2D image as a model-in-the-loop optimization problem. Existing approaches often regress the shape, pose, and translation parameters of a parametric statistical model assuming a weak-perspective camera. In contrast, we first estimate 2D pixel-aligned vertices in image space and propose PLIKS (Pseudo-Linear Inverse Kinematic Solver) to regress the model parameters by minimizing a linear least squares problem. PLIKS is a linearized formulation of the parametric SMPL model, which provides an optimal pose and shape solution from an adequate initialization. Our method is based on analytically calculating an initial pose estimate from the network predicted 3D mesh followed by PLIKS to obtain an optimal solution for the given constraints. As our framework makes use of 2D pixel-aligned maps, it is inherently robust to partial occlusion. To demonstrate the performance of the proposed approach, we present quantitative evaluations which confirm that PLIKS achieves more accurate reconstruction with greater than 10% improvement compared to other state-of-the-art methods with respect to the standard 3D human pose and shape benchmarks while also obtaining a reconstruction error improvement of 12.9 mm on the newer AGORA dataset.


Introduction
Estimating human surface meshes and poses from single images is one of the core research directions in computer vision, allowing for multiple applications in computer graphics, robotics and augmented reality [16,50]. Since humans have complex body articulations and the scene parameters are typically unknown, we are essentially dealing with an ill-posed problem that is difficult to solve in general.
Thanks to models such as SMPL [38] and SMPL-X [48] additional constraints on body shape and pose became avail- able. They made the problem somewhat more tractable.
Most state-of-the-art methods [21,23,27,30,56] directly regress the shape and pose parameters from a given input image. These approaches rely completely on neural networks, while making several assumptions about the image generation process. One typical assumption is the use of a simplified camera model such as the weak perspective camera. In this scenario, the camera is assumed to be far away from the subject, which is generally realized by setting a large focal length constant for all images. A weak perspective camera can be described based on three parameters, two with respect to translation in the horizontal and vertical directions, and the third being scale. While these methods can estimate plausible shape and pose parameters, it can happen that the resulting meshes are either misaligned in the 2D image space or in the 3D object space. This is because the underlying optimization problem is often not constrained enough such that it is difficult for the underlying networks to optimize between the 2D re-projection loss and the 3D loss. Some existing methods [26,28,34] propose a workaround by tackling the problem using a hybrid approach involving learning-based and optimization-based techniques while incorporating a full perspective camera [26]. Optimization-based approaches are, however, prone to local minima, and they are computationally expensive. In [28], the authors propose to regress the SMPL parameters by conditioning on features from a CamCalib network meant to predict the camera parameters. Unfortunately, this camera prediction network needs a specialized dataset to train on, which is very hard to acquire in practice. It also prevents end-to-end learning.
On the other hand, recent non-parametric or model-free approaches [36,43] directly regress the mesh vertex coordinates based on their 2D projections, aligning well to the input image. However, by ignoring the effects of a perspective camera, even these methods suffer from the same limitations as the parametric models.
Motivated by the above observations, we present a novel approach, named PLIKS, for 3D human shape and pose estimation that incorporates the perspective camera while analytically solving for all the parameters of the parametric model. The pipeline of our approach comprises of two modules, namely the mesh regressor and PLIKS. The mesh regressor provides a mapping between an image and the 3D vertices of the SMPL model. Given a single image, any off-the-shelf Convolution Neural Network (CNN) can be used for feature extraction. The extracted features can then be used to obtain a mesh representation either by using 1D CNNs [43], GraphCNNs [31], or even transformers [36]. This way, correspondences to the image space can be found and a relative depth estimate can be computed. From the image-aligned mesh prediction, we can roughly estimate the rotations with respect to a template mesh in canonical space with the application of Inverse Kinematics (IK), denoted in this work as the Approximate Rotation Estimator (ARE). Finally, we reformulate the SMPL model as a linear system of equations, with which we can use the 2D pixel-aligned vertex maps and any known camera intrinsic parameters to fully estimate the model without the need for any additional optimization. As our approach is endto-end differentiable and fits the model within the training loop, it is self-improving in nature. The proposed approach is benchmarked against various 3D human pose and shape datasets, and significantly outperforms other state-of-the-art approaches.
To summarize, the contribution of our paper is the following: (1) We bridge the gap between the 2D pixel-aligned vertex maps and the parametric model by reformulating the SMPL model as a linear system of equations. Since the proposed approach is fully differentiable, we can perform endto-end training. (2) We propose a 3D human body estima-tion framework that reconstructs the 3D body without relying on weak-perspective assumptions. (3) We show that our approach can improve upon other state-of-the-art methods when evaluated across various 3D human pose and shape benchmarks.

Related Work
Recovering human pose and shape from monocular images has been extensively studied using both modelbased [23,27,56] and model-free approaches [12,36,43]. Model-based methods estimate the parameters of a parametric body model such as SMPL [38] based on a single input RGB image. The use of parametric body models makes it possible to enforce strong statistical priors of the human mesh. Model-based methods can be further split into optimization and regression techniques. Optimization-based approaches [4,7] make use of 2D keypoints estimated by a Deep Neural Network (DNN) which are iteratively fit with the SMPL model. These methods however are sensitive to initialization and are susceptible to local minima. Regression-based techniques based on DNNs directly estimate the pose and shape parameters [11,23,45,49,56,58]. However, these approaches typically require a large amount of training data. SPIN [30] provided a revolutionary architecture combining optimization-based techniques with regression-based methods. This allowed for much stronger supervision and improved performance on mesh accuracy. However, due to the difficulty in directly estimating the mapping from a single image to the shape and pose space, the mesh alignment with respect to the input image is often imperfect [19,32]. Model-free approaches directly regress the vertices based on intermediate representations, such as IUV maps [12,60,63], 2D/3D heatmaps [19,32], silhouette [53,61], and direct vertex regression [31,35,36,43] where correspondence is established between the model and the input image.
To overcome the issues of the learning-based and optimization-based approaches, hybrid techniques combining both approaches have been proposed [8,19,32,33]. In contrast to SPIN [30], here we have a closed-form solution. First, the learning-based approach localizes 3D human joint coordinates [5,6,18,41], utilizing volumetric heatmaps or GraphCNNs for the target representation. From the localized 3D joints, the swing rotations are then determined analytically, whereas the twist rotations, shape parameters, and root translation are either predicted by a network or iteratively optimized [19,32].
Typical human reconstruction methods [23,30] take a cropped input image while using 3D predictions projected onto the cropped image for 2D supervision. This, however, ignores the effects of perspective warping. This happens when the cropped image is off-center resulting in inaccurate rotations [62]. To overcome this, some approaches [19,26] perform iterative optimization [4] after the initial DNNbased predictions. SPEC [28] proposes to condition the image features from a camera calibration network. However, a few methods include perspective warping during the cropping process and add an implicit camera rotation as post-processing [39,62]. CLIFF [34] addresses the problem by incorporating the bounding box information into the cropped image.

Methodology
In this section, we present our network architecture which includes an analytical solver for inverse kinematics and is end-to-end trainable. As illustrated in Fig. 2, our network consists of two parts, first, a mesh regressor and second, a Pseudo-Linear Inverse Kinematic Solver (PLIKS). In Sec. 3.1, we go over the SMPL model along with its forward kinematics process, and in Sec. 3.2, we explain the full pipeline for mesh reconstruction.

Parametric Mesh Representation
We use the Skinned Multi-Person Linear (SMPL) model to parameterize the human body [38]. The SMPL model is a statistical parametric function M(β, θ; Φ). The output of this function is a triangulated surface mesh with N = 6890 vertices. The shape parameters β are represented by a low dimensional principal component which maps the linear basis B from R |β| → R 3N , representing offsets to the average meshx m as x=x m +βB. The pose of the model is defined with the help of a kinematics chain involving a set of relative rotation vectors θ = [θ 1 , . . . , θ K ] ∈ R K×3 made up of K = 24 joints represented using axis-angle rotations. Additional model parameters summarized as Φ are involved in the deformation process of the SMPL model. They are used as joint regressor J ∈ R K×3N , blend weights W ∈ R K×N and shape deformations conditioned on the body pose B P (θ). Starting from a mean template mesh, the desired body mesh is obtained by applying forward kinematics based on the relative rotations θ and shape deformations β. The 3D body joints can be obtained by a linear combination of the mesh vertices using any desired linear regressor J (M(β, θ; Φ)).
Simplified SMPL Although the SMPL model described above is intrinsically linear, it cannot be defined as a linear system of equations to solve the IK due to the forward kinematics and pose-related shape deformations B P (θ).
To deal with this problem, we make use of a simplified model, where we ignore the pose-related shape deformations B P (θ) in the formulation. We further split the SMPL mesh into K=24 segments s k∈(1,K) corresponding to the influence of the 24 rotation vectors θ of the SMPL model as shown in Fig. 3. A vertex on the mesh with index i belongs to a particular segment that has the maximum blend weight of all the joints, i.e., arg max k∈(1,K) W(i). For ease of notation, we represent a set of vertices/indices corresponding to a segment s k with the superscript k. Here, we also consider the individual segments to be a set of rigid 3D points which enables us to orient these segments from the template meshx k m to a network predicted mesh X k to obtain an initial rotation estimate using ARE explained in the following section.

Pseudo-Linear Inverse Kinematic Solver
Solving Inverse Kinematics (IK) from 2D correspondences is a challenging task due to the inherent non-linearity that exists in the parameterized mesh generation process. We propose to solve IK from 2D pixel-aligned vertex inputs by building a linear system of equations of the form Ax=b. We add shape constraints to obtain the optimal world poseθ, shape β, and world translation t. Linear-Least-Squares is used to estimate the optimal parametric solution making the entire pipeline end-to-end differentiable. From the world poseθ, the relative rotations θ can be inferred by recursively solving the kinematic tree. For the linear system, we assume the rotations as a first-order Taylor approximation [3] with the individual segments of the predicted mesh model approximately orientedθ along the optimal solution. Based on this assumption, using the rotations estimated from ARE should provide the exact solution in a single optimization step.
Regression Network Assuming that some form of dense vertex correspondences exists that facilitates mapping between 3D vertices to pixels in the 2D image plane, we can then incorporate the IK solver into the network. As shown in Fig. 2, our architecture is comprised of an encoder, a mesh regressor, and a parameter regressor. The encoder is based on HRNet-W32 [55], the mesh regressor is based on MeshNet [43], whereas the parameter regressor is a set of fully connected layers.
The encoder acts as a feature extractor F ∈ R C×c ×c , with a channel dimension C=480, height and width c =58. It takes a cropped image I ∈ R 3×224×224 of a person as input. Similar to MeshNet [43], we make use of 1D convolutions to generate four feature vectors for each mesh vertex. Here, P u and P v represent the features along the u and v axis, P d represents the features along the root-normalized depth, and w represents the weighting factor. They are obtained as follows (1) Here f 1D i (.) and ψ i (.) represent a 1D convolution along the i−th dimension, which converts the features F from C×c → N ×c and C×1 → (c ×C) T , respectively. The average function avg i (.) averages the features along the i−th dimension. For the weighting factor w, we average across the channel dimension, followed by applying the sigmoid activation function σ(.) [44]. More details about the weighting function w are explained in the following sections.
We then concatenate and processP={P u , P v , P d } using a graph convolution network (GCN) to predict uvd ∈ Figure 3. We split the SMPL model into 24 segments based on the blend weights. These segments help in determining the initial rotation estimate in the ARE pipeline. Here the segments are assumed to be rigid. R N ×3 . We use GCN rather than the heatmap-based approach from MeshNet [43] to avoid any truncation-based artifacts, which are prominent when partial images of humans are supplied as inputs. For the GCN we use the formulation from Kipf et al. [25], defined as G=σ(ÃPW), whereÃ ∈ R N ×N denotes the graph adjacency matrix, W ∈ R c ×l denotes the trainable weights with l as the output channel dimension, and σ denotes the ReLU [1] activation function. We make use of 3 GCNs in series, with channel sizes 64, 32, and 3 respectively. The final output X I uvd ∈ R N ×3 acts as the vertex correspondence in the image coordinate system.
The parameter regressor is a set of fully connected layers to obtain an approximate shapeβ and a weak perspective camerac ∈ R 3 , which are used to determine the approximate world rotations later. We make use of the estimated depthc d from the camera prediction to obtain a mesh in the world coordinate system X as follows, where K ∈ R 3×3 is an intrinsic matrix with fixed focal length of 1000.
Approximate Rotation Estimator (ARE) Given two sets of corresponding points in 3D space, it is possible to obtain an optimal rotation as a closed-form solution using the Kabsch solver [22]. For a given segment from network mesh prediction X k we make use of the Kabsch solver to determine the rotation that a same segment from the template mesh needs to go through from its rest pose. As the mesh prediction X k can represent a wide range of human shapes, we make use of the shape predictionsβ on the mean shape as x k =x k m +(βB) k . The pose solver minimizes the squared distances between a set of 3D correspondences to obtain an optimal pose as follows, Here C k is the covariance matrix between the correspondences. In this context,x k andX k represent the means over the associated point sets. The world rotation for the k-th segmentθ k =(V U T ) k is obtained by applying singular value decomposition (SVD) over the covariance matrix. Since the SVD is differentiable, gradients can be backpropagated during the training process. Due to the blend skinning W process, a vertex may have rotation influence from on one or more joints. To tackle this, we multiply the squared distances in Eq. (3) with a weight term w k from Eq. (1) and the maximum blend influence W k . The weight is learned in an unsupervised manner to offset any influence the blending process contributes.
The obtained rotations are in the world space, whereas the SMPL model expects relative rotations in its axis space. The rotation around the pelvisθ k=1 corresponds to the global root rotation. The relative rotations for the other joints can be recursively based on the parent rotation following a pre-defined kinematic tree for the SMPL model. To this end, letR k = R(θ k ) ∈ SO(3) represent the estimated world rotation determined for segment k. Then the relative rotation for segment k iŝ where p(k) represents the parent joint of k andθ k the estimated relative pose.
Pseudo-Linear Inverse Kinematic Solver (PLIKS) An approximate SMPL model projected onto the image plane, with no pose-related blend shapes can be represented as Here, the superscript k refers to the k-th's subset of vertices defined in Sec. 3.1. The matrixK ∈ R 3×4 represents a perspective projection matrix taking into account the affine transformation (only crop and resize) for the image fed into the network asK Further,R k represents the approximate world rotation obtained from the previous step, ∆R k is defined as the additional rotation required to get an optimal solution, t k represents the joint translation, and W are the blend weights.
By combining all segments, we get i = X I uv1 ∈ R 3 . This represents the correspondence between mesh vertices, defined in homogeneous coordinates, and the pixels in the image plane .
Since the additional rotation required is considered to be small we linearize the rotation matrix which needs to be determined based on Taylor expansion with angles α, β, γ along the x, y and z axes, respectively, as follows Further, we simplify the projected SMPL model in Eq. (5) by making some additional assumptions using the definitions introduced in Eq. (6). For the x k r term we assume that the majority of the rotation is significantly effected by the rotation with respect to the primary segment, i.e., we ignore the impact of neighbouring rotations as they are usually minuscule. This assumption is also made for the term B k r by assuming that for small rotations ∆R kRk βB k ≈R k βB k .
The equation is now linear, with 154 unknown parameters corresponding to ∆R k , β and t k . If the focal lengths f x , f y and principal points p x , p y are not known, we assume a fixed focal length of 1000 mm and the image center as the principal point. Note that the fixed values here apply to the entire image, i.e., not to the cropped and resized input fed to the network.
Using Direct Linear Transform (DLT) [13] we can rewrite Eq. (6) in the form Ax = b. The optimal parameters can be obtained by minimizing the analytical error using linear least square defined by x = A + b, where A + represents the pseudo-inverse of A. As the pseudoinverse of a tall matrix is differentiable [9], gradients can be back-propagated during the training process. One of the major drawbacks when using DLT is that it minimizes the analytical loss rather than the geometric loss. One option to overcome this is to use Iterative Reweighted Least Squares (IRLS) [10], which robustly minimizes the objective function in an iterative manner by reweighing the geometric loss. However, we make use of the network predicted weighting w k to weight the correspondences [29,51]. To further enforce the predicted shape to be close to the mean shape, we add an additional constraint such that ||β|| 2 ≈ 0 Method 3DPW (14) Human3.6M (14 with a regularizing weight ω β as To get the final relative pose θ k for each joint k, we apply Eq. (4) on the obtained world rotations ∆R kRk . The global translation for the camera systemK is obtained from the root joint as t 1 −(j 1 −∆R 1R1 j 1 ), where j 1 is the root joint for a rest pose mesh with shape coefficient β.

PLIKS Error Analysis
We measure the effective reconstruction capability of ARE and PLIKS with ground truth conditions for the 3DPW [59] test set and the AGORA [47] validation set. For the ARE module experiment, we provide the GT SMPL mesh vertex projections and the GT shape parameters. Additionally, we set the root depth as 7 m for all the images [26,52] Table 3. Evaluation on Human3.6M dataset, with respect to the MRPE and MRPE on x, y and z axis. ‡ means the network was additionally trained with 3DPW [59] and AGORA [47]. as the SMPL model in its template pose can be well represented inside a 224 × 224 image when assuming weak perspective settings with a focal length of 1 m. For the PLIKS module experiment, we only provide the GT SMPL mesh vertex projections and a root depth of 7 m to the ARE module along with the GT bounding-box camera intrinsics as inputs. We also set the weights for the shape regularizer ω β =0. We measure the Mean Per Joint Position Error (MPJPE), and the Per Vertex Error (PVE). Following previous works [23,27,30], we use the LSP joint regressor [30] to determine the 14 joints which are regressed from the body mesh. We report the results in Table 2 showing that the assumptions made in Eq. (6) are reasonable providing a stable and accurate fit. We observe a larger error on the AGORA dataset with the PLIKS module, due to larger perspective warping effects, which results in incorrect global rotation estimation from the ARE module. As PLIKS uses a linearized rotation matrix, running the PLIKS module twice, that is iteratively optimizing the rotation estimates provides a more accurate fit. Though ARE is not an accurate IK solver, we can safely conclude about the drawbacks of using a weak perspective camera model.

Comparison with the State-of-the-art
We compare our method with previous human mesh reconstruction approaches based on Human3.6M, MPI-INF-3DHP, and 3DPW datasets. To remain consistent with pre- In Table 1, we provide quantitative results for previous 3D human shape and pose estimation results. We use the best results reported in all the other works for our comparison. Our network outperforms all previous state-ofthe-art techniques on MPI-INF-3DHP and 3DPW datasets along with the lowest MPJPE on Human3.6M. Further finetuning with the AGORA dataset results in a significant performance improvement. We also provide results of our network when trained on a larger HRNet-W48 [55] backbone following previous works [34,36,64]. We provide qualitative results of our approach comparing prior methods [32,34] in Fig. 4. Though all methods shown in Fig. 4 align well in the 2D projected space, it is apparent that the prior methods fail to align well in 3D space due to perspective warping. The effect of perspective warping is observed significantly when the human is off-centered. On the MPI-INF-3DHP dataset, the advantage of incorporating PLIKS is more pronounced since the images are captured with a wide Field-of-View (FOV) (66 • ∼90 • ) showing a 26.3 mm MPJPE improvement.
In Table 5, we list quantitative results on the AGORA benchmark. It can be seen that the proposed approach significantly outperforms all prior methods. AGORA additionally measures the F1 score to get Normalized Mean Joint Error (NMJE) and Normalized Mean Vertex Error (NMVE), which penalizes missed or faulty human detection. Here the human bounding box and camera parameters are not provided for the test set. We make use of YOLO-v5 [20,42] for person detection. From Table 5, PLIKS was fine-tuned only on AGORA, whereas, PLIKS † was trained on all the 2D and 3D datasets which helps in generalizing to real-world images. As the focal length is not available during the test, we set f =1 m for all images. Making use of the estimated focal length from CamCalib [28] on PLIKS † shows a 3.4 mm NMJE improvement (PLIKS ‡ ).
Most methods reported in Table 1 generate root relative meshes using a weak perspective camera. As our results are in absolute camera coordinates, we present root local-  Table 5. Reconstruction errors on the AGORA test set. All results are taken from the official evaluation platform. PLIKS † was trained with all 2D and 3D datasets. PLIKS ‡ represents the usage of CamCalib [28] for focal length estimation during evaluation.
ization results in Table 3. We report previous results on Human3.6M provided from [41] on the mean root position error (MRPE). Here baseline [39,52] depth is obtained using the least squares fit from the predicted 3D joints and its corresponding 2D projections. Though [41] has explicitly been designed to learn the absolute depth, we observe comparable results along the depth while performing better in the horizontal and vertical directions. Using 3DPW and AGORA for training results in significant improvement of the root localization error. We also report the Absolute PCK result on the MuPoTs-3D [40] dataset. Note that the results are not strictly comparable due to differences in network backbones, the training datasets, and the joint prediction techniques.

Occlusion Analysis
To validate the stability under occlusion, we evaluate PLIKS on the object-occluded benchmark dataset 3DOH50K [65] and the person-occluded dataset PW3D-OCC [59]. Following [27], we use only COCO, Hu-man3.6M, and 3DOH datasets for training. To arrive at a fair comparison, we also train a network with ResNet50 [14] as the backbone. We observe better occlusion performance on both backbones from Table 7 for our approach. This can be attributed to the fact, that our approach is intrinsically trained on dense correspondences. Here, we also observe that the HRNet backbone performs better, as it provides higher resolution features.

Ablation Studies
In this context, we evaluate the individual components of our approach. For all experiments, we use only the COCO, Human3.6M, and MPI-INF-3DHP datasets for training and evaluation on all 3D datasets. We use the 3DPW validation set to determine the best model. In Table 6, we provide all the results for our ablation studies. The ARE module on its own can reconstruct 3D models provided the shape and camera parameters are regressed by the network. Hence, we evaluate its performance without the PLIKS module.
Input Image
Here, we observe that it performs poorly on all metrics for all datasets except for the PA-MPJPE of 3DPW. Next, on the same ARE module, we replace the HRNet-32 backbone with a ResNet-50 backbone. In this setting, similar to the occlusion test, HRNet performs better than ResNet. Next, we evaluate the importance of accounting for camera intrinsics by training a network with the PLIKS module but with all cropped images given as input to the network having a fixed focal length of f =0.3 m. The low focal length value violates the assumption of a weak-perspective setting. We observe slight improvements compared to that of ARE on the Human3.6M, whereas we see on-par results with the other datasets. Finally, we determine the effect of using a focal length of f = √ W 2 + H 2 proposed in [26] only during inference. While there is no significant difference in PW3D and Human3.6M, we observe a drop in performance for the MPI-INF-3DHP dataset compared to PLIKS with known camera intrinsics. This is due to the actual FOV being larger than the estimated FOV. Our ablations clearly demonstrate the importance of incorporating camera intrinsics into a network.

Conclusion
In this paper, we bridge the gap between 2D correspondence and body mesh estimation by creating a pipeline from which the inverse kinematics can be solved in closed form. This can be effectively leveraged to use the full perspective projection rather than having to rely on weak-perspective counterparts. Our approach yields considerable improvements in both root-relative and absolute-3D estimation for human pose estimation. PLIKS further enables our method to be fully differentiable facilitating end-to-end training. We validated the effectiveness of our method on various 3D pose and shape datasets and achieved state-of-the-art on multiple benchmarks. Due to the inherent nature of a built-in solver, we can extend our work with additional constraints like using multi-view systems for even higher reconstruction accuracy or temporal constraints for smooth video reconstruction should be feasible and foster future research in this direction.

Regularizer
To demonstrate the importance of a strong regularizer, we perform a similar experiment (from Sec 4.1) where we add random noise to the GT of the mesh vertices from the 3DPW [59] test set. Here we vary the shape regularizer weights ω β and observe the final MPJPE obtained. From Table 8, it is evident that larger weights for ω β is more robust to noise. However, training the network using larger weights has its own drawbacks as shown in Figure 6. The network forces the shape components β to always be close to zero. As the shape β is determined by a solver, it enables us to switch to a male, female, or neutral model seamlessly by replacing the shape coefficients B during inference. For our training, we set ω β = 0.1, as this is a good mixture between stability and shape variations.

Focal Length
We conduct experiments on the 3DPW and MPI-INF-3DHP test sets by varying the focal lengths. As shown in Fig. 5, PLIKS is robust to a wide range of focal lengths when the FOV is small (e.g., 3DPW), but it suffers from the effects of perspective warping on large focal lenghts for wide FOV images (e.g., MPI-INF-3DHP). Using Cam-Calib [28] on the MPI-INF-3DHP to determine the FOV and consequently the focal length of the image, we could only obtain a reduction in MPJPE of 72.01 mm, i.e., a drop of just 3%. In particular, when, the ground truth camera matrix is known, our approach can be expected to yield optimal performance.  Table 8. Ground truth errors in the presence of per vertex noise ranging from ±10mm to ±30mm and the effect of using a shape regularizer, ω β .

Qualitative Results
In this section, we show comparisons to SOTA methods on AGORA and provide more qualitative results on various other datasets.

Qualitative Comparison
We display several examples of PLIKS on the AGORA test set in Fig. 7. We use YOLO [20] for the bounding box estimation and CamCalib [28] for the focal length estimation. The images demonstrate that PLIKS performs better than previous approaches, by aligning the bodies well in 3D as well as 2D.

Inference Modification
As mentioned in the main paper, one of the strengths of our method is the application of constraints during inference. Here, we discuss a proof-of-concept for two use cases, where we show the benefits of using a solver without any retraining of the network. We discuss dynamic shape and translation constraints.  Figure 6. Effect of the regularizer weight used during training on the shape coefficients. Picking a higher ω β reduces the error, but causes the network to output meshes getting progressively closer to the identity representation. Here neutral represents the neutral SMPL model, and gender refers to the gender-specific model on the 3DPW [59] dataset. Dynamic Shape Although our network was trained only on a neutral SMPL model with 10 shape components, it can make use of other shape models during inference if they follow the same design principle as SMPL. As an extreme scenario, we show the application using the kid-SMPL model [47,57]. The kid-SMPL is an extended version of the SMPL model supporting children by linearly blending the SMPL and Skinned Multi-Infant Linear Model (SMIL) [15] by a weighting factor α ∈ (0, 1) [47]. Here, larger weights represents infants, while smaller weights are associated with adults. For simplicity, we denote the kid-SMPL model as having 11 shape components. Qualitative results of using the kid-SMPL model on the Relative Human (RH) dataset [57] are shown in Fig. 8. The only modification performed was adapting the shape coefficients B k r in Eq. 9 from the SMPL to their kid-SMPL counterparts. In that context, we further empirically set ω β to 0.5. From the RH dataset we employ the GT age classifier, i.e., we use SMPL for adults, and kid-SMPL for child or infant. We observe visually satisfactory results, with sufficiently reliable depth reasoning. A top-down approach [57] or a simple age classifier could be designed to determine the age as a future work.

Translation Constraints
Previous examples of just using dynamic shapes is not a complete solution, due to the illposed nature of the problem. This is quite evident from the fifth column of Fig. 9. As a proof-of-concept, we show the application of translation constraints during inference. We add a simple depth constraint to Eq. 9 as ω t t k 0,z = ω t t a 0,z . Here, t a 0,z is the root depth of the adult in the image, and t k 0,z is the constrained setting for the root-depth of the kids in the image, with ω t being a weighting factor. We make the assumption that the children in the images are standing close to the adults. The solver optimizes the shape such that the translation constraint is satisfied. We empirically set ω t to 0.2. Though, strictly not comparable, we visualize the results of BEV [57] in Fig. 9. There, all images are in fact from the RH training set on which BEV was trained. We quickly add that this is not a real-world solution to the problem, but it emphasizes the importance of using constraints during inference or training. As future work, one could make use of the RH dataset with the depth-level information by adding a top-down approach [57] for better constraints.