{'fraction_non_alphanumeric': 0.03935627788400255, 'fraction_numerical': 0.016730401529636712, 'mean_word_length': 4.509304775280899, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Total Scale: Face-to-Body Detail Reconstruction from Sparse RGBD Sensors

While the 3D human reconstruction methods using Pixelaligned implicit function (PIFu) develop fast, we observe that the quality of reconstructed details is still not satisfactory. Flat facial surfaces frequently occur in the PIFubased reconstruction results. To this end, we propose a twoscale PIFu representation to enhance the quality of the reconstructed facial details. Specifically, we utilize two MLPs to separately represent the PIFus for the face and human body. An MLP dedicated to the reconstruction of 3D faces can increase the network capacity and reduce the difficulty of the reconstruction of facial details as in the previous onescale PIFu representation. To remedy the topology error, we leverage 3 RGBD sensors to capture multiview RGBD data as the input to the network, a sparse, lightweight capture setting. Since the depth noise severely influences the reconstruction results, we design a depth refinement module to reduce the noise of the raw depths under the guidance of the input RGB images. We also propose an adaptive fusion scheme to fuse the predicted occupancy field of the body and face to eliminate the discontinuity artifact at their boundaries. Experiments demonstrate the effectiveness of our approach in reconstructing vivid facial details and deforming body shapes, and verify its superiority over state-of-the-art methods.


Introduction
Three-dimensional reconstruction of humans, which aims to obtain a dense surface geometry from single-view or multi-view human images, is a fundamental topic in computer vision and computer graphics. While high-fidelity 3D human models can be reconstructed using commercial multi-view stereo software under the customized studio setting [7,16,34,50,53], it is highly desirable to lift the studio setting constraint that is hard to access by users. Low-cost RGBD sensors are recently popular in 3D human reconstruction, and tracking-based methods have been developed to fuse the depth data from RGBD sensors as the recon-Figure 1. Our volumetric capture results with the high-fidelity body and face-enhanced details from three groups of raw RGBD images. First two columns: geometric and normal results on our testing dataset. Last three columns: geometric and textured results from our real shots. struction results [41]. During fusion, the estimation of nonrigid human body deformation is essential to improve the reconstruction quality [40,64]. However, it is technically challenging to guarantee the stability of the depth fusion algorithms due to occlusions and severe noises in the depth data.
Learning-based 3D human reconstruction methods can significantly simplify the capture setting. After training on high-quality 3D human models, they can even recover 3D human shapes from single images [25,67]. The focus of learning-based 3D human reconstruction methods has been shifted from parametric human model [36,43] to pixel-aligned implicit function (PIFu) representation [45]. As demonstrated in [45], different types of details, such as hair and clothing, can be reconstructed using PIFu from images. However, topology errors in the reconstructed human models often occur in the regions invisible in the input images. PIFuHD [46] synthesized the normal maps for a captured human image at both frontside and backside, which can partially remedy the topology errors. However, the details reconstructed from the synthesized normals might not be consistent with the target. Through a carefully designed multi-view feature fusion scheme, the topology error of the reconstructed human model can be significantly reduced [22,63]. While PIFu-based methods developed fast, we observe that the quality of reconstructed details is still not satisfactory. For instance, flat facial surfaces frequently occur in the PIFu-based reconstruction results. It is technically challenging for PIFu-based methods to achieve high-fidelity detail reconstruction results without topology errors.
In this paper, we propose a two-scale PIFu representation to enhance the quality of the reconstructed facial details. Specifically, we utilize two MLPs to represent the PI-Fus for the face and human body separately. It is inspired by the fact that the complexity of a function can be reduced when observing it locally. Thus, we expect that a separate MLP for the reconstruction of 3D faces can increase the network capacity and reduce the difficulty of reconstructing facial details as in the previous one-scale PIFu representation. Moreover, we propose an adaptive fusion scheme to fuse the predicted occupancy field of the body and face to eliminate the discontinuity artifact at their boundaries. Additionally, to address the topology issue, we leverage 3 RGBD sensors to capture multiview RGBD data as the input to the network, a sparse, lightweight capture setting. Since the depth noise severely influences the reconstruction results, we design a depth refinement module to reduce the noise of the raw depths under the guidance of the input RGB images. As illustrated in Fig. 1, our method can generate reconstruction results with the high-fidelity body and enhanced facial details for different actions.
In summary, the main contributions of our work are: • We propose a two-scale PIFu representation to reconstruct 3D humans from very sparse RGBD inputs. Moreover, a novel depth refinement module is incorporated into our network to reduce noise and improve reconstruction details under the supervision of the differentiable surface reconstruction module. • Extensive experiments show that the proposed human total reconstruction model can obtain high-quality body reconstruction results, especially face, under the noisy depth maps taken by the Kinect sensors.

Related Work
Tracking-based Human Reconstruction. A line of human reconstruction methods [4,5] is tracking-based, which leverage the pre-captured mesh templates [17-19, 29, 72] or temporal fused mesh [28,40,41,48,64] as the reference model and track the human motions by inferring the deformations of the reference model. Some methods [3,15,19,32,59,62,64] are proposed to use 3D pose to parameterize the reference model, while others [21,23,44,61,64,65] typically leverage the parametric body model SMPL or SMPL-X [36,43]. Furthermore, Hanbyul et al. [24] constructed parametric models for the human face, body, and hand to improve the reconstruction results. Liu et al. [35] propose to combine image segmentation and tracking with 3D shape prior to multi-person reconstruction. Similarly, Mustafa et al. [38] leverage the segmentation with the optical flow to reconstruct humans in dynamic scenes. To tackle the occlusion and large motion problems, some methods develop high-end capture systems for the dense capture of human performance, which consist of a large number (up to 100) of RGBD sensors [7,34,50] or custom color lights [16,53] (e.g., there are 1200 individually controllable light sources in the acquisition setup in [53]).
Learning-based Human Reconstruction. Another line of methods [1,14,39,47,51,55,60,70,71] is learning-based, which leverage the advanced neural 3D representations for reconstructing the geometry and/or texture details. Some methods proposed to reconstruct 3D human meshes from a single RGB [47] or RGBD [55] image, by incorporating the SMPL [49,70]. To avoid being constrained to the SMPL template, some methods adopt the image-to-image translation pipeline to regress the input image into 3D meshes via 2D estimations of intermediate textures [1], silhouettes [39], and depth information [14], while others jointly exploit the 2D and 3D information, e.g., body joints and per-pixel shading information [71], and 2D/3D poses and segmentation map [51]. However, these methods typically suffer from over-smoothed surfaces in the occluded regions. Some methods [10,11,31] propose to leverage multi-view RGBD images for high-frequency detail reconstruction.
Recently, the pixel-aligned implicit function (PIFu) [45] has attracted much attention, as this implicit representation enables a direct infer of 3D object surfaces and textures from the 2D image pixels. Saito et al. [46] propose to estimate the front and back normal maps to reduce the geometry errors of PIFu in the occluded regions. The success of PIFu in detail reconstruction drives many methods to combine it with parametric model [56,66] and semantic segmentation [12], camera parameters estimation [27], novel view synthesis [30], single-view [31] or multi-view depth information [63], or apply it for point clouds based human reconstruction [2,6,37].
Our method tackles the limitation of lacking facial details of PIFu-based methods by learning a two-scale PIFu representation, enhancing our network capacity to reconstruct the geometric details at all scales.

Method
Our proposed face-to-body neural human reconstruction network is a two-stage network, as illustrated in Fig. 2. Given raw RGBD images from N (e.g., three) perspective views, our network can reconstruct a full-body human model with the face-enhanced details by leveraging the proposed depth refinement module (RDM) and the face-tobody neural surface estimation module (FBSM). In general, our system performs the following three steps sequentially:  Figure 2. Overview of our face-to-body neural human reconstruction network architecture. "x N " indicates that the transformer encoder is repeated N times. Depth Refinement: predict the refined depth maps of the raw depth maps with the guidance of RGB inputs. Face-to-body Neural Surface Reconstruction: predict the occupancy fields for the human body and face independently based on pixel-aligned depth and RGBD features, then integrate the two inferred fields to generate a complete model with high-quality details.
1) Depth Refinement. Given the full-body portrait (RGB) with binary mask and the raw aligned depth map as inputs, we design a contextual auto-encoder to obtain the refined depth with the guidance of the full-body portrait.
2) Face-to-body Neural Surface Reconstruction. To reconstruct the body along with the face-enhanced details, we design two pixel-aligned implicit functions for face and body, respectively, i.e., the two-scale PIFu representation for the 3D human. For the body part, a PIFu F b , called PIFu-Body receives refined depth maps from N views as inputs and predicts the occupancy field of the body part. For the facial part, we first detect and crop the human face from the designated front-view RGBD body portrait of the original resolution. Afterward, we design a light-weighted network as another PIFu F f , called PIFu-Face, to predict the occupancy field for the face according to the features learned from the up-sampled RGBD face image.
3) Face-to-body Occupancy Fields Fusion. We design an adaptive occupancy fields fusion scheme during inference to get a complete model with a smooth transition between face and body occupancy fields. Specifically, for the queried points projected on the face region, we calculate their projected signed distance function (PSDF) values through the refined front-view facial depth map and construct Gaussian probability distributions to weigh the occupancy values of the two fields. The PSDF value is defined as the difference between the depth component of a 3D point and its depth value obtained at its projection in a depth map [8,32]. Finally, we produce the global mesh through the marching cube algorithm.

Depth Refinement
Since the captured depth maps from consumer-level cameras have non-negligible noises and holes, directly utilizing the raw depth maps for static reconstruction will introduce the noises into the final results. Therefore, we propose a Depth Refinement Module (DRM) to reduce the sensor noises and refine the eroded boundaries of the input raw depth map with the guidance of RGB body portrait as inputs. This design also provides relatively cleaner depth inputs for our FBSM.
The network structure of our DRM is shown in Fig. 3. For clarity, we denote RGB body portrait as I, the corresponding binary body mask as M obtained by the background-matting method [33], D raw as the normalized result of the captured depth map, and the refined depth map as D rf , where D rf = DRM(D raw , I). The architecture with two different types of inputs, i.e., I and D rf , is inspired by the hypercolumn structure in [20] and DDR-Net [60]. Correspondingly, we design a contextual autoencoder with two encoding branches to fill the D raw 's holes and add high-frequency details from I to the refined depth map D rf .
We input I and D raw to the two branches of the encoder in DRM so as to extract texture and depth features. After each branch's last Conv layers, the network fuses these two types of features in an additive manner, a late fusion scheme to fuse RGB and depth information. Since we use fewer Conv layers to extract textured features, it also makes highlevel depth features contain more original high-frequency details, leading to a lighter encoder. In the bottleneck layers of the encoder, we first use four dilated Conv layers with different scales to extract global features, and then additionally leverage non-linear CBAM (Convolutional Block Attention Module) [58] blocks to compute the channel-wise and spatial attention. The decoder part upsamples the encoded depth feature to the original size, together with the skip connections to preserve depth details in D raw . Finally, we multiply the mask M by the output of the last Conv layer of the decoder to get the refined depth map D rf , resulting in zero values inside the background regions. Training Loss. We render the ground-truth depth map, denoted as D gt from a model in our training dataset. The first loss L depth is used to penalize the per-pixel difference between D rf and D gt : However, we observe that this loss alone might lead to blurring in the refined depth map.
To this end, we introduce a normal loss, denoted as L nor , to measure the difference between the normal maps computed from the predicted and the ground-truth depth maps, denoted by N rf and N gt respectively. In addition, we combine it with SSIM (structural similarity index) [57] loss to improve the details in the predicted results: where the weight β is set to 0.16, following the design in [68]. L is the smooth l 1 loss. The normal maps are calculated at each pixel according to the difference among its four surrounding pixels, a differentiable operation.
The final training loss for our DRM is then defined as: where λ 1 and λ 2 are the weights to balance the two terms.
In addition to the designed loss L DRM , D rf is also supervised by the loss terms used in FBSM, since D rf is the input of our differentiable FBSM and the back-propagated gradient will flow through D rf to DRM. Such a design  makes L DRM decline faster during joint training. The mean absolute error (MAE) for each pixel of our DRM results on our testing dataset is 0.0769cm. To simulate the RGBD sensor data, we synthesize the depth noises of D raw and impose the noise on D gt . As shown in Fig. 4, our DRM can effectively refine the raw depth map with eroded boundaries and holes (black-boxes), leading to better reconstruction results using our FBSM.

Face-to-body Neural Surface Reconstruction
Although the refined multi-view depth maps {D rf } i (i = 1, .., N ) output by the DRM can be fused (e.g., TSDF-Fusion in [41]) to obtain a 3D human model, the reconstructed mesh in this way contains large holes and low-quality regions due to the very sparse inputs and the self-occlusions. Therefore, we design a face-to-body neural surface estimation Module (FBSM), as shown in Fig. 5, to generate high-fidelity reconstruction results. This module consists of two-scale PIFus: PIFu-Body for the body part and PIFu-Face for the facial details. The PIFu-Body takes the multi-view depth maps as the input and predicts the volumetric occupancy field for the whole body, while the PIFU-Face takes both RGB and depth map cropped from the front-view RGBD image as the input and predict the volumetric occupancy field for the 3D face. Finally, these two occupancy fields are adaptively fused to reconstruct the final 3D human mesh. In the following, we first introduce some notations and then proceed to the details of PIFu-Body and PIFu-Face, multi-view feature aggregation, the fusion of two predicted occupancy fields, and the training of the two-scale PIFus.
Notation: Let us denote a 3D point in the bounding box of a human body as X, and its 2D projection x i at i-th view as x i = π i (X), where π i is the projection function. The symbol z i = z(X i ) is used to denote the depth of X in the local coordinate system of the i-th view, and p i (X) is the truncated-PSDF, where p i (X) = T (z i − S(x i ; D i rf )).  For the input RGBD images, we employ HRNet [54] as the backbone to extract pixel-aligned features. The HRNet for multi-view depth maps in PIFu-Body is denoted as H b , and H c denotes the HRNet for PIFu-Face. Two MLPs are used to represent the volumetric occupancy fields of human body and face, denoted by M b and M f respectively. Finally, the detected facial regions in the front-view RGBD image is denoted as R f . PIFu-Body. F b can be formulated as follows:

Face-to-body GeoNet
where one of the input to A is c i (X) = [z i , p i (X)]. It is a concatenation of local depth and Truncated-PSDF of X as in [63]. The original resolution of D raw when training is 2048 × 2048, but reduced to 512 × 512 after aligning it to the RGB image as the input of our DRM, which results in the resolution 512 × 512 of D rf . The symbol A indicates the multi-view feature aggregation module (see details below). PIFu-Face. F f can be formulated as follows: , depth information c(X f ) is defined in the same way with PIFu-Body, but the truncated-PSDF value is computed using the cropped D rf to reduce noises. The RGB image of the facial region is beneficial to enhance the reconstructed facial details.
Multi-view Feature Aggregation Module A. It is inspired by the attention mechanism in [52] and the transformerbased feature fusion methods in [63,69]. Specifically, we first adopt the transformer encoder with multi-head self-attention layers to denoise the features using attention weights sampled from multi-view feature maps for a query point X. Besides, we observe that the attention mechanism can avoid over-blurred features in the denoising. Our implementation uses multi-head attention with four heads and two linear layers in the transformer. After the feature denoising, we introduce a depth-based adaptive feature fusion method to aggregate the features a i (i = 1, ..., N ) output by the transformer encoder. For a 3D point X, we first calculate p i (X) at i-th view, and then linearly combine the features according to the fusion weights: exp (−σ 0 · (p i (X)) 2 ), where σ 0 is a factor to control the degree of aggregating (default 20).
Face-to-body Occupancy Fields Fusion. During inference, two occupancy fields must be fused to eliminate the discontinuity artifacts at their boundaries. For a 3D point X f projected to the detected facial region R f in the front view, we first calculate its 2D projection x f located between [−1, 1] in normalized image coordinates and then its PSDF value p f (X f ). These two quantities are used to compute weights using a mixture of Gaussian distributions: where the first component is larger at the center of facial region and decrease smoothly to the boundary. It is used to improve the smoothness of the fused occupancy field around the boundary. The second component emphasizes the occupancy value computed by PIFu-Face according to p f (X f ). Two weights, σ 1 (default 1) and σ 2 (default 1e 3 ), are used to control the fusion degree for two fields. Finally, we can leverage ω to fuse the two inferred occupancy values: O b , O f output by PIFu-Body and PIFu-Face respectively. The fusion operation is defined as Fig. 6.b.
Training Loss. We adopt the extended Binary Cross Entropy (BCE) loss [46,70] to train our FBSM on the sampled pointsX = [X b , X f ]: where O * b and O * f are the ground-truth occupancy values of the 3D pointsX of human body and face respectively, S 0 and S 1 denote the sets of samples, L BCE represents the BCE loss, α 0 and α 1 are the weights to balance PIFu-Body and PIFu-Face.
Given {D rf } i (i = 1, .., N ), Q f and the 3D query point X, we jointly train these two-scale PIFus. We first sample 3D points near the surface and inside the bounding box of the ground-truth scan, according to PIFuHD [46]. For facial point, we obtain the 2D projection x f and the depth z f = z f (X) of X in the front view: f . With the rendered groundtruth facial depth D c gt = Crop(D gt ) of the front view, we can set a flag v f (X) to mark face point for X through R f and the absolute PSDF value as follows: When computing the training loss, we will replace the It means that we expect the gradients of the 3D points on the face to flow through the F f such that F f can not be ignored in the training. In order to balance the training of the PIFu-Body and the PIFu-Face, we sample the same number of 3D points on face and body simultaneously, as illustrated in Fig. 6.a.

Experiments
In this section, we report implementation details, how we prepare the training dataset, comparisons as well as the ablation studies of our method. Implementation Details. We implement our two-scale PI-Fus using PyTorch [42] on a PC with two Nvidia Geforce RTX 3090 GPU. To minimize the loss terms for our DRM and FBSM, we adopt ADAM optimizer [26] to train our network for 30 epochs with a learning rate starting from 1e −4 and use an exponential learning rate scheduler to update it every 10 epochs by multiplying with the factor 0.1. The batch size is set to be 6. The β 1 and β 2 in ADAM are set to 0.5, 0.99, respectively. The network weights are initialized using a normal distribution (mean:0, variance: 0.02). The λ 1 and λ 2 appeared in our L DRM are set to 1, 2, respectively. And we set δ p as 0.01m, τ as 0.15m, α 0 and α 1 as 1 in our FBSM.
For H b in PIFU-Body, we follow [30,63] to use HRNetV2-W18-Small-v2 [54] as the backbone.The resolution of its output feature map is 128 × 128, and the number of channels is 128. In particular, we reduce the channel number of the inner blocks of H f of PIFu-Face to 2/3 of the original. For the transformer encoder, we set the repeated number N to 4. As for M b and M f , we use multi-layer perceptron (MLP) with skip connections [45,46,63] and set the channels dimension of hidden layers as (512, 256, 128, 128), (128,128,64,64,64), respectively.
We adopt three-view RGBD images with an interval of (135 degree, 135 degree, 90 degree) as the capture setting. The user is required to face the camera in the middle, i.e., front-view f . The real depth data is captured using Microsoft Kinectv4 sensors with the same camera pose parameter settings. The captured RGB resolution is 2560 × 1440 and depth image resolution 1024 × 1024. During testing, we use background-matting-v2 [33] to obtain the mask M of body portraits and then use RetinaFace [9] to detect the front face for the front-view. It takes about 3.57s for our model (32-bit floating-point precision) to predict the occupancy field of resolution 256 with the given inputs.
Training Dataset. We use the THuman2.0 dataset from [63] as our training dataset, which contains 500 highquality 3D human scans with various poses and cloth styles. First, we rotate each scan at 6-degree intervals on the yaw axis. Then we render RGB body portraits with an image resolution of 512 × 512 with CUDA acceleration based on the assumption that the body surface is mainly Lambert material, we also apply random shifts to augment the rendered results. Afterward, we render the ground-truth depth maps aligned with body portraits as D gt and synthesize the sen-  [45] (c) IPNet [2] (d) StereoPIFu (two RGBs, w/o Depth) [22] (e) Ours   sor noises on D gt according to [13] to obtain the raw depth map D raw . When training, we sample surface points, volume points according to PIFuHD [46] and obtain their occupancy values as the ground-truth labels. Besides, we randomly select 2 views along with 1 front view from the rendered 60 views of a human scan.

Comparisons
To evaluate the performance of our method, we compare it with 3 state-of-the-art deep implicit human surface reconstruction methods, including re-implemented PIFu with multi-view RGBD images as input [45], denoted as Multiview PIFu (RGBD) , IPNet (voxelized points cloud as input) [2] and StereoPIFu (stereo RGB images as input) [22]. We retrain the Multi-view PIFu (RGBD) and IPNet on our training dataset for fair comparisons. And we use point-tosurface (P2S) distance(mm), chamfer distance (mm), and MSE for normal map (1e −2 ) as metrics to measure the error between the reconstructed and the ground-truth surfaces on our testing dataset, which contains 106 high-quality models with different poses, clothes, and human-object interactions. The lower metric value means better performance. Quantitative Comparisons. Tab. 1 reports the comparisons on our testing dataset. It can be seen that our method is ranked as top-1 in both three metrics. Fig. 7 shows two reconstructing results in the testing. Although we use RGBD

Ablation Study
We perform ablation studies by removing the model components, including DRM, PIFu-Face, and Transformerbased multi-view feature aggregation (TFA), to better analyze our network's architecture. The statistics of P2S distance, Chamfer distance, and Normal error are obtained by evaluating the re-trained models on our testing dataset.  In Tab. 2, we first show that the three modules in our network all contribute to the reconstruction performance. DRM seems to be the most critical component to improve the metric. The reason might be that F b in our FBSM only takes the depth maps as inputs only. When our DRM is removed, the noise of the original depth map will directly appear in the results, e.g., (b, h) in Fig. 9. Moreover, as shown in the blue boxes in (c,e,i,k) of Fig. 9, our PIFu-Face significantly improved the geometric details of the face. As for the multiview feature aggregation module, The black boxes in Fig 9 show that directly averaging the multiview features may lead to the loss of some details (e.g., backpack strap) and the jagged-edges at the stitching (d, j). In contrast, adopting self-attention layers can more effectively remove the noises and preserve more body details.

More Results
We also conduct an experiment to explore our DRM performance by gradually adding different degrees of noise to the raw depth maps. The bottom row of Fig. 9 shows that: even if there is almost no depth information in D raw , our network can still predict a reasonable reconstruction result based on D rf . We hypothesize that our DRM is not highly dependent on the depth input, and it owns the ability to predict the depth maps of the whole body due to the joint endto-end training. Please refer to the supplementary material for more experimental results on our DRM. Fig. 1 and Fig. 10 show our reconstruction results on real RGBD data and our testing dataset. It can be seen that our reconstruction results can restore the high-fidelity details and the pose of the input body portraits. Furthermore, our facial reconstruction results significantly improved the high-frequency details of the facial features, e.g., eyes, nose, and mouth.

Conclusion and Discussion
In this paper, we propose a two-scale PIFu representation to reconstruct the human body and enhance the facial details. The key feature is that we leverage two implicit functions, i.e., PIFu-Body and PIFu-Face, to reduce the modeling complexity. In addition, we designed a DRM for reducing the noise of inputs and a transformer-based feature aggregation module to avoid over-smooth features. After the occupancy-field fusion, our method can generate more reliable, high-fidelity 3D human reconstruction results. Limitations and Future Work. It remains difficult for our method to handle the cases when the depth noise is significant in high-frequency regions, such as hands and feet. Adding more scales into our two-scale PIFu (refer to Total-Capture [24]) may help to solve this problem. Moreover, our method may not generate facial details when there is occlusion on the face caused by special material (hair, eye-glasses, etc.) due to the depth error, imposing prior to the facial region may handle this issue. We also plan to investigate the camera setting to improve the quality of reconstructed facial details.

C. Evaluation of Occupancy Fields Fusion
We conduct an ablation study by removing the face-tobody occupancy fields fusion in our FBSM. As a result, the occupancy values of the 3D points projected on the face region will be independently predicted by the face MLP. As shown in Fig. C, when the occupancy fields fusion operation is removed, the geometric noise and errors appeared on the back of the head and the marginal areas of the face (blackboxes) can be seen clearly. In contrast, our complete approach can generate results with less noise in these regions.
Since the PIFu-Face F f in our FBSM is trained only on the sampling points near the face region for learning facial geometry. For those points far away from but projected on the facial region, F f will not be able to generate reasonable occupancy prediction values. Then, it is necessary to perform occupancy field fusion on those points to eliminate errors.

D. The Process of Obtaining Real RGBD Data
In the capturing stage, we placed 8 groups of color and depth (KinectV4 sensors) cameras in a circle at 45-degree intervals to ease the calibration, 4 groups of photography lights at 90-degree intervals to provide uniform illumination, as illustrated in Fig. D.a. For each camera, we used the infrared calibration board to calibrate the intrinsic parameters and initial extrinsic Kinect camera parameters. Then we used the iterative closest point (ICP) method to match the points clouds captured by multiple depth sensors to further optimize the extrinsic parameters. Afterward, we recorded the captured RGBD images and performed the de-distortion operation on the captured RGB images. When testing on these real RGBD images, we will select three-view RGBD images with intervals of (135 degree, 135 degree, 90 degree) as shown in Fig. D.b, and the user is required to face the camera in the middle.

E. More Results
In this section, we first show more qualitative comparison results between ours and 3 state-of-the-art learningbased human reconstruction methods (i.e., PIFu [45], IP-Net [2], StereoPIFu [22]). As shown in Fig. E, our method can reconstruct high fidelity 3D human models at both front (a) Multi-view PIFu (RGBD) [45] (b) IPNet (outer and inner layers) [2] (c) StereoPIFu [22] (d) Ours (e) Ground-truth 3D Mesh  and back side of the model and 3D faces with more details. For the comparison with multi-view PIFu (RGBD), we also use HRNetV2-W18-Small-v2 [54] as the backbone (with the same setting as H b in our PIFu-Body) but set the number of input channels to 4 to facilitate the input of RGBD data. Besides, we show more reconstruction results with our network on real RGBD images in Fig. F.