Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation
  We propose a method for object-aware 3D egocentric pose estimation that
tightly integrates kinematics modeling, dynamics modeling, and scene object
information. Unlike prior kinematics or dynamics-based approaches where the two
components are used disjointly, we synergize the two approaches via
dynamics-regulated training. At each timestep, a kinematic model is used to
provide a target pose using video evidence and simulation state. Then, a
prelearned dynamics model attempts to mimic the kinematic pose in a physics
simulator. By comparing the pose instructed by the kinematic model against the
pose generated by the dynamics model, we can use their misalignment to further
improve the kinematic model. By factoring in the 6DoF pose of objects (e.g.,
chairs, boxes) in the scene, we demonstrate for the first time, the ability to
estimate physically-plausible 3D human-object interactions using a single
wearable camera. We evaluate our egocentric pose estimation method in both
controlled laboratory settings and real-world scenarios.
