MESH2IR: Neural Acoustic Impulse Response Generator for Complex 3D Scenes
We propose a mesh-based neural network (MESH2IR) to generate acoustic impulse
responses (IRs) for indoor 3D scenes represented using a mesh. The IRs are used
to create a high-quality sound experience in interactive applications and audio
processing. Our method can handle input triangular meshes with arbitrary
topologies (2K - 3M triangles). We present a novel training technique to train
MESH2IR using energy decay relief and highlight its benefits. We also show that
training MESH2IR on IRs preprocessed using our proposed technique significantly
improves the accuracy of IR generation. We reduce the non-linearity in the mesh
space by transforming 3D scene meshes to latent space using a graph convolution
network. Our MESH2IR is more than 200 times faster than a geometric acoustic
algorithm on a CPU and can generate more than 10,000 IRs per second on an
NVIDIA GeForce RTX 2080 Ti GPU for a given furnished indoor 3D scene. The
acoustic metrics are used to characterize the acoustic environment. We show
that the acoustic metrics of the IRs predicted from our MESH2IR match the
ground truth with less than 10% error. We also highlight the benefits of
MESH2IR on audio and speech processing applications such as speech
dereverberation and speech separation. To the best of our knowledge, ours is
the first neural-network-based approach to predict IRs from a given 3D scene
mesh in real-time.