Model-Architecture Co-Design for High Performance Temporal GNN Inference
  on FPGA
  Temporal Graph Neural Networks (TGNNs) are powerful models to capture
temporal, structural, and contextual information on temporal graphs. The
generated temporal node embeddings outperform other methods in many downstream
tasks. Real-world applications require high performance inference on real-time
streaming dynamic graphs. However, these models usually rely on complex
attention mechanisms to capture relationships between temporal neighbors. In
addition, maintaining vertex memory suffers from intrinsic temporal data
dependency that hinders task-level parallelism, making it inefficient on
general-purpose processors. In this work, we present a novel model-architecture
co-design for inference in memory-based TGNNs on FPGAs. The key modeling
optimizations we propose include a light-weight method to compute attention
scores and a related temporal neighbor pruning strategy to further reduce
computation and memory accesses. These are holistically coupled with key
hardware optimizations that leverage FPGA hardware. We replace the temporal
sampler with an on-chip FIFO based hardware sampler and the time encoder with a
look-up-table. We train our simplified models using knowledge distillation to
ensure similar accuracy vis-\'a-vis the original model. Taking advantage of the
model optimizations, we propose a principled hardware architecture using
batching, pipelining, and prefetching techniques to further improve the
performance. We also propose a hardware mechanism to ensure the chronological
vertex updating without sacrificing the computation parallelism. We evaluate
the performance of the proposed hardware accelerator on three real-world
datasets.
