Modality Bank: Learn multi-modality images across data centers without
  sharing medical data
  Multi-modality images have been widely used and provide comprehensive
information for medical image analysis. However, acquiring all modalities among
all institutes is costly and often impossible in clinical settings. To leverage
more comprehensive multi-modality information, we propose a privacy secured
decentralized multi-modality adaptive learning architecture named ModalityBank.
Our method could learn a set of effective domain-specific modulation parameters
plugged into a common domain-agnostic network. We demonstrate by switching
different sets of configurations, the generator could output high-quality
images for a specific modality. Our method could also complete the missing
modalities across all data centers, thus could be used for modality completion
purposes. The downstream task trained from the synthesized multi-modality
samples could achieve higher performance than learning from one real data
center and achieve close-to-real performance compare with all real images.
