{'fraction_non_alphanumeric': 0.01861547411285631, 'fraction_numerical': 0.0017452006980802793, 'mean_word_length': 6.166666666666667, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Single-shot hyperspectral imaging based on dual attention neural network with multi-modal learning.

Hyperspectral imaging is being extensively investigated owing to its promising future in critical applications such as medical diagnostics, sensing, and surveillance. However, current techniques are complex with multiple alignment-sensitive components and spatiospectral parameters predetermined by manufacturers. In this paper, we demonstrate an end-to-end snapshot hyperspectral imaging technique and build a physics-informed dual attention neural network with multimodal learning. By modeling the 3D spectral cube reconstruction procedure and solving that compressive-imaging inverse problem, the hyperspectral volume can be directly recovered from only one scene RGB image. Spectra features and camera spectral sensitivity are jointly leveraged to retrieve the multiplexed spatiospectral correlations and realize hyperspectral imaging. With the help of integrated attention mechanism, useful information supplied by disparate modal components is adaptively learned and aggregated to make our network flexible for variable imaging systems. Results show that the proposed method is ultra-faster than the traditional scanning method, and 3.4 times more precise than the existing hyperspectral imaging convolutional neural network. We provide theory for network design, demonstrate training process, and present experimental results with high accuracy. Without bulky benchtop setups and strict experimental limitations, this simple and effective method offers great potential for future spectral imaging applications such as pathological digital stain, computational imaging and virtual/augmented reality display, etc.