{'fraction_non_alphanumeric': 0.03865556621583981, 'fraction_numerical': 0.00999659558664232, 'mean_word_length': 4.776188773686092, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Motion Deblurring with Real Events

In this paper, we propose an end-to-end learning framework for event-based motion deblurring in a self-supervised manner, where real-world events are exploited to alleviate the performance degradation caused by data inconsistency. To achieve this end, optical flows are predicted from events, with which the blurry consistency and photometric consistency are exploited to enable self-supervision on the deblurring network with real-world data. Furthermore, a piece-wise linear motion model is proposed to take into account motion non-linearities and thus leads to an accurate model for the physical formation of motion blurs in the real-world scenario. Extensive evaluation on both synthetic and real motion blur datasets demonstrates that the proposed algorithm bridges the gap between simulated and real-world motion blurs and shows remarkable performance for event-based motion deblurring in real-world scenarios.


Introduction
Due to motion ambiguities as well as the erasure of intensity textures [12], the task of motion deblurring is severely ill-posed [11,25]. With the help of an event camera which can "continuously" emit events asynchronously with extremely low latency (in the order of µs) [15,6], inherently embedding motions and textures [2], the task of motion deblurring [24,22,28] can be essentially alleviated. Many event-based motion deblurring methods have been proposed by learning from synthesized dataset composed of simulated events and blurry images as well as sequences of sharp clear ground-truth images [10,31]. However, the inconsistency between synthetic and real data degrades the performance of inference on real-world event cameras [29].
The physical intrinsic noise of event cameras raises the Figure 1. Illustrative example of inconsistency between synthesized and real-world motion blurs with respect to the event timesurface [14]: (a) a real-world motion blurred image; (b) the timesurface of real-world events corresponding to (a); (c) the timesurface of events collected with the same trajectory as (a) but at a slow motion speed; (d) deblurred results respectively by eSL-Net [31] (top row), LEDVDI [16] (middle row) and our proposed RED-Net (bottom row) that trained with real-world events and real-world motion blurs, sequence predictions of the blue area are shown on the right. Our proposed RED-Net generates less halo artifacts and achieves the best visualization performance.
difficulty of simulating labeled events that highly match the real event data [26]. Even though the event simulator to some extent reduces the gap by considering the pixel-topixel variation in the event threshold [26], additional noise effects such as background activity noise and false negatives [1] still exist, leading to tremendous discrepancy between the virtual events synthesized from event simulators [26] and the real events emitted by event cameras. An alternative approach is to build a labeled dataset composed of real-world events accompanying with synthesized blurry images, and then train networks on it [10]. Unfortunately, obtaining such pairs is not always easy, which needs to be captured with a slow motion speed as well as under good lighting conditions to avoid motion blur. Subsequent blurry image synthesis and alignment on temporal domain is also tedious but indispensable. Furthermore, inconsistency still exists between the events associated with synthesized and real-world motion blur in that limited read-out bandwidth leads to more event timing variations, as shown in Fig. 1.
In this paper, we propose a novel framework of learning the event-based motion deblurring network in a selfsupervised manner, where real-world events and real-world motion blurred images are exploited to alleviate the performance degradation caused by data inconsistency and bridge the gap between simulations and real-world scenario. The proposed framework consists of two neural networks, an event-based motion deblurring network (Deblur-Net) and an event-based optical flow estimation network (OF-Net). The Deblur-Net is fed with both events and a single motion blurred image, and outputs a sequence of sharp clear images, while the OF-Net receives events and provides motions between the reconstructed sharp clear images. We relate motions and sharp clear images according to the photometric constancy [36]. The real-world motion blurs with non-linearities are considered by a piece-wise linear motion (PLM) model which improves the accuracy of optical flow and thus provides a more precise blurring model between the reconstructed intensity images from Deblur-Net and the blurry input. The overall network is jointly trained endto-end over partially labeled dataset composed of synthetic data using ESIM [26] with ground-truth sharp clear images and real-world data only containing real-world events and real-world motion blurred images, which can be captured simultaneously by the DAVIS [3] or with a dual camera set connecting the event camera and the RGB camera with a beam splitter [32].
The main contributions of our work are in three folds: • We propose a framework of learning the event-based motion deblurring network with real-world events, which remarkably improves the performance of motion deblurring in the real-world scenario.
• We propose a piece-wise linear motion model to consider the motion nonlinearity, based on which the OF-Net is ameliorated for real events and outputs accurate and dense motion flows.
• Extensive experiments show that the proposed method can yield high quality sharp frames and achieve stateof-the-art results on the real blurry event dataset.

Related Work
Motion Deblurring. Even though motion blur degenerates photographs by averaging over the exposure time intervals, it inherently embeds both motions and textures of the moving objects and thus enables the possibility of motion deblurring for visualization of the dynamic scene behind blurred photographs [12,11,25]. Early attempts address this problem by assuming spatially uniform motion where blurs can be modelled as the convolution of a blur kernel with a sharp image, leading to the kernel-based motion deblurring approaches by means of regularized deconvolution [13,30]. For motion blurs caused by complex motion behaviors, pixel/patch-wise motion flows are often required to flexibly depict non-uniform motions. To achieve this end, the optical flow is often estimated from a single blurry image [7] or consecutive frames [34] and provides photometric constancy [8] between the recovered latent sharp images, leading to flow-based motion deblurring approaches. Motion blurs bring the ambiguity of temporal ordering as well as the erasure of spatial textures [12], thus several priors have been successfully employed for blur kernels, e.g., sparsity [33], Gaussian scale mixture [5], and dark channel [21], and for optical flows, e.g., linear motion [17], and order invariance [12], which however may generate artifacts and degrade deblurring performance if the above assumptions are not fulfilled [35]. An alternative approach is to learn a neural network for motion deblurring directly from the data, which often achieves prominent performance [12,11,25,7,20]. Jin et al. pioneer to recover a sequence of sharp frames from a single motion-blurred image [12], where they sequentially train multiple neural networks with a temporal invariant loss. Purohit et al. utilize a single recurrent neural network to generate the entire sequence [25] which, however, may suffer from ambiguities in temporal ordering. Take this into account, Jin et al. tackle the problem of the temporal ambiguity by feeding multiple motion blurred images [11], while Rengarajan et al. additionally use two consecutive images over a short exposure time [27]. On the other hand, Chen et al. propose a Reblur2Deblur network [4] for motion deblurring via self-supervised learning to leverage the physical model and the data prior, where multiple blurry images and a linear motion assumption between them are essentially required. Event-Based Motion Deblurring. Event camera measures per-pixel brightness change and outputs an event once the change exceeds a threshold [6]. The triggered events can be "continuously" emitted asynchronously with extremely low latency and thus provide missing information during exposure intervals if the motion blur happens [24]. The temporal ambiguity and texture erasure can be easily tackled by introducing events into the deblurring algorithms [24,16]. Event-based motion deblurring methods can be categorized into two groups, i.e., model driven and data driven algorithms. Model-based algorithms relate events, blurry images and the corresponding latent sharp clear images according to the physical event generation principle [24,23,28]. However, due to the imperfections of physical implementation including intrinsic noise and limited readout bandwidth [9,6], real events are essentially with noise both in temporal and spatial domains [15] which inevitably degrades performance [28].
Data driven algorithms relax the above limitations by utilizing neural networks and directly learn the relation from a blurry image to a sequence of sharp clear images with the aid of events [16,31,10]. For training purpose, a synthesized dataset composed of labeled events and blurry images is commonly simulated from sharp clear video sequences [10,31] which, however, may have inconsistency to the real events due to event noise in the spatio-temporal domain. Even though data variations have been considered by manually adding noise [31,29], the generalizability remains limited for real event cameras.
Jiang et al. [10] build the Blur-DVS dataset based on real events triggered at a slow motion speed (to obtain a sequence of sharp clear intensity images as the ground truth) and train their network on it, where the sim-to-real gap is reduced to some extent [16]. However, building such a dataset is strenuous and requires strict conditions to obtain blur-free ground-truths, e.g., relative slow motion between camera and scenario and good lighting environment. Besides, due to the limited read-out bandwidth, the inconsistency between events emitted at a slow motion speed and their counterparts at a fast motion speed also exists.
Learning with real-world data is more adaptive to the real-world scenario than synthetic data, and moreover, realworld events and motion blurs can be easily captured at extremely low cost without sophisticated procedures, which motivates us to propose a new framework to learn event-based motion deblurring by leveraging the real-world events and real-world motion blurred images.

Problem Statement
Event-based motion deblur (ED) aims at reconstructing a sequence of sharp and clear latent images {I t } t∈T from a single motion blurred image B captured with exposure time T and corresponding events E T ≜ {(x i , p i , t i )} ti∈T triggered in T where t i and x i respectively denote the timestamp and the pixel location of the i-th event, and p i ∈ {+1, −1} is the polarity. Many algorithms are proposed to tackle the problem of ED by learning-based approaches [16,10,31], i.e., which are commonly trained over a synthetic dataset Real-World Events. Real-world eventsẼ T k are different from the simulated eventsÊ T k in two aspects: (1) event noises in spatial domain are principally induced by physical intrinsic camera imperfections (etc., variant event threshold), background activities, false negatives, etc., [6]; (2) event noises in temporal domain are produced owing to the limited read-out bandwidth and bring timing variances [9], which commonly exist at a fast motion speed when the motion blur happens. Statistics of noises from both aspects are too complicated to be addressed in current event simulators [26], leading to inconsistency between synthetic dataset D s and its real-world counterparts.
For spatial event noises, [16] has built a real event dataset D e ≜ {B k ,Ẽ T k ,G k } k composed of the real-world events E T k , the real-world sharp clear ground-truth imagesG k captured at an ultra slow speed to avoid motion blurs, and synthesized blurry imagesB k = Avg(G k ). Theoretically, training on D e is able to take into account the spatial noises, while the temporal noises cannot be considered since they always accompany with real-world motion blurs where no ground-truth images can be obtained. It raises the problem of learning on unlabeled real-world dataset D r ≜ {B k ,Ẽ T k } k , with real-world blurry imagesB k and real-world eventsẼ T k . Real-World Motions. Motions and intensity textures are coupled together based on the photometric consistency and thus commonly exploited for the motion deblur [11,27]. For simplicity, most existing motion deblur algorithms assume linear motion during the exposure time T [23,4], where the motion flow stays constant. However, it is not always correct for dynamic scenes since real-world motions might be nonlinear especially at a fast motion speed.
Thus the main obstacles of leveraging the power of events to learn motion deblurring in real world are in two folds. First, the network should be trained on real-world data with real events and real motion blurs. Second, the motion non-linearity should be considered to deal with motion blurs in real-world complex dynamic scenes.

Method
Existing event-based deblurring methods are usually developed within a supervised learning framework, of which performance is limited to the specific training data (simulated events or synthesized blurred image from the dataset captured with slow motion). Taking into account both deblurring and the physical image blur formation process, we construct a semi-supervised learning framework which can generalize well to real-world motion-blurred images caused by fast motion. Fig. 2 illustrates the overall architecture of our framework. The Deblur-Net takes a single motion-blurred image B and its event data E as input and outputs a sequence of sharp frames {I m } M −1 m=0 , where M is the number of estimated sharp frames. The supervised loss that compares the estimated sharp frames with ground truth only works for labeled synthesized dataset. To guide the semi-supervised learning of Deblur-Net, our framework consists of an additional OF-Net for motion estimation, which takes event data E as input and outputs optical flow serving as motion information. On the strength of the physical blur formation process, a motion-blurred image can be re-rendered on the basis of estimated sharp images and motion information, which provides a blur-consistency constraint by comparing with the original motion-blurred image. Besides, estimated sharp images are propagated by the estimated optical flow, which provides an additional constraint of photometricconsistency by comparing the propagated sharp images with the directly recovered ones.
Considering the motion non-linearity in real-world complex dynamic scenes, we take full advantage of high temporal resolution of event camera. Since event data is able to provide continuous motion information in a period, we do not need strong assumption on linear constant motion in a large time interval as other deblurring methods [7,23,4,17]. Event data can be used in modeling highly nonlinear motion happened in the exposure time.

Blur-consistency Constraint
Providing the latent sharp image L t (x) during the exposure time interval T , the blurring process can be physically formulated as the average of them in the duty cycle, where {L n } N −1 n=0 is the discrete version of L t and a large enough N ≫ M is required to achieve small discretization error. Providing recovered sequences {I m } M −1 m=0 output from the Deblur-Net, one can resolve L n by interpolating with {I m } M −1 m=0 over the motion field u n (x) representing per-pixel flow from L 0 to L n i.e., L n (x) = L 0 (x + u n (x)). ( For simplicity, a linear motion (LM) model is commonly assumed [17,23,7], i.e., with v denoting the per-pixel unit motion flow which is assumed to be constant during the exposure time interval T . To consider the motion non-linearity for real-world scenario, we propose a piece-wise linear motion (PLM) model by making full use of rich motion information encoded in events. We divide event stream within the exposure time T into M − 1 equal time intervals and the LM model is assumed within each interval. Finally, we can get the following PLM model, for n ∈ [mK, (m + 1)K), with output from the Deblur-Net, the n-th latent image L n with n ∈ [mK, (m + 1)K) can be resolved by warping the m-th frame I m using the motion field defined in (5), Hence, we can re-render a corresponding reblurred image based on (2) using the generated latent sharp images {L n } N −1 n=0 . A blur-consistency loss is constructed between the reblurred imageB and the original blurry input B: By imposing this loss in the whole framework, we build a bridge between the motion-blurred image and its corresponding sharp clear images, which provides a selfsupervision to the network. For real-world motion blurs with motion non-linearities, PLM is important to ensure the model accuracy and thus provide faithful supervision even with noise disturbances.

Photometric-consistency Constraint
In addition to the blur-consistency constraint on the sequence level, further photometric-consistency constraint is imposed, which utilizes the inter-frame connection between the temporal consistency of estimated sharp images and motion information encodes in events. We warp I m+1 to I m with the predicted optical flow v m via the OF-Net by feeding events in the m-th time interval and obtain the warped image:Ī m (x) = I m+1 (x + Kv m (x)).
Then photometric-consistency loss is constructed by comparing the propagated sharp images with the recovered ones directly predicted by Deblur-Net:

Optimization
We propose to train the Deblur-Net and OF-Net over a partially labeled dataset, composed of the synthetic dataset D s with ground-truth sharp clear images and the real-world dataset D r without ground truth. Apparently, the blurconsistency and the photometric-consistency are applicable for both D s and D r , and provide self-supervised losses respectively defined as L s blur , L s photo and L r blur , L r photo according to (7) and (9). Besides, for the synthetic dataset D s with a sequence of ground-truth sharp clear imagesG, we use it to supervise the network via a reconstruction error loss, where G m ∈G. Thus the overall function is as follows: L = L s error +αL s blur +βL s photo +γL r blur +δL r photo , (11) with α, β, γ and δ denoting the balancing parameters.

Experimental Settings
Dataset. The proposed Real-world Event-based motion Deblurring network (RED-Net) is trained in a self-supervised manner, where one synthetic dataset (GoPro) is provided for training with ground-truth and two datasets respectively captured by a DAVIS240C camera (HQF provided in [29] containing real events but blur-free intensity frames) and a DAVIS346 camera (RBE built in this paper for the realworld scenario with real events and real motion blurs). GoPro: Based on the GoPro dataset [18], we build a synthetic dataset as [31] composed of simulated events and synthetic blurry images as well as sharp clear ground-truth images. We first increase the frame rate by interpolating 7 images between consecutive frames [19] and then generate both events and blurry images based on the interpolated high frame-rate sequences. ESIM [26] is exploited to simulate events with consideration of per-pixel threshold variation. And blurry images are simply obtained by averaging over 49 consecutive images.
HQF: We construct a similar dataset as Blur-DVS [16] based on HQF dataset [29], which contains real-world events and sharp clear ground-truth frames captured simultaneously from a DAVIS240C that are well-exposed and minimally motion-blurred. Providing the ground-truth frames, motion blurs can be synthesized using the same manner as the GoPro dataset. Finally, the HQF dataset contains real-world events and synthesized blurry images as well as the ground-truth frames. The ground-truth frames are provided only for quantitative evaluation. In training stage of our proposed RED-Net over the HQF dataset, only real events and synthesized blurry images are used.
RBE: A Real-world Blurry images and Events (RBE) dataset is built with a DAVIS346 camera and only contains real-world events and real-world motion blurs, which can be collected in a facilitated manner. Thus the RBE dataset can provide a large number of paired real-world data, which can be fed into the RED-Net and improve the adaptivity to the real-world scenario. Implementation Details For the Deblur-Net and OF-Net modules, we take advantage of existing neural network architectures which have performed well in the past for the respective learning tasks. Inspired by that a sharp image can be mapped from a blurry image using a residual term encoded with events [24], we select a residual network to predict a sequence of sharp frames from a single motionblurred image and its event data. In particular, we adopt the residual network from Jin et al. [11]. For the OF-Net, we adopt the EV-FlowNet architecture from Zhu et al. [36], which is widely used to predict optical flow from events.
Our network is implemented using Pytorch on a single NVIDIA Geforce RTX 3090 GPU. During training, we randomly crop the samples into 128 × 128 patches. Adam optimizer is used and the maximum epoch of training iterations is set to 30. The learning rate starts at 10 −4 , then decays by 25% every five epochs from the 15-th epoch. The weighting factors α, β, γ and δ are all set to 1, the number of deblurred images {I m } M −1 m=0 and latent sharp images L n warped from I m are respectively M = 7 and K = 11. The optical flow module is pretrained on the Multi-Vehicle Stereo Event Camera dataset (MVSEC) [36], where the reference ground-truth optical flow is provided. RED-Nets. To validate the effectiveness of exploiting realworld data, RED-Net is trained respectively over three different datasets, i.e., GoPro, GoPro+HQF and GoPro+RBE, and final networks are respectively denoted as RED-GoPro, RED-HQF and RED-RBE. Specifically, RED-GoPro is trained only over synthesized GoPro dataset with the supervision of ground-truth frames. Both RED-HQF and RED-RBE are trained in a self-supervised manner, where RED-HQF uses real-world events but synthetic motion blurs, while RED-RBE uses real-world events and motion blurs.
To validate the superiority of piece-wise linear motion model (PLM), we replace it with the linear motion model (LM) as (4), and train RED-Net respectively on GoPro and GoPro+RBE datasets. Specifically, the resulted network is denoted as RED-(GoPro/RBE)-LM and its counterpart with PLM is denoted as RED-(GoPro/RBE)-PLM. By default, RED-(GoPro/RBE) means RED-(GoPro/RBE)-PLM in the following sections. given. Apparently, the output of OF-Net from RED-RBE trained on real data achieves the best performance with consistent motion and sharp clear edge. Thus the deblur image of RED-RBE has the best qualitative appearance as well.

Results of Optical Flow
To reveal the advantage of PLM, we compare the outputs of OF-Net from RED-GoPro-LM as shown in Fig. 3(b) and RED-GoPro-PLM as shown in Fig. 3(c). It is obvious that RED-GoPro-PLM gives better OF result with clear edges and stable background while RED-GoPro-LM suffers from blurry effects and background movement. Furthermore, when violating the linear motion assumption, the result of OF-Net from RED-GoPro-LM is even worse than its origin, i.e. EvFlowNet as shown in Fig. 3(a) and 3(b).
Besides, we compare the OF results of EvFlowNet and two OF-Nets respectively from RED-GoPro-PLM and RED-RBE as shown in Fig. 3(a), 3(c) and 3(d). With the supervision of blurry image and the intensity sequence according to the blur-consistency loss (7), OF-Nets from both RED-GoPro-PLM and RED-RBE can give better results than EvFlowNet. It reveals that the motion information embedded in the blurry images helps the prediction of OF within our framework. Moreover, OF-Net from RED-RBE trained over the real dataset outperforms that of RED-GoPro-PLM trained over the synthetic dataset and gives more consistent motions of the cube surface as shown in Fig. 3(c) and 3(d). And it validates the advantage and necessity of training on real-world dataset.

Comparisons with State-of-the-art Methods
Since our method is able to remove motion blur and reconstruct a sequence of latent sharp clear images, the deblur performance is evaluated on both single frame and sequence reconstructions. We compare the proposed RED-Nets to state-of-the-art conventional deblurring methods including blur2mflow [7] and LEVS [12], and event-based motion deblurring methods including EDI [24], eSL-Net [31] and LEDVDI [16]. Due to the lack of ground-truth for realworld motion blurs, we exploit the HQF dataset constructed from blur-free frames for quantitative evaluations, where the effectiveness of training with real-world events are validated. The quantitative results are presented in Tab.1 and show that the proposed RED-Nets bring remarkable improvements comparing to the state-of-the-arts, on average 3.5 dB gain on single frame prediction and 2.5 dB gain on sequence prediction over the HQF dataset with real-world events. Detailed analyses are presented in the following. Inconsistency of Simulated and Real-world Events. We first compare the deblurring performance of conventional methods, i.e., blur2mflow and LEVS to event-based methods, i.e., EDI and eSL-Net, over the synthetic GoPro dataset and the HQF dataset with real-world events. It is shown that EDI provides comparable performance to learning based conventional methods with the help of events. For the learning based approaches, eSL-Net enhanced with simulated events outperforms blur2mflow and LEVS by a large margin on the synthetic GoPro dataset. However, the overwhelming performance of eSL-Net is not retained on the HQF dataset with real-world events, which reflects the inconsistency of simulated and real-world events.
Effectiveness of Training with Real-world Events. Comparing to RED-GoPro trained only on synthetic dataset, the other two RED-Nets, i.e., RED-HQF and RED-RBE, trained respectively over the HQF and RBE dataset with real events, bring a performance drop on the synthetic GoPro dataset but achieve the PSNR gain on the HQF dataset with real-world events. Specifically, RED-HQF obtains on average 1.1 dB gain on PSNR, which shows the effectiveness of As shown in Tab. 1, RED-HQF/RBE outperforms EDI and eSL-Net over the HQF dataset with real-world events by a large margin. Furthermore, we exploit the recently proposed network, i.e., LEDVDI trained with real-world events for fair comparisons. Different from our proposed method, LEDVDI is trained with real-world events in a supervised manner. Our approach requires neither blurry synthesis nor strict slow moving speed to collect blur-free groundtruth images, and thus is much easier to be adopted in realworld scenario. From the quantitative results in Tab. 1, it is shown that both RED-HQF and RED-RBF achieve better performance than LEDVDI for single frame prediction and sequence prediction, which validates the superiority of our proposed framework. Note that LEDVDI deals with single frame prediction and sequence prediction separately with different networks, while our proposed RED-HQF/RBE tackle both problems in one network. Real-World Motion Deblur. The above quantitative comparisons are all based on synthesized motion blurs. Due to the lack of ground-truth frames, the performance with real-world motion blurs are qualitatively investigated, as shown in Fig. 4. We choose two scenes captured manually with a DAVIS346 camera, i.e., labfloor and window. Without the aid of events, blur2mflow and LEVS fail to recover sharp clear images. For event-based methods, EDI fails to cope with severe motion blurs as illustrated in window. Among the learning based approaches, RED-RBE achieves the best visualization performance, while the others are likely to give over smoothed effects or generate halo artifacts around black edges, for instance the bottom of garbage can in labfloor and the top-left corner of TV in window. Compared with RED-HQF and LEDVDI trained with synthesized motion blurs, RED-RBE is trained with real-world motion blurs which can alleviate the problem of event noise in temporal domain, e.g., timing chaos caused by limited read-out bandwidth and thus give less halo artifacts.

Ablation Study
The proposed RED-Net improves the performance of motion deblurring by self-supervising with real-word events. To achieve this end, optical flows are predicted from events with which the piece-wise linear motion model (PLM) is proposed to accurately model the blurry procedure even with motion non-linearities. To find out what contributes to the superior performance of the approach, we compare a few variants with and without use of synthesized events (SynEv), blurring with LM (LM), blurring with PLM (PLM) and real-world events (ReEv), as shown in Tab. 2. From the table, we can draw the following conclusions: Importance of Events. We validate the importance of events by training the Deblur-Net without events over the synthetic GoPro dataset, and the quantitative deblur results over 2 dataset are shown in the first row of Tab. 2. Comparing to RED-Nets with events, there is a large performance gap. It shows that the rich motion information encoded in events can effectively improve the deblurring performance. Linear vs. piece-wise Linear Motion Model. With the optical flow output of the OF-Net, motions provide the interframe relationship inside the reconstructed sequence as well as the blurry consistency which relies on the physical blurry procedure. Inaccurate motion model will violate the blurry consistency and thus degrades the deblurring performance.  Figure 4. Qualitative results of motion deblur for 2 blurry scenes by 7 different methods where the top-two rows correspond to labfloor and the bottom-two correspond to window. For each scene, from top-left to bottom-right are respectively the blurry image and its deblurred result by blur2mflow [7], LEVS [12], EDI [24], eSL-Net [31], LEDVDI [16] and our proposed RED-HQF and RED-RBE.
in (2) and their performance can be referred from Tab. 2 in the 3rd, 4th, 5th and 6th rows. Apparently, RED-Nets with PLM outperform that with LM and achieve PSNR gains up to 0.5 dB on the real-world dataset after training with realworld events, while it only gains 0.16 dB without real-world events. It demonstrates that PLM plays an important role especially for real-world events.
Synthetic vs. Real-world Events. To validate this point, we train RED-Net respectively on the synthetic GoPro dataset, i.e., RED-GoPro and on the real-world RBE dataset, i.e., RED-RBE. Their results are shown in the 4th and 6th rows of Tab. 2. And we also trained RED-GoPro and RED-RBE with LM and present their results respectively in the 3rd and 5th rows of Tab. 2. By comparisons, we can easily find that the self-supervision with real-world events can achieve PSNR gains 1.6 dB with PLM (1.3 dB with LM) on realworld dataset. It shows that the supervision of real-world events plays a major role in improving the deblurring per-formance on real-world dataset.

Conclusion
A self-supervised learning framework for event-based motion deblurring is proposed where real-world events and real-world motion blurs are exploited to alleviate the performance degradation caused by data inconsistency. Motion non-linearities are also considered through a PLM model to improve the accuracy of physical blur-consistency. Within the proposed learning framework, we evaluate RED-Net over different datasets and validate the effectiveness of the proposed PLM-based blur-consistency and photometricconsistency over the real-world dataset. With a blurry image and corresponding events, the proposed RED-Net can produce a sequence of sharp clear intensity images as well as motion flows between them. Extensive experiments demonstrate that the proposed method can achieve the stateof-the-art with real-world events.