{'fraction_non_alphanumeric': 0.02070285316631872, 'fraction_numerical': 0.00527719786592438, 'mean_word_length': 4.843781768891901, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Device for ECG prediction based on retinal vasculature analysis

Pulsatile changes in retinal vascular geometry over the cardiac cycle have clinical implications for the diagnosis of ocular and systemic vascular diseases, including ischemia, coronary heart diseases, and diabetes mellitus and its complications. Thus, analysis of the pulsatile changes over the cardiac cycle is a potential non-invasive assessment for the presence of ocular and systemic vascular diseases. The cardiac rhythm influences these pulsatile changes in the retina, is a result of the change in blood volumetric flow entering the ophthalmic-vascular system under a certain level of intraocular pressure during the peak systolic and diastolic phases of the cardiac cycle. Assessment of fundus images generally requires ophthalmologic expertise. However, the availability of an expert is not always guaranteed, and even if an expert is available, the assessment is performed manually. Thus, there is also a need for an automated device that can analyze the fundus images and keep track of the pulsations. Such a device can be synchronized with the Electrocardiogram and can be programmed to assess the presence of various ocular and systemic vascular diseases. In this project, we proposed and worked on a portable embedded system device that automatically captures retinal images and analyzes them for the estimation of vessel diameters. The device is hand-held and attaches to the ophthalmoscope as an extension. Using the device, the time variation of the vessel diameters can be estimated, which can further be used to predict ECG and the presence of various other systemic vascular diseases.


INTRODUCTION
Pulsatile changes in retinal vascular geometry over the cardiac cycle have clinical implications for the diagnosis of ocular and systemic vascular diseases, including ischemia, coronary heart diseases, and diabetes mellitus and its complications. Thus, analysis of the pulsatile changes over the cardiac cycle is a potential non-invasive assessment for the presence of ocular and systemic vascular diseases. The cardiac rhythm influences these pulsatile changes in the retina, is a result of the change in blood volumetric flow entering the ophthalmic-vascular system under a certain level of intraocular pressure during the peak systolic and diastolic phases of the cardiac cycle. An example of pulsatile change observable in the retina is the spontaneous venous pulsation (SVP), which is available in approximately 90 percent of the patients. It is caused by the variation in the pressure gradient between the intraocular retinal veins and the retrolaminar portion of the central retinal vein, visible as rhythmic changes in the diameter of one or more veins near or on the optic nerve head. In addition to SVP, pulsation of veins outside the optic disk (OD), such as the serpentine movement of principal arteries, the pulsatile motion of small arterioles, and movement of optic nerve head are other features that can be visualized with dynamic fundoscopy.
Assessment of fundus images generally requires ophthalmologic expertise. However, the availability of an expert is not always guaranteed, and even if an expert is available, the assessment is performed manually. Thus, there is also a need for an automated device that can analyze the fundus images and keep track of the pulsations. Such a device can be synchronized with the Electrocardiogram and can be programmed to assess the presence of various ocular and systemic vascular diseases. In this project, we proposed and worked on a portable embedded system device that automatically captures retinal images and analyzes them for the estimation of vessel diameters. The device is hand-held and attaches to the ophthalmoscope as an extension. Using the device, the time variation of the vessel diameters can be estimated, which can further be used to predict ECG and the presence of various other systemic vascular diseases.

STRUCTURE OF THE DEVICE
The device is integrable with an ophthalmoscope and acts as an extension to the ophthalmoscope. The functions performed by it are to capture and sample the retinal images, process them for vessel diameter estimation and output the time variation of those estimations, which are to be used further for ECG prediction.

COMPONENTS AND THEIR FUNCTIONS
A Raspberry Pi processor was used as the processing part of the device, whose function is to initiate the image capturing and process the images for vessel diameter estimation. The main sensing element of the device is the camera, and the Raspberry Pi camera module provided a Sony IMX219 8-megapixel sensor for the device which can support 1080p 30, 720p 60, and VGA 90 video modes. The device also needed an input-output tool, and a 3.5 inch LCD screen was chosen to be pertinent with the device. Apart from the above main parts, various secondary parts like connecting battery, connecting wires, and screws were used. Details of the components are also summarised in Table 1. All the components mentioned above work hand-in-hand.
The ophthalmoscope focuses on the retina, and the Raspberry Pi camera captures the retina's focussed image. The images are passed on to a Raspberry Pi powered by a battery. The Raspberry Pi processes the images and tracks the time variations in the vessel diameters. The processed output is passed on to the user via the LCD screen. The device can also be used synchronized with the ECG to visualize retinal pulsations. For this, a portable ECG machine is used to continuously capture the ECG and passes it to the Raspberry Pi. The processor uses those measurements to detect peaks and trigger image capture at the ECG's specified peaks. The images are then processed to obtain the time variation of vessel diameters, which can be used to study the pulsations. The block diagram is shown in Figure 1

3D MODEL
All the components will function only when there is harmony in their physical arrangement, and that is where modeling the device comes into the picture. The 3D model of the device was made using the Autodesk Inventor software. The model was designed to provide space for every component and the connections between the components. Ports like USB, HDMI, and micro USB were incorporated into the model. A unique cylindrical protrusion was designed to attach to the eye-piece of the ophthalmoscope and thus provide a pseudo-continuum between the ophthalmoscope and the device. The stability of the model was also taken into consideration, and support was designed to provide the device robustness when connected with an ophthalmoscope. The 3D model and the internal breakout are shown in the

WORKING OF THE DEVICE
The Raspberry Pi triggers image capture when input is given through the LCD screen. The captured image is processed for the estimation of vessel diameters. The state of the art algorithms for diameter estimation generally use Convolutional Neural Networks. The dataset these algorithms are trained on is DRIVE, STARE, and REVIEW. These datasets offer high-quality retinal images but because the device needs to work for images captured through an ophthalmoscope which, are of much lower quality. Using such supervised algorithms offer minimal accuracy when used on images captured through an ophthalmoscope because the distribution of the testing images is entirely different from that of the training images. Thus, an unsupervised approach is used in this project. Most unsupervised algorithms have a segmentation step before the diameter estimation step. The segmentation step segments the blood vessels from the background, and this segmented vessel map is further used to estimate the diameters. Estimating the diameters can be done in various ways; one of the ways is to track the vessel edges and evaluate the shortest distance to the other parallel edge, for each point on the vessel edge. The other approach involves finding vessel centrelines and clustering the pixels perpendicular to the centreline based on their intensities; the vessels offer high contrast with the background and thus correspond to the low-intensity cluster. We used the latter approach.
The processing thus involves two steps: 1. Vessel segmentation and centreline extraction 2. Interpolation normal to the centrelines and clustering

Vessel segmentation and centreline extraction
The segmentation step is crucial to locate the vessels in the image. Most segmentation algorithms are supervised in nature and use architectures like RetinaNet trained on the DRIVE and STARE dataset. Again, these algorithms are not suitable for images captured through an ophthalmoscope because of the low field of view and low quality of the ophthalmoscopic images. Various unsupervised algorithms for segmentation based on match filtering, global thresholding, and morphological operations, etc., can be found in the literature. All the unsupervised algorithms generally found to attain almost similar accuracies on the DRIVE and STARE dataset. We have chosen a global thresholding based algorithm for segmentation. As most studies have shown that the green channel offers the most contrast with the background, the green channel of the image was selected, and to further enhance the image Contrast Limited Adaptive Histogram Equalization (CLAHE) was applied. To obtain a field of view(FOV) mask for the image, thresholding was used with a low threshold. To counter the boundary effects of processing the FOV mask was morphologically eroded using an elliptic structuring element. The CHALE was applied using a grid of size 9x9, and the contrast clip limit value of 3.0. Background subtraction is further carried out to extract the features from the background. The background was estimated by applying a small median filter (size=5) and a large Gaussian filter (size=55) on the enhanced image. The enhanced image was then subtracted from the estimated background, and the negative pixels were truncated to a value of 0. Global thresholding was further applied with a low threshold value (≈8), to obtain a rough segmented vascular map. The resulting map contained much noise in the form of false-positive blobs. This noise was removed by filling the blobs having an area less than 200px 2 . The result was a segmented binary map of the retinal vessels. The result of various steps on the image can be seen in Figure 5 The segmented vessel map was used to estimate an upper limit for the vessel diameters, and this was done using distance transform. Distance transform gives maps each white pixel of a binary image to its least distance from the black pixels.
The twice of the maximum value computed by the distance transform corresponds to the maximum vessel diameter. This estimate was very crucial for successive steps to work, as is explained further. Once we have obtained a segmented vessel map, the centreline can be obtained using skeletonization. We have used the Zhang-Seun's thinning algorithm to obtain the centrelines from the segmented vessel map. The morphological closing operation was used with a linear structuring element applied at every 15 points, and removing these points was an essential step because calculating normals at these points was ambiguous. The bifurcation points were detected using template kernels, as shown in Figure 6, applied at every 15 • . The detected bifurcation points were removed, and the resulting image was the centreline map. Due to the removal of bifurcation points, some noise was expected to be induced in the image and thus must be removed. To remove the noise, lines that were less than 25px in length were removed. The vessel centrelines were randomly colored for better visualization. The results at each step ate shown in Figure 7. Figure 6: Tempelate kernels to detect bifurcation points

Interpolation normal to the centrelines and clustering
To select a particular vessel centreline from the centreline map, input from the user was used. The user was prompt to click on the screen. The centreline closest to the clicked point was selected. The points on the centreline were stored and normals were to be drawn at each of these points. To draw a normal at a point, we need the slope at that point, and to calculate the slope, a window of 6 pixels centered at that point was used. The earlier computed maximum diameter estimate from the distance transform was the most crucial element in drawing the normals. Let the maximum the estimated diameter was "d," then normals drawn were of the length "1.5xd". This ensured that no part of the vessel along the normal was missed. The points along this normal were linearly interpolated and intensity value each of these points was computed using bilinear interpolation. Thus, we can plot intensity values along any of the normal and atypical such variation is shown in Figure 8,c). All such normal variations can be stacked together to obtain a 3D intensity plot of intensity variations aligned along a common center as shown in Figure 8, d). One can observe from the plot that the vessels correspond to the 3D valley. Another way to visualize these stacked-aligned intensity variations is as an image, which is shown in Figure 9, a). The vascular pixels are of lower intensity in such an image. To obtain the diameters pixels in such an image can be clustered together, and the lowest intensity cluster will correspond to the vessels. Thus, K-Means clustering was used to form 3 clusters in the image, and the resulting clustered image is shown in Figure 9, b). To separate the vascular pixels, thresholding was performed about the lowest intensity cluster, and the resulting thresholded image is shown in Figure 9, c).
This resulting image presented a profile of the vessel aligned horizontally. However, it was noisy and had breaks. Central Light Reflux(CLR) in vessels was also a hurdle as in images with CLR; the centreline is considerably bright and, thus, usually get clustered as non-vascular. To make our algorithm immune to CLR, we filled broken verticle lines in the profile, filling the false-negative pixels. Further, the vessel is supposed to be symmetric about the centreline. Thus, to exploit this symmetry, the image was flipped about the horizontal axis, and 'logical and operation' was performed to find the common vascular region between the flipped and the original image. The resulting image is shown in Figure 9, d). To further smoothen the profile, horizontal points separated by a distance of less than 20pixels were joined, and the resulting final estimated vessel profile is shown in Figure 9, e). To compute the diameters, the number of white pixels along each vertical line (corresponds to the earlier normals) was calculated. These computed diameters are shown in the Figure 8, e).  Table 2.
For the diameter estimation algorithm, CLRIS section of the REVIEW dataset was used to test the algorithm. Predictions were made for various annotated vessel segments and the standard-deviation error (σ error ) the metric was used to judge the algorithm. Let at a given vessel profile i, where ω i is the estimated width and ψ i is the correspondent ground truth. Let the total number of points be n p then the standard deviation of the width differences is given by where, The average σ error achieved by the algorithm is 0.40, which is in the range 1.2 -0.2, achieved by the algorithms in the literature. Detailed results are shown in the Table 3.
A 2-second long video of fundoscopy, captured through an ophthalmoscope, was sampled at a sample rate of 30 to obtain 60 images. Each of the images was individually processed using the above algorithms. The diameters of a particular artery and a vein were tracked. The results are shown in Figure 10. The blue curve is estimated using the algorithm. The orange curve is the smoothened version of the blue curve. Savitzky-Golay filter with a window size of 0.1×window-size (5) was used to smoothen the curve, and a low pass filter was applied with a cutoff frequency of 2Hz. The resulting curve clearly shows the sinusoidal variations corresponding to the SVP, as per expectations. Due to the stretchy nature of the SVP, when the arteriole diameter increase, the venule diameters are expected to decrease. Thus, the arteriole and venue diameters variation curves are expected to be a 180 • out-of-phase, which is apparent in the estimated curve.

CONCLUSIONS
The proposed device attached to an ophthalmoscope fulfills the need for funduscopy with in-situ vasculature analysis. The results discussed above are harmonious with the predictions. The accuracy achieved by the algorithms is comparable to that of the state-of-the-art. The efficiency of the algorithms is the topic of focus in the future. Complete automation of the device is also currently a bottleneck because the automatic alignment is susceptible to noise, and thus, the pulsations get lost in the noise, and no meaningful data is extractable. This problem can be solved by employing commercial registration programs like i2k-Retina-align, but, still, these programs cannot process the images in real-time. Thus, complete automated estimation in real-time with high efficiency is currently being worked on and will be a part of a future project.