{'fraction_non_alphanumeric': 0.02647709858675294, 'fraction_numerical': 0.009194619444917418, 'mean_word_length': 4.972038637519065, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The Oxford Road Boundaries Dataset

In this paper we present the Oxford Road Boundaries Dataset, designed for training and testing machine-learning-based road-boundary detection and inference approaches. We have hand-annotated two of the 10 km-long forays from the Oxford Robotcar Dataset and generated from other forays several thousand further examples with semi-annotated road-boundary masks. To boost the number of training samples in this way, we used a vision-based localiser to project labels from the annotated datasets to other traversals at different times and weather conditions. As a result, we release 62605 labelled samples, of which 47639 samples are curated. Each of these samples contains both raw and classified masks for left and right lenses. Our data contains images from a diverse set of scenarios such as straight roads, parked cars, junctions, etc. Files for download and tools for manipulating the labelled data are available at: oxford-robotics-institute.github.io/road-boundaries-dataset


I. INTRODUCTION
Obtaining well-generalising, high-performance deep networks typically requires large quantities of training samples. Even if easy to hand-annotate -and this is often not the case -this is an inordinate workload for the researcher. To cope with the variability of road boundaries, the required data should equally incorporate great variability changes in environment, scale, appearance, colour, background clutter, occlusion, perspective and illumination. Yet, finegrained annotation of data requires time-consuming human effort where labels of different classes must be assigned to outlined distinct regions. Nevertheless, finding true road boundaries is crucial for autonomous vehicles as they legally and intentionally delimit drive-able surfaces. Indeed, as somewhat durable parts of the road infrastructure, lane markings, traffic signs, feature points or road boundaries are often used as features for localisation [1]. Moreover, detecting only visible road boundaries is often not sufficient in urban scenarios -examples in Figure 2 show road boundaries either partially or fully occluded. Indeed, there is often no visible road boundary to detect, as in Figure 2. In these sorts of scenarios, the vehicle should still sufficiently sense the environment to move safely about it. For this reason, datasets incorporating road boundaries should include a large number of samples in order to capture the variability of road boundaries. They should also contain annotations for both visible and occluded road boundaries. To this end, we have developed a framework for efficient annotations within a reasonable amount of time. In this paper, we present The Oxford Road Boundaries Dataset, which we generated using this framework.

II. RELATED WORK
Road-boundary detection is useful for autonomous driving in that it can facilitate safe driving in a legally demarcated area. Work in this area includes [2]- [5].
Autonomous driving datasets abound [6]- [13] and accurate, human-derived annotations for a wide number of semantic classes are available in these releases. However, road boundaries are not included, regardless of their importance for scene understanding, lane-level localisation, mapping and planning in autonomous driving.
The work which is mostly related to ours is [14]; yet, since it is based on aerial imagery, it does not provide synchronisation to vehicle-mounted sensors -and is not released for download at the time of writing.
Works based on the proposed dataset include [15]- [17], where CNN-based deep-leaning models -Visible Road Boundary Detection (VRBD) and Occluded Road Boundary Inference (ORBI) -were proposed for detection and inference of visible and occluded road boundaries and were applied for scene understanding and lateral localisation problems.

III. THE OXFORD ROAD BOUNDARIES DATASET
This section describes the vehicle, collection regime, annotation procedures and dataset partitioning.
The overall pipeline of the proposed road boundary data generation framework is shown in Figure 3. We generate camerabased annotation from the mutimodal collected data (Section III-A) by projecting laser-based annotations to the stereo, forward-facing camera (Section III-B) through Visual Odometry (VO) interpolation. Visual localisation tools are used to refer ground-truth annotations to other traversals for fast extension of the labels (Section III-C). Manual curation is carried out to check the created labels (Section III-D) and a deep-learning pipeline is used to automatically classify the boundaries as either visible or occluded (Section III-E).

A. Data Collection
This release is an annotative exercise atop the original Oxford RobotCar Dataset, of which we used seven forays from which approximately 62 000 pairs of samples were generated. Table I summarises the forays driven.

B. Annotation
Fine-grained hand-annotation of road boundaries from images would be a very time-consuming process and it would be impossible to exactly annotate the position of occluded road boundaries from the images themselves. To tackle this problem,, we annotated the 3D pointcloud data collected by the 2D laser: as shown in Figure 4, the laser is attached vertically to the front of the vehicle; as such, the light pulses not only hit the road boundaries almost perpendicularlywhich made the road boundaries easily distinguishable for annotation -but also reach road boundaries behind parked vehicles as the pulses can pass under the vehicles (see Figure 5). 3D pointclouds are constructed by accumulating the push-broom returns along a pose-chain formed by VO which is only required to be accurate in the local region around the example to be annotated -drift is not a significant problem. Specifically, to obtain a 3D pointcloud from the Oxford RobotCar Dataset from 2D laser, we integrated subsequent vertical laser scans in a coherent coordinate frame. We used VO to estimate the vehicle's motion and compute the transformations between subsequent scans. Note that, as VO uses camera images to calculate the vehicle's ego-motion, it provides transformations between camera timestamps which in general differ from the laser timestamps. For this reason, we use interpolation to obtain transformations between subsequent 2D laser scans at time frames and as follows: . .

≤ , ≥
where and are the closest time steps of the VO with respect to the laser frames and where ( , ) is defined as follows: Once we have generated 3D point clouds of the datasets, we assign the same IDs to the points lying on the same continuous boundary. This enabled us to connect consecutive points according their IDs and fully annotate road boundary segments between the points, as shown in Figure 5. A Bird's-eye view of the annotations of the 2015-05-26-13-59-22 foray is shown in Figure 6.
Annotation error in projecting from laser to camera due to offset between these two sensors is mitigated by hand-tuning the already fairly accurate extrinsic calibration provided by [18]. Additionally, centimetre-level accuracy is available from our vision-based localiser. Finally, drift in odometry is minimal over the limited distances over which estimates are accumulated.

C. Extension to other traversals
Annotating road boundaries in laser point clouds and projecting them into images enables us to easily generate hundreds of groundtruth masks within a short period of time, approximately 750 raw ground-truth masks per hour. To boost the number of training samples, we used a vision-based localiser [19] to project labels from the annotated datasets to other traversals at different times and in different weather conditions, as shown in Figure 8.
When annotating road boundaries we did not assume that every road boundary was a curb and we annotated all road boundaries even if they were on a flat surface with no height differences between roads and pavements. Although seeing flat road boundaries in the laser pointcloud was more difficult than curbs, we were simultaneously displaying projections of annotations of road boundaries on camera images in our annotation tool and using the projections as a reference to precisely position the annotations. This makes the proposed road boundary annotation framework applicable to any type of road or road network (e.g., urban roads, dirt roads, highways and motorways). An example of annotated road boundary with no height difference is shown in Figure 7.

D. Manual Curation
We manually curated the raw masks of the dataset, removing sections where projected annotations were unreasonable, e.g. when localisation failed or where the location of road boundaries changed over time. As a result, we obtained 62 605 labelled samples, and from those, we edited 47 639 samples while curating. Note that every sample is a pair of images from the left and right lenses.   Table I: Summary of data released. Manual: labels are obtained by projecting road boundaries from hand-annotated laser point cloud to images. Automatic: labels are obtained by projecting road boundaries from the hand-annotated datasets to other traversals using vision-based localiser.

E. Partitioning
Having data samples separated in two classes, visible and occluded, can be beneficial for operational safety [15]. To partition our raw road boundary masks into these two classes we trained the U-Net architecture [20] with raw masks. The resulting network can detect visible road boundaries, but fails to infer correct structure and position of occluded road boundaries for two reasons: (1) the network does not have a large enough receptive field to capture contextual information in the scene and to estimate position and structure of occluded road boundaries and (2) the network does not have any structure to force/bias outputs to be in a thin long shaped form for the occluded road boundaries. As such, the network can detect visible road boundaries when trained with raw masks, but fails to infer occluded ones and outputs blurry masks over occluding obstacles as shown in Figure 9, enabling automatic partitioning of the raw training data into two classes. We obtain masks for detected visible boundaries by applying threshold to the outputs. AND operation between the raw labels and thresholded outputs give us labels for visible road boundaries. Labels for occluded road boundaries are obtained by subtracting labels for visible from the raw labels (see Figure 1 for examples of partitioned training data masks). In total, 33% of all labels and 32% of curated labels are classified as occluded road boundaries. We refer the reader to [15] for more details on the performance of this network.

F. Recommended Usage
Our suggested usage is that all samples can be used for pretraining and that fine-tuning with curated samples is sensible. We used a subset of this dataset, approximately 23 000 samples, for training our models in [15] and achieved 1 score of 0.9477 for visible road boundary detection and 0.8934 for occluded road boundary inference. In this application, we dilated masks before training models.
The thickness of road boundary line annotations on the ground truth masks are the same regardless of the height of road boundaries. This is because the annotated points do not contain any such information. To compensate for this, we recommend to use tolerancebased evaluation metrics -evaluation should be done by considering a sweep of thresholded pixel distances of predictions from true road boundaries.   The road boundary on the left hand side of the road with no height difference between road and pavement was precisely annotated in the laser pointcloud by simultaneously displaying its projection on the camera image and using it as a reference while annotating.