"Is not the truth the truth?": Analyzing the Impact of User Validations
  for Bus In/Out Detection in Smartphone-based Surveys
  Passenger flow allows the study of users' behavior through the public network
and assists in designing new facilities and services. This flow is observed
through interactions between passengers and infrastructure. For this task,
Bluetooth technology and smartphones represent the ideal solution. The latter
component allows users' identification, authentication, and billing, while the
former allows short-range implicit interactions, device-to-device. To assess
the potential of such a use case, we need to verify how robust Bluetooth signal
and related machine learning (ML) classifiers are against the noise of
realistic contexts. Therefore, we model binary passenger states with respect to
a public vehicle, where one can either be-in or be-out (BIBO). The BIBO label
identifies a fundamental building block of continuously-valued passenger flow.
This paper describes the Human-Computer interaction experimental setting in a
semi-controlled environment, which involves: two autonomous vehicles operating
on two routes, serving three bus stops and eighteen users, as well as a
proprietary smartphone-Bluetooth sensing platform. The resulting dataset
includes multiple sensors' measurements of the same event and two ground-truth
levels, the first being validation by participants, the second by three
video-cameras surveilling buses and track. We performed a Monte-Carlo
simulation of labels-flip to emulate human errors in the labeling process, as
is known to happen in smartphone surveys; next we used such flipped labels for
supervised training of ML classifiers. The impact of errors on model
performance bias can be large. Results show ML tolerance to label flips caused
by human or machine errors up to 30%.
