Understanding Medical Conversations: Rich Transcription, Confidence
  Scores & Information Extraction
  In this paper, we describe novel components for extracting clinically
relevant information from medical conversations which will be available as
Google APIs. We describe a transformer-based Recurrent Neural Network
Transducer (RNN-T) model tailored for long-form audio, which can produce rich
transcriptions including speaker segmentation, speaker role labeling,
punctuation and capitalization. On a representative test set, we compare
performance of RNN-T models with different encoders, units and streaming
constraints. Our transformer-based streaming model performs at about 20% WER on
the ASR task, 6% WDER on the diarization task, 43% SER on periods, 52% SER on
commas, 43% SER on question marks and 30% SER on capitalization. Our recognizer
is paired with a confidence model that utilizes both acoustic and lexical
features from the recognizer. The model performs at about 0.37 NCE. Finally, we
describe a RNN-T based tagging model. The performance of the model depends on
the ontologies, with F-scores of 0.90 for medications, 0.76 for symptoms, 0.75
for conditions, 0.76 for diagnosis, and 0.61 for treatments. While there is
still room for improvement, our results suggest that these models are
sufficiently accurate for practical applications.
