ONNX for VAD inference time improvement small
Hello, 

I am using the offline speaker diarization module and I managed to export vad_multilingual_marblenet.nemo model to onnx. 

Running voice activity detection with ONNX results in an improvement of about 20% less inference time. From what I have read ONNX usually results in a greater imrovement. 

Is this result normal? Did I implement it wrong? Did anybody try this before and has some profiling results? Are there any ways to improve  inference speed even more?

My code in /nemo/collections/asr/models/clustering_diarizer:

```
    def _run_vad(self, manifest_file):
        """
        Run voice activity detection. 
        Get log probability of voice activity detection and smoothes using the post processing parameters. 
        Using generated frame level predictions generated manifest file for later speaker embedding extraction.
        input:
        manifest_file (str) : Manifest file containing path to audio file and label as infer

        """

        shutil.rmtree(self._vad_dir, ignore_errors=True)
        os.makedirs(self._vad_dir)

        self._vad_model.eval()

        self._vad_model.export('vad.onnx')
        ort_session = onnxruntime.InferenceSession('vad.onnx')
        time_unit = int(self._vad_window_length_in_sec / self._vad_shift_length_in_sec)
        trunc = int(time_unit / 2)
        trunc_l = time_unit - trunc
        all_len = 0
        data = []
        for line in open(manifest_file, 'r', encoding='utf-8'):
            file = json.loads(line)['audio_filepath']
            data.append(get_uniqname_from_filepath(file))

        status = get_vad_stream_status(data)
        for i, test_batch in enumerate(
            tqdm(self._vad_model.test_dataloader(), desc='vad', leave=True, disable=not self.verbose)
        ):
            test_batch = [x.to(self._vad_model.device) for x in test_batch]
            with autocast():
                # ONNX inference
                audio_signal, audio_signal_len = test_batch[0], test_batch[1]
                audio_signal, audio_signal_len = audio_signal.to(self._vad_model.device), audio_signal_len.to(self._vad_model.device)
                processed_signal, processed_signal_len = self._vad_model.preprocessor(input_signal=audio_signal, length=audio_signal_len,)

                ort_inputs = {ort_session.get_inputs()[0].name: self.to_numpy(processed_signal), }
                ologits = ort_session.run(None, ort_inputs)
                alogits = np.asarray(ologits)
                logits = torch.from_numpy(alogits[0])
                probs = torch.softmax(torch.as_tensor(logits), dim=-1)
                pred = probs[:, 1]

```