MSDD inference is too slow
I run the MSDD model on Nvidia A10 (24GB), but the inference is too slow, I looked on the code and there is a lot of traffic between the CPU and GPU and vice versa.

most of the time GPU utilization is on 0%

First the data is split into short segments according to the number of scales (I have 5 scales).
After each scale splitting the embedding extraction is applied and save the embedding to pkl file.
Then the the clustering is applied and finally the MSDD is applied.

Is there something that can be done in order to speed up the inference?
Is there any flag for parallelism the embedding extraction stage?

please help.  
