Inconsistent image processing times
Hi,
I have encountered some inconsistencies w.r.t. image pre-processing times. Inside the modeling_clip.py.JinaCLIPModel.encode_image() method, I am logging the image preprocessing time with the following:
def encode_image(...):
...
start = time.time()
pixelvals = self.preprocess(_processed_images)
torch.cuda.synchronize()
print(f"INSIDE JINA: Took {time.time() - start} to preprocess {len(_processed_images)} images")
pixelvals = pixelvals.to(self.device)
...
In my experiments, this sometimes lasts approx. 1sec, but mostly it lasts roughly 8sec for the same 100 images batch (these are given as PIL Image objects). Measurements are recorded through independent runs, where the Jina embedding model is initialized and loaded onto the GPU from scratch.
Are there any reasons that can justify this variability in image pre-processing times?
Hi George,
No, in principle they are not. They are typically small images though, ranging from 150x150 to 700x700. I tried resizing all of them to 500x500 before passing them to the model, but the inconsistency remained.
What would be a reasonable img processing time to look for?
The exact same batch usually takes 8s but once in a while takes 1s? Or you re measuring different batches?
Yes, I have noticed this behavior many times using the exact same batch (i.e., the same images input).
Another interesting thing is that, when I am experimenting with different batches (e.g., 1000 batches of 100 images), the processing time per batch is usually 8s. But once in a while it drops to 1s for a few consecutive batches and then it increases again.