Continuous execution lead to decreasing inference time
for i in range(50):model.generate()I found the inference time is decreasing. I guess it is relative to cuda cache or something. I want to know what factors influenced this result？