Spectrogram Enhancer doesn't generalize to spectrogram lengths unseen during training
**Describe the bug**

If one trains TTS spectrogram enhancer (#5565) on short spectrograms, at inference time it doesn't generalize to longer ones. A patch in the beginning gets enhanced but further frames do not.

Example (before, after):
![before](https://user-images.githubusercontent.com/8864149/218520151-230501b1-4e9d-4307-b51e-526440b915c8.png)
![after](https://user-images.githubusercontent.com/8864149/218520199-22187a4b-9cf8-4ac9-b3ce-07c320d3d3b6.png)

**Steps/Code to reproduce bug**

1. Train a spectrogram enhancer
2. Apply it to a spectrogram that's longer than anything from the training set
3. Only a patch in the beginning gets enhanced

**Expected behavior**

The whole spectrogram should have got additional details, not just the first patch