Conditional generation from hidden states only
Hi,I’m working on a relatively simple problem where I would like totrain an encoder-decoder (seq2seq) architecture to obtain a latent space on my own corpus of sequencessample from the latent space (ignoring the encoder entirely), and feed that into the decoder, and get an output sequenceI’ve noticed from the docs thatinput_idsare always required for the decoder (i.e. when using.generate), which makes sense when you have an input prompt (or even a single token to start from). However, in my case, I don’t have an input prompt, only a hidden state.Is this workflow something that’s easily possible withinTransformers? or perhaps only within a specific architecture?