Should we use tie_weights always? (intend to separate source/target embedding)
HelloI’m developing a machine translation model(Seq2seq), which have different source and target tokenizer. (As a beginner, I even do not know if this is the correct way to solve the machine translation problem.)The problem happens when I want to divide shared(input, output) embedding into source/target embedding (I’m using T5ConditionalGeneration). After that, there is a function self.init_weights() and here they operate self.tie_weights() method.As far as I know, the self.tie_weights() make output embeddings the same as input embeddings. But in my case, then target embedding will be changed and cause indexing error.My question is:Should I call self.tie_weights() in this case?Is there any better approach that I can attempt?Here is the related post below:What is the `tie_word_embeddings` option exactly doing?🤗TransformersHi, 
for some models there is this tie_word_embeddings parameter. I think it is for the text 2 text models. 
Can someone please explain what exactly this parameter is doing? 
Many thanks 
PhilipThanks in advance for any answers!