gpt2 translation task

[jzl0166]

I want to fine-tune the model and do some machine translation based on Gpt-2. I created my dataset according to the Gpt-2 paper in this format: 'sentence1 = translation1 \n sentence2 = translation2 \n ...' and did the fine-tune training. After training, I try to do the translation by 'python interactive_conditional_samples.py --top_k 40' but when I type in my input, it just show me a paragraph including several sentences(A = B \n B = C...), not the translation sentence of my input. Is there anything wrong with my input dataset or training? How could I do the machine translation by Gpt-2?

[jaimu97]

Make sure you're properly formatting the data with <|startoftext|> and <|endoftext|> between samples otherwise it will think that it's one continuous stream and should continue like that.