Does run_generation.py has a "no_repeat_ngram_size" attribute?
Good day,I’m generating text withrun_generation.py. I would like to know if there is any way to pass the attributeno_repeat_ngram_size. I have seen that attribute withmodel. generatebut not inrun_generation.py.
2020-09-02T08:29:54Z
hi@kintaro,run_generation.pyis used for auto-regressive generation which uses sampling.no_repeat_ngram_sizeis used for beam search sorun_generation.pydoesn’t have that attribute since it only uses sampling for generation
2020-09-02T14:17:03Z
Hi@kintaroand@valhallaThanks@valhallafor replying right away! In this case,no_repeat_ngram_sizecan actually be applied both in the beam search and sampling generation. I think we just left the option out of therun_generation.pyscript because we didn’t want it to be too crowded, but you can just pass the argument in:github.comhuggingface/transformers/blob/master/examples/text-generation/run_generation.py#L252)else:encoded_prompt = tokenizer.encode(prompt_text, add_special_tokens=False, return_tensors="pt")encoded_prompt = encoded_prompt.to(args.device)if encoded_prompt.size()[-1] == 0:input_ids = Noneelse:input_ids = encoded_promptoutput_sequences = model.generate(input_ids=input_ids,max_length=args.length + len(encoded_prompt[0]),temperature=args.temperature,top_k=args.k,top_p=args.p,repetition_penalty=args.repetition_penalty,do_sample=True,num_return_sequences=args.num_return_sequences,)(just add a e.g.no_repeat_ngram_size=3,line in the arguments for thegeneratefunction)
2020-09-03T04:13:24Z
Thanks@yjernite! Didn’t know thatno_repeat_ngram_sizecan also be used with sampling.
2020-09-04T14:49:56Z
Thanks a lot for your great help