Colab's session crashed after using all available RAM when loading falcon-7B
Hi there!How can I load falcon-7b in anything that requires less vRAM than bfloat? When I try this, Colabmodel = AutoModelForCausalLM.from_pretrained("tiiuae/falcon-7b-instruct",trust_remote_code=True, device_map="auto", load_in_8bit=True,  )session crashes even though I’m using gpu wih 15 gb of ram
2023-06-06T12:27:40Z
Same here, any solution?
2023-10-26T15:31:17Z
same issue here