How do I load an SFTTrainer model finetuned falcon-7b-sharded-bf16 using custom dataset, and make prediction with it
How do I load a saved SFTTrainer model after uploading it to HuggingFace and how do I make a prediction with the model?The model was trained usingColab notebook to fine-tune Falcon-7B on Guanaco dataset using 4bit and PEFTIt was trained on a custom dataset similar to the Guanaco dataset.How do I use the saved trained model that was uploaded to HuggingFace?HUGGING_FACE_USER_NAME = "username" 
from huggingface_hub import notebook_login 
notebook_login()After a successful login, I pushed to HuggingFace. model_name = “model_name”trainer.model.push_to_hub(f"{HUGGING_FACE_USER_NAME}/{model_name}",use_auth_token=True)Now I want to load and Make Inference, Can you help out with that because I am getting an error here?from peft import LoraConfig
 model_name = "username/model_name" lora_config = LoraConfig.from_pretrained(model_name)
 '---------------------------------------------' 

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, AutoTokenizer

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    quantization_config=bnb_config,
    trust_remote_code=True
)
model.config.use_cache = FalseWhen I run the above code, I am getting an error.OSError: "username/model_name" does not appear to have a file named config.json. Checkout 'https://huggingface.co/username/model_name/main' for available files.
2023-07-15T14:06:31Z
chage the username/model_name to a specific model path
2023-08-01T13:19:25Z
Try:model.config.to_json_file(“config.json”)the file is saved, but then I get this other error:OSError: Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory…