AutoModelForCausalLM and transformers.pipeline
what is the different?which method is good?huggingface.cometa-llama/Meta-Llama-3-8B · Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.pipeline = transformers.pipeline(
    "text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto"andgithub.comPromtEngineer/localGPT/blob/a1dea3becb8b1ae28a87369b1636c4c4a4501c27/load_models.py#L115model_basename=model_basename,use_safetensors=True,trust_remote_code=True,device_map="auto",use_triton=False,quantize_config=None,)return model, tokenizerdef load_full_model(model_id, model_basename, device_type, logging):"""Load a full model using either LlamaTokenizer or AutoModelForCausalLM.This function loads a full model based on the specified device type.If the device type is 'mps' or 'cpu', it uses LlamaTokenizer and LlamaForCausalLM.Otherwise, it uses AutoModelForCausalLM.Parameters:- model_id (str): The identifier for the model on HuggingFace Hub.- model_basename (str): The base name of the model file.model = AutoModelForCausalLM.from_pretrained(model_id,
                                            #  quantization_config=quantization_config,
                                            #  low_cpu_mem_usage=True,
                                            #  torch_dtype="auto",
                                             torch_dtype=torch.bfloat16,
                                             device_map="auto",
                                             cache_dir="./models/")

        tokenizer = AutoTokenizer.from_pretrained(model_id, cache_dir="./models/")

    pipe = pipeline(
        "text-generation",
        model=model,
        tokenizer=tokenizer,
        max_length=MAX_NEW_TOKENS,
        temperature=0.2,
        # top_p=0.95,
        repetition_penalty=1.15,
        generation_config=generation_config,
    )

    local_llm = HuggingFacePipeline(pipeline=pipe)
2024-08-28T10:23:10Z
hi@alice86it’s concerned with degrees of abstraction. There is no good or bad. It’s about being easyPipeline takes care of some of the details under the hood for you. It’s better to stick with the simple one until you reach the limits.huggingface.coBehind the pipeline - Hugging Face NLP CourseWe’re on a journey to advance and democratize artificial intelligence through open source and open science.
2024-08-29T07:37:49Z
I already download the model and directly use the path.model_id = "./Llama3"

pipeline = transformers.pipeline(
    "text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto"
)It show the errorValueError: You are trying to offload the whole model to the disk. Please use the `disk_offload` function instead.Q1.What is the define forYou are trying to offload the whole model to the diskIf I use themodel_id=meta-llama/Meta-Llama-3-8B.It will automatically download the folder models–meta-llama–Meta-Llama-3-8B on ./cacheSo the download model is also the case that  offload the whole model to the disk ?Q2. Because I saw the offload the whole model.what is the onlineload the model code