Running batch transform in Sagemaker on a Huggingface model from the Hub with parameters
Hi,I am trying to run batch transform using facebook/bart-large-mnli  on Sagemaker following the directions in this article:Deploy models to Amazon SageMakerIs there a way to run it in a multi-label mode?I see inthisexample its possible when creating a classifier, but I couldn’t find a way to do it with the batch transform sagemaker SDK
2022-07-26T21:53:05Z
Did you manage to run it. Im trying to run the exact same model in batch. So far the closest I’ve gotten is putting the parameters at each record in the json file like this and then putting each record in a line{“inputs”: “pues claro que presenta domiciliacion, se cobraron de mi cuenta. mejor digame como recuperar ese dinero porque si no voy directo con profeco”, “parameters”: {“candidate_labels”: “tarjeta de credito,retiro,cajero,efectivo,transferencia,aclaracion,credito”, “multi_label”: false}}{“inputs”: “pues claro que presenta domiciliacion, se cobraron de mi cuenta. mejor digame como recuperar ese dinero porque si no voy directo con profeco”, “parameters”: {“candidate_labels”: “tarjeta de credito,retiro,cajero,efectivo,transferencia,aclaracion,credito”, “multi_label”: false}}but my processing job still fails with this error
2023-02-02T15:24:04Z
I have a similar issue. I could not use an inference.py overwrite a batch transform huggingface model from the hub. Nothing in my inference.py is printed in the log and it still gives me the error from the original functionsagemaker-huggingface-inference-toolkit/handler_service.py at main · aws/sagemaker-huggingface-inference-toolkit · GitHub.hub = {
    'HF_MODEL_ID':'facebook/bart-large-mnli', 
    'HF_TASK':'zero-shot-classification' 
}

huggingface_model = HuggingFaceModel(
    env=hub,
    role=role,
    entry_point="inference1.py",
    source_dir = "./code" ,
    transformers_version='4.17.0',
    pytorch_version='1.10.2',
    py_version='py38',
)

batch_job = huggingface_model.transformer(
    instance_count=1,
    instance_type='ml.m5.4xlarge',
    assemble_with = 'Line',
    accept = "application/json",
    max_payload =1, 
    output_path=output_s3_path, 
    strategy='SingleRecord'
)
batch_job.transform(
    data= s3_data_input, 
    split_type="Line",  
    content_type="application/json"
)```