Error code 400 when deploying huggingface bigscience/bloom to SageMaker
Using the sample code below:Screen Shot 2022-08-23 at 11.28.08 PM1520×1556 295 KBI’m getting the following error:{
  "code": 400,
  "type": "InternalServerException",
  "message": "\u0027bloom\u0027"
}Any ideas on what could be causing this issue?
2022-11-09T18:26:24Z
Having the same issue. Did you end up resolving it?
2022-11-09T19:25:04Z
@mel-zhengit’s because BLOOM requires transformers version 4.21.0 but the inference containers offered only supports up to version 4.17.0. I ended up not using SageMaker. I went with a serverless approach and leverage our custom container that has transformers version 4.21.0 installed. Even tho I got the BLOOM model working, it is so large it is practically unusable. It’s unbearably slow running any inference.
2022-11-11T20:43:02Z
Thanks for the update! I actually resolved it by supplying my own custom inference.py with the latest transformers version. Ended up using bloom-560m because bloom is too large like you said.
2023-05-19T22:37:01Z
Hi@mel-zhengdid you solve the issue?Could you share your solution?