tencent

/

Hunyuan-MT-7B

like 538

Follow

Tencent 5.77k

Downloading Model from https://www.modelscope.cn to directory: /Users/xing/.cache/modelscope/hub/models/Tencent-Hunyuan/Hunyuan-MT-7B-fp8
Downloading Model from https://www.modelscope.cn to directory: /Users/xing/.cache/modelscope/hub/models/Tencent-Hunyuan/Hunyuan-MT-7B-fp8
Traceback (most recent call last):
File "/Users/xing/AI/Project/test.py", line 7, in
model = AutoModelForCausalLM.from_pretrained(model_name_or_path, device_map="cpu") # You may want to use bfloat16 and/or move to GPU here
File "/opt/anaconda3/envs/llm/lib/python3.10/site-packages/modelscope/utils/hf_util/patcher.py", line 281, in from_pretrained
module_obj = module_class.from_pretrained(
File "/opt/anaconda3/envs/llm/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 604, in from_pretrained
return model_class.from_pretrained(
File "/opt/anaconda3/envs/llm/lib/python3.10/site-packages/transformers/modeling_utils.py", line 288, in _wrapper
return func(*args, **kwargs)
File "/opt/anaconda3/envs/llm/lib/python3.10/site-packages/transformers/modeling_utils.py", line 5130, in from_pretrained
hf_quantizer.preprocess_model(
File "/opt/anaconda3/envs/llm/lib/python3.10/site-packages/transformers/quantizers/base.py", line 238, in preprocess_model
return self._process_model_before_weight_loading(model, **kwargs)
File "/opt/anaconda3/envs/llm/lib/python3.10/site-packages/transformers/quantizers/quantizer_compressed_tensors.py", line 79, in _process_model_before_weight_loading
apply_quantization_config(model, ct_quantization_config, self.run_compressed)
File "/opt/anaconda3/envs/llm/lib/python3.10/site-packages/compressed_tensors/quantization/lifecycle/apply.py", line 162, in apply_quantization_config
if matches := find_name_or_class_matches(name, submodule, config.ignore):
File "/opt/anaconda3/envs/llm/lib/python3.10/site-packages/compressed_tensors/quantization/lifecycle/apply.py", line 331, in find_name_or_class_matches
targets = sorted(targets, key=lambda x: ("re:" in x, x))
TypeError: 'NoneType' object is not iterable