May i know if it can run on 16GB macbook AIR m2?
Would there be any issues with throttling or overheating? lags,
What are the minimum requirements to use this model on the local MacBook M2 Air, 16 GB RAM machine?
Suggest using LmStudio ; with quant in the Q4 range.
Limit to 2 experts activated to start. [less load]
Running an LLM/AI on any machine will be taxing; increase cooling / fan if possible.
Also see this repo - ; lots of MAC specific quants in small sizes too:
@SaeedML MacBooks are prone to overheating/throttling, so I suggest lifting up the laptop and/or use external cooling.
Hugging Face does a pretty good job of estimating usability if you put your hardware specs into your profile (under Local Apps and Hardware). I'm running these exact specs (M2 Air 16GB). Q4_K_M is the largest quant that shows up green. It's definitely usable, but it's a bit slow and it will heat up your laptop if you don't give it external cooling. Q3_K_L is significantly faster and cooler.