SimpleChat-4B-V1-f32-GGUF

OpenBuddy’s SimpleChat-4B-V1 is a 4B-parameter conversational AI model built for streamlined, casual everyday interactions, emphasizing concise, rational, and creative responses without relying on complex chain-of-thought reasoning. Optimized for natural dialogue, SimpleChat-4B-V1 is part of the SimpleChat series exploring Non-Chain-of-Thought (Non-CoT) models, making it especially suitable for direct, engaging conversations that prioritize clarity and brevity while retaining the ability to enhance creativity and expressive capacity in outputs.

Model Files

File Name	Quant Type	File Size
SimpleChat-4B-V1.BF16.gguf	BF16	8.05 GB
SimpleChat-4B-V1.F16.gguf	F16	8.05 GB
SimpleChat-4B-V1.F32.gguf	F32	16.1 GB
SimpleChat-4B-V1.Q2_K.gguf	Q2_K	1.67 GB
SimpleChat-4B-V1.Q3_K_L.gguf	Q3_K_L	2.24 GB
SimpleChat-4B-V1.Q3_K_M.gguf	Q3_K_M	2.08 GB
SimpleChat-4B-V1.Q3_K_S.gguf	Q3_K_S	1.89 GB
SimpleChat-4B-V1.Q4_K_M.gguf	Q4_K_M	2.5 GB
SimpleChat-4B-V1.Q4_K_S.gguf	Q4_K_S	2.38 GB
SimpleChat-4B-V1.Q5_K_M.gguf	Q5_K_M	2.89 GB
SimpleChat-4B-V1.Q5_K_S.gguf	Q5_K_S	2.82 GB
SimpleChat-4B-V1.Q6_K.gguf	Q6_K	3.31 GB
SimpleChat-4B-V1.Q8_0.gguf	Q8_0	4.28 GB

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):