microsoft/phi-4-gguf
Text Generation
•
15B
•
Updated
•
6.08k
•
167
Doug PRO
dougeeai
AI & ML interests
CUDA, Sovereign AI, OSS GenAI, LLM fine-tuning, VRAM optimization, RAG, SLM Agents
Recent Activity
liked
a model
about 1 month ago
microsoft/phi-4-gguf
upvoted
an
article
about 1 month ago
🌳 QAT: The Art of Growing a Bonsai Model
updated
a model
about 2 months ago
dougeeai/llama-cpp-python-wheels
Organizations
None yet
liked
a
model
about 1 month ago
upvoted
an
article
about 1 month ago
Article
🌳 QAT: The Art of Growing a Bonsai Model
•
15
updated
a
model
about 2 months ago
posted
an
update
about 2 months ago
Post
305
Llama.cpp wheels for Windows - Hot off the press!
I got tired of fighting with Visual Studio and CUDA Toolkit every time I wanted to use llama-cpp-python on Windows, so I've been building pre-compiled wheels for the community.
## What's Available:
✅ RTX 50/40/30/20 Series support (Blackwell, Ada, Ampere, Turing)
✅ CUDA 11.8, 12.1, 13.0 (Blackwell is CUDA 13 only)
✅ Python 3.10-3.13
✅ Just 'pip install' and run - no build tools needed
## Why this matters:
Windows users face a painful setup process with llama-cpp-python. These wheels eliminate:
- Visual Studio installation
- CUDA Toolkit setup
- Compilation errors
- Hours of troubleshooting
**Download:** https://github.com/dougeeai/llama-cpp-python-wheels
Linux wheels coming soon! Let me know what configs you need.
Tested on Ada Lovelace & Ampere
dougeeai/llama-cpp-python-wheels
#llama-cpp #gguf #windows #prebuilt
I got tired of fighting with Visual Studio and CUDA Toolkit every time I wanted to use llama-cpp-python on Windows, so I've been building pre-compiled wheels for the community.
## What's Available:
✅ RTX 50/40/30/20 Series support (Blackwell, Ada, Ampere, Turing)
✅ CUDA 11.8, 12.1, 13.0 (Blackwell is CUDA 13 only)
✅ Python 3.10-3.13
✅ Just 'pip install' and run - no build tools needed
## Why this matters:
Windows users face a painful setup process with llama-cpp-python. These wheels eliminate:
- Visual Studio installation
- CUDA Toolkit setup
- Compilation errors
- Hours of troubleshooting
**Download:** https://github.com/dougeeai/llama-cpp-python-wheels
Linux wheels coming soon! Let me know what configs you need.
Tested on Ada Lovelace & Ampere
dougeeai/llama-cpp-python-wheels
#llama-cpp #gguf #windows #prebuilt
- 1 reply
published
a
model
about 2 months ago
updated
a
collection
about 2 months ago
reacted to
abidlabs's
post with 🔥
about 2 months ago
Post
8695
Why I think local, open-source models will eventually win.
The most useful AI applications are moving toward multi-turn agentic behavior: systems that take hundreds or even thousands of iterative steps to complete a task, e.g. Claude Code, computer-control agents that click, type, and test repeatedly.
In these cases, the power of the model is not how smart it is per token, but in how quickly it can interact with its environment and tools across many steps. In that regime, model quality becomes secondary to latency.
An open-source model that can call tools quickly, check that the right thing was clicked, or verify that a code change actually passes tests can easily outperform a slightly “smarter” closed model that has to make remote API calls for every move.
Eventually, the balance tips: it becomes impractical for an agent to rely on remote inference for every micro-action. Just as no one would tolerate a keyboard that required a network request per keystroke, users won’t accept agent workflows bottlenecked by latency. All devices will ship with local, open-source models that are “good enough” and the expectation will shift toward everything running locally. It’ll happen sooner than most people think.
The most useful AI applications are moving toward multi-turn agentic behavior: systems that take hundreds or even thousands of iterative steps to complete a task, e.g. Claude Code, computer-control agents that click, type, and test repeatedly.
In these cases, the power of the model is not how smart it is per token, but in how quickly it can interact with its environment and tools across many steps. In that regime, model quality becomes secondary to latency.
An open-source model that can call tools quickly, check that the right thing was clicked, or verify that a code change actually passes tests can easily outperform a slightly “smarter” closed model that has to make remote API calls for every move.
Eventually, the balance tips: it becomes impractical for an agent to rely on remote inference for every micro-action. Just as no one would tolerate a keyboard that required a network request per keystroke, users won’t accept agent workflows bottlenecked by latency. All devices will ship with local, open-source models that are “good enough” and the expectation will shift toward everything running locally. It’ll happen sooner than most people think.
- 8 replies
Totally agree!
updated
a
collection
about 2 months ago
updated
a
model
about 2 months ago
updated
a
collection
about 2 months ago
updated
a
model
about 2 months ago
published
2
models
about 2 months ago
updated
a
collection
about 2 months ago