Docker huggingface/transformers-pytorch-gpu and spaCy compatibility
Hi everyone,I’m trying to run a Docker container using the huggingface/transformers-pytorch-gpu image for a test project. As part of this project, I need to installspaCy, but I’m running into various installation issues.I’ve tried several approaches, including installing different versions of spaCy and its dependencies, but every time I attempt this, new issues arise. Some of the problems I’ve encountered include conflicting dependencies, missing packages, or version incompatibilities with the Hugging Face image.Here’s a summary of what I’ve tried so far:Basic pip install: I started by runningpip install spacyinside the container, but this led to dependency conflicts with some of the Hugging Face libraries.Specific spaCy version: I tried installing spaCy versions 3.8.2 and earlier, but the installation failed with similar dependency issues.spaCy transformers: I also attempted to install thespacy-transformerspackage to ensure compatibility with the Hugging Face transformer models, but it didn’t resolve the conflicts.I’m hoping someone can guide me on how to properly set up spaCy with the huggingface/transformers-pytorch-gpu Docker image or suggest an alternative approach that works smoothly. If anyone has successfully integrated these tools in a Docker environment, your help would be greatly appreciated!
2024-10-29T23:42:54Z
Try this.pip install git+https://github.com/explosion/spaCy# requirements.txt
git+https://github.com/explosion/spaCy
2024-11-04T13:02:57Z
Thank you John6666,After hours of trying, I finally found the working solution for my project. The tricky part was identifying compatible versions of all the packages I needed. The result of this effort is this Docker image:spacy-intents, which is optimized for intent recognition tasks and fine-tuning spaCy models. Maybe someone will find it useful.