Best practice for using Transformers on GPU on EC2?
I’ve fired up an EC2 instance with a PyTorch deep learning AMI. The problem is transformers isn’t in the conda environment (someone should convince Amazon to add it!). I’ve tried installing it but that seems to be leading to other things breaking, so before I go down a giant rabbit hole, thought I’d ask here for the simplest way to get up and running with Transformers running on a GPU instance on EC2. Should I use the Huggingface Docker repo, for example? Thanks!
2021-02-01T08:40:45Z
Hey@thecity2, I am currently using HF on a EC2 g4dn.xlarge with Deep Learning Base AMI (Ubuntu 18.04) Version 32.0.The good part about the AMI is that you have CUDA ready. What I do is to create a new conda environment for HF, then Iinstall PyTorch(checking the CUDA version with the commandnvidia-smi) and HuggingFace in the new env simply by using pip.Let me know if you have any more questions.
2021-02-01T14:27:46Z
Thanks@marcoabrate! That is basically what I ended up doing although just using a plain old Python virtual environment. Seems to be working.
2021-02-03T14:15:31Z
cc@philschmid
2021-04-09T13:54:33Z
Hello,I’m also considering getting an EC2 instance to train a NER model. I’m using Spacy and BERT-multilingual-cased, and want to train it to recognize customized entities.Previously I trained this on my laptop’s CPU but it is pretty long (24hours). I also tried to run it with my main computer on a GTX-1060 but it runs out of memory immediately (4GB of VRAM is admitedly not enough but I just wanted to give it a go).Will the g4dn.xlarge be powerful enough for this task?
2021-04-10T19:17:15Z
g4dn.xlarge will do the job.Btw If you want to try your hands on training with a proper GPU for the first time I would start with a google colab notebook. You can get one with a nvidia T4 (same GPU as g4dn.xlarge).