Apple silicon Installation of Transformers
What follows is a ‘hint sequence’ to get Transformers working on Big Sur 11.1 @ Apple Silicon M1.Please try it with most care, and consider it “AS IS” -it has been proven twice, but I cannot be responsible for any unintended behaviour-Install Xcode cmd line tool chain$ xcode-select --installInstall homebrew/bin/bash -c "(curl -fsSLhttps://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"…and a couple of utils. Rustup from source is the key to get Transformers working on M1.brew install vim wgetbrew install --build-from-source rustup-initInstall miniforge -put it in a temp directory of your choice-wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.shbash Miniforge3-MacOSX-arm64.shrefresh your env variables (run again the shell bash/zsh: effective while inelegant)bashconda config --set auto_activate_base false && conda deactivateDownload TensorFlow 2.4 from Apple M1 BUT DON’T INSTALL IT, YET!$ cd $HOME && git clonehttps://github.com/apple/tensorflow_macos.git&& cd tensorflow_macos/arm64Create a Conda (virtual) environment -and always restart a shell session-bash && conda create --name tf24transformersconda activate tf24transformers && conda install -y python=3.8.6$ conda install -y pandas matplotlib scikit-learn jupyterlabInstall Apple Tensorflow (ATF)2.4. You’re already in arm64. (If not cd to $HOME/tensorflow_macos/arm64).7.1)$ pip install --force pip==20.2.4 wheel setuptools cached-property six7.2) Install the packages from Apple$ pip install --upgrade --no-dependencies --force numpy-1.18.5-cp38-cp38-macosx_11_0_arm64.whl grpcio-1.33.2-cp38-cp38-macosx_11_0_arm64.whl h5py-2.10.0-cp38-cp38-macosx_11_0_arm64.whl tensorflow_addons-0.11.2+mlcompute-cp38-cp38-macosx_11_0_arm64.whl7.3) Some dependencies…$ pip install absl-py astunparse flatbuffers gast google_pasta keras_preprocessing opt_einsum protobuf tensorflow_estimator termcolor typing_extensions wrapt wheel tensorboard typeguard7.4) Install ATF$ pip install --upgrade --force --no-dependencies tensorflow_macos-0.1a1-cp38-cp38-macosx_11_0_arm64.whl7.5) Refresh your shell once more and you are donebash && conda activate tf24transformerspython -m pip install --upgrade pippip install tokenizers transformerspython -c “from transformers import pipeline; print(pipeline(‘sentiment-analysis’)(‘we love you’))”[{‘label’: ‘POSITIVE’, ‘score’: 0.9998704791069031}]Many thanks to Fabrice Daniel for the tutorial about AFT2.4.CiaoErnesto
2021-01-14T16:03:57Z
what’s up with step 2? What does it say?thiscan't be that hardDo you mean?brew install vimwgetand thenbrew install --build-from-source rustup-init
2021-01-18T16:22:33Z
@emininno?
2021-01-29T11:58:50Z
How did you manage to install tokenizers? The tokenizers version required for transformers is not compatible with the native rust version. I got builds error whenpip install tokenizers transformers
2021-04-17T16:46:42Z
How did you get transformers to see the TensorFlow installation?None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.