Error with scikit-learn when downloading sentence-transformers on MacOS, which I didn't get in Google Colab
I have managed to useSentenceTransformerto use BERT and establish comparisons between texts and wordshere, with Google Colab. I am now trying to do it on my Mac, in the code I will deliver, before creating a Docker that could be run on all platforms.My problem is that while I was able to download the library on Google Colab, it is not the same for MacOS. Indeed, I get:(venv) remplacement@remplacements-MacBook-Pro SugiPerso % python3 -m pip install sentence-transformers 
Collecting sentence-transformers
  Using cached sentence-transformers-2.2.2.tar.gz (85 kB)
  Preparing metadata (setup.py) ... done
Collecting transformers<5.0.0,>=4.6.0
  Using cached transformers-4.27.4-py3-none-any.whl (6.8 MB)
Requirement already satisfied: tqdm in ./venv/lib/python3.8/site-packages (from sentence-transformers) (4.65.0)
Collecting torch>=1.6.0
  Using cached torch-2.0.0-cp38-none-macosx_11_0_arm64.whl (55.8 MB)
Collecting torchvision
  Using cached torchvision-0.15.1-cp38-cp38-macosx_11_0_arm64.whl (1.4 MB)
Requirement already satisfied: numpy in ./venv/lib/python3.8/site-packages (from sentence-transformers) (1.24.2)
Collecting scikit-learn
  Using cached scikit-learn-1.2.2.tar.gz (7.3 MB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [75 lines of output]
      Ignoring numpy: markers 'python_version == "3.10" and platform_system == "Windows" and platform_python_implementation != "PyPy"' don't match your environment
      Collecting setuptools
        Using cached setuptools-67.6.1-py3-none-any.whl (1.1 MB)
      Collecting wheel
        Using cached wheel-0.40.0-py3-none-any.whl (64 kB)
      Collecting Cython>=0.29.24
        Using cached Cython-0.29.33-py2.py3-none-any.whl (987 kB)
      Collecting oldest-supported-numpy
        Using cached oldest_supported_numpy-2022.11.19-py3-none-any.whl (4.9 kB)
      Collecting scipy>=1.3.2
        Using cached scipy-1.10.1.tar.gz (42.4 MB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Installing backend dependencies: started
        Installing backend dependencies: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'error'
        error: subprocess-exited-with-error
      
        × Preparing metadata (pyproject.toml) did not run successfully.
        │ exit code: 1
        ╰─> [41 lines of output]
            The Meson build system
            Version: 1.0.1
            Source dir: /private/var/folders/9y/n84kv4sd3g3g957m35d_p3z40000gn/T/pip-install-06dhlrwi/scipy_d577af5217fe40cfaae75bed6ff2b162
            Build dir: /private/var/folders/9y/n84kv4sd3g3g957m35d_p3z40000gn/T/pip-install-06dhlrwi/scipy_d577af5217fe40cfaae75bed6ff2b162/.mesonpy-in9wd6xp/build
            Build type: native build
            Project name: SciPy
            Project version: 1.10.1
            C compiler for the host machine: cc (clang 12.0.5 "Apple clang version 12.0.5 (clang-1205.0.22.11)")
            C linker for the host machine: cc ld64 650.9
            C++ compiler for the host machine: c++ (clang 12.0.5 "Apple clang version 12.0.5 (clang-1205.0.22.11)")
            C++ linker for the host machine: c++ ld64 650.9
            Cython compiler for the host machine: cython (cython 0.29.33)
            Host machine cpu family: aarch64
            Host machine cpu: aarch64
            Compiler for C supports arguments -Wno-unused-but-set-variable: NO
            Compiler for C supports arguments -Wno-unused-function: YES
            Compiler for C supports arguments -Wno-conversion: YES
            Compiler for C supports arguments -Wno-misleading-indentation: YES
            Compiler for C supports arguments -Wno-incompatible-pointer-types: YES
            Library m found: YES
            Fortran compiler for the host machine: gfortran (gcc 12.2.0 "GNU Fortran (Homebrew GCC 12.2.0) 12.2.0")
            Fortran linker for the host machine: gfortran ld64 650.9
            Compiler for Fortran supports arguments -Wno-conversion: YES
            Checking if "-Wl,--version-script" : links: NO
            Program cython found: YES (/private/var/folders/9y/n84kv4sd3g3g957m35d_p3z40000gn/T/pip-build-env-8q2l3aja/overlay/bin/cython)
            Program python found: YES (/Users/remplacement/Documents/Work/SugiPerso/venv/bin/python3)
            Did not find pkg-config by name 'pkg-config'
            Found Pkg-config: NO
            Program pythran found: YES (/private/var/folders/9y/n84kv4sd3g3g957m35d_p3z40000gn/T/pip-build-env-8q2l3aja/overlay/bin/pythran)
            Run-time dependency threads found: YES
            Library npymath found: YES
            Library npyrandom found: YES
            Did not find CMake 'cmake'
            Found CMake: NO
            Run-time dependency openblas found: NO (tried pkgconfig, framework and cmake)
            Run-time dependency openblas found: NO (tried framework)
      
            ../../scipy/meson.build:134:0: ERROR: Dependency lookup for OpenBLAS with method 'pkgconfig' failed: Pkg-config binary for machine 1 not found. Giving up.
      
            A full log can be found at /private/var/folders/9y/n84kv4sd3g3g957m35d_p3z40000gn/T/pip-install-06dhlrwi/scipy_d577af5217fe40cfaae75bed6ff2b162/.mesonpy-in9wd6xp/build/meson-logs/meson-log.txt
            + meson setup --prefix=/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8 /private/var/folders/9y/n84kv4sd3g3g957m35d_p3z40000gn/T/pip-install-06dhlrwi/scipy_d577af5217fe40cfaae75bed6ff2b162 /private/var/folders/9y/n84kv4sd3g3g957m35d_p3z40000gn/T/pip-install-06dhlrwi/scipy_d577af5217fe40cfaae75bed6ff2b162/.mesonpy-in9wd6xp/build --native-file=/private/var/folders/9y/n84kv4sd3g3g957m35d_p3z40000gn/T/pip-install-06dhlrwi/scipy_d577af5217fe40cfaae75bed6ff2b162/.mesonpy-native-file.ini -Ddebug=false -Doptimization=2
            [end of output]
      
        note: This error originates from a subprocess, and is likely not a problem with pip.
      error: metadata-generation-failed
      
      × Encountered error while generating package metadata.
      ╰─> See above for output.
      
      note: This is an issue with the package mentioned above, not pip.
      hint: See above for details.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.It seems that the installation ofscikit-learnhas failed due to an error when compiling the compilation dependencies. Specifically, there is an error when preparingpyproject.tomlmetadata fromscipy.If it is useful, that is what I get with Google Colab:Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting sentence-transformers
  Downloading sentence-transformers-2.2.2.tar.gz (85 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.0/86.0 KB 3.1 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting transformers<5.0.0,>=4.6.0
  Downloading transformers-4.27.4-py3-none-any.whl (6.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.8/6.8 MB 25.8 MB/s eta 0:00:00
Requirement already satisfied: tqdm in /usr/local/lib/python3.9/dist-packages (from sentence-transformers) (4.65.0)
Requirement already satisfied: torch>=1.6.0 in /usr/local/lib/python3.9/dist-packages (from sentence-transformers) (1.13.1+cu116)
Requirement already satisfied: torchvision in /usr/local/lib/python3.9/dist-packages (from sentence-transformers) (0.14.1+cu116)
Requirement already satisfied: numpy in /usr/local/lib/python3.9/dist-packages (from sentence-transformers) (1.22.4)
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.9/dist-packages (from sentence-transformers) (1.2.2)
Requirement already satisfied: scipy in /usr/local/lib/python3.9/dist-packages (from sentence-transformers) (1.10.1)
Requirement already satisfied: nltk in /usr/local/lib/python3.9/dist-packages (from sentence-transformers) (3.8.1)
Collecting sentencepiece
  Downloading sentencepiece-0.1.97-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 28.9 MB/s eta 0:00:00
Collecting huggingface-hub>=0.4.0
  Downloading huggingface_hub-0.13.3-py3-none-any.whl (199 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 199.8/199.8 KB 20.0 MB/s eta 0:00:00
Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.9/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers) (6.0)
Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.9/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers) (23.0)
Requirement already satisfied: requests in /usr/local/lib/python3.9/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers) (2.27.1)
Requirement already satisfied: filelock in /usr/local/lib/python3.9/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers) (3.10.7)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.9/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers) (4.5.0)
Collecting tokenizers!=0.11.3,<0.14,>=0.11.1
  Downloading tokenizers-0.13.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.6/7.6 MB 48.9 MB/s eta 0:00:00
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.9/dist-packages (from transformers<5.0.0,>=4.6.0->sentence-transformers) (2022.10.31)
Requirement already satisfied: joblib in /usr/local/lib/python3.9/dist-packages (from nltk->sentence-transformers) (1.1.1)
Requirement already satisfied: click in /usr/local/lib/python3.9/dist-packages (from nltk->sentence-transformers) (8.1.3)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.9/dist-packages (from scikit-learn->sentence-transformers) (3.1.0)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.9/dist-packages (from torchvision->sentence-transformers) (8.4.0)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers) (3.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers) (2022.12.7)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers) (1.26.15)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers) (2.0.12)
Building wheels for collected packages: sentence-transformers
  Building wheel for sentence-transformers (setup.py) ... done
  Created wheel for sentence-transformers: filename=sentence_transformers-2.2.2-py3-none-any.whl size=125942 sha256=6374175b3d7f7989f81b48b1bd797203ecadd00d30b65bc4a7deacfdb004c9e5
  Stored in directory: /root/.cache/pip/wheels/71/67/06/162a3760c40d74dd40bc855d527008d26341c2b0ecf3e8e11f
Successfully built sentence-transformers
Installing collected packages: tokenizers, sentencepiece, huggingface-hub, transformers, sentence-transformers
Successfully installed huggingface-hub-0.13.3 sentence-transformers-2.2.2 sentencepiece-0.1.97 tokenizers-0.13.2 transformers-4.27.4I have a MacOS BigSure with an Apple M1 chip