Unable to import faiss
Hello,I am running Transformers on a Mac OS with Python 3.8 in a virtual environment.I have faiss-cpu 1.7.0 installed in the env.(venv) sergey_mkrtchyan transformers (master) $ python
Python 3.8.6 (v3.8.6:db455296be, Sep 23 2020, 13:31:39)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> import faiss
>>> faiss.__version__
'1.7.0'However transformers repo is doing an additional check of the version of the package using importlib_metadata intransformers/src/file_utils.pywhich ends up failing on me with “RagRetriever requires the faiss library but it was not found in your environment.”_faiss_available = importlib.util.find_spec("faiss") is not None
try:
    _faiss_version = importlib_metadata.version("faiss")
    logger.debug(f"Successfully imported faiss version {_faiss_version}")
except importlib_metadata.PackageNotFoundError:
    _faiss_available = FalseThis is where it fails, mind you_faiss_available = importlib.util.find_spec("faiss")line above works just fine, but fails on the_faiss_version = importlib_metadata.version("faiss")line unable to find the faiss package.Not sure if it’s an issue in the repo or something wrong on my side. Any experience with this?Thank you!
2021-02-08T17:27:47Z
I am facing the same issue. No version of the faiss library is being recognized as present in the environment I am running (installed through conda). I did solve this problem before, but I remember it being a pain; I built the library from source but had to deal with many issues along the way.Edit:I was just able to get it to be recognised.Solution:if you are able to import the library ( from terminal start python):pythonimport faissprint(faiss)/home/akinwilson/miniconda3/envs/envRag/lib/python3.7/site-packages/faiss/init.pytake the path;/home/akinwilson/miniconda3/envs/envRag/lib/python3.7/site-packages/faissthen in your bashrc file:nano ~/.bashrcadd this line to the end of the file:export PYTHONPATH=$HOME/miniconda3/lib/python3.8/site-packages/faissNote that I have changed the home directory to a variable.save it.Then finally, set the bashrc file as source again to reload the paths;source ~/.bashrcRestart any virtual environment and everything should work fine.
2021-02-17T20:07:58Z
Thanks, just to let you know, this has been fixed on the master branch now._faiss_available = importlib.util.find_spec("faiss") is not None
try:
    _faiss_version = importlib_metadata.version("faiss")
    logger.debug(f"Successfully imported faiss version {_faiss_version}")
except importlib_metadata.PackageNotFoundError:
    try:
        _faiss_version = importlib_metadata.version("faiss-cpu")
        logger.debug(f"Successfully imported faiss version {_faiss_version}")
    except importlib_metadata.PackageNotFoundError:
        _faiss_available = False
2024-03-12T20:16:40Z
This issue still persists (if you have faiss-gpu installed)I think it should be updated as:# We need to check all three `faiss` and `faiss-cpu` and `faiss-gpu`
try:
    _faiss_version = importlib.metadata.version("faiss")
    logger.debug(f"Successfully imported faiss version {_faiss_version}")
except importlib.metadata.PackageNotFoundError:
    try:
        _faiss_version = importlib.metadata.version("faiss-cpu")
        logger.debug(f"Successfully imported faiss version {_faiss_version}")
    except importlib.metadata.PackageNotFoundError:
        try:
            _faiss_version = importlib.metadata.version("faiss-gpu")
            logger.debug(f"Successfully imported faiss version {_faiss_version}")
        except importlib.metadata.PackageNotFoundError:
            _faiss_available = False
2024-03-13T07:38:54Z
Would be great to open a PR