Using a paid inference end point to query llamaindex knowledge graph gives worse results than the free inference api
HiI have successfully followed this:https://medium.aiplanet.com/implement-rag-with-knowledge-graph-and-llama-index-6a3370e93cddand used it to read a 2 page pdf. When I submit queries, the results are very good (well, as good as I need).To speed things up to use a larger document I have created an inference endpoint based on HuggingFaceH4/zephyr-7b-beta and access it viallm = HuggingFaceInferenceAPI(
    model_name="https://my_endpoint_ref.aws.endpoints.huggingface.cloud", token=REDACTED_TOKEN
)If I now provide a 20 page pdf, submitting the same query now gives me poor results, often resulting in responses which inform that there is no relevant data in the document even though the 2 pages that I used originally are contained in the larger document.I have tried using locally hosted LLMs but as you can imagine they are too slow on my machine.Can anyone give me a clue as to why using the same model, but on a paid endpoint should give worse results?Thanks.
2024-03-08T03:05:11Z
Hi ,I am also trying to follow the same medium article and i am facing this error "ModuleNotFoundError: No module named ‘HuggingFaceInferenceAPI’ " , how did u solve this ?Thanks
2024-03-08T15:29:36Z
HiA recent update to llama index requires you to call it like this:pip install llama-index-llms-huggingface
from llama_index.llms.huggingface import HuggingFaceInferenceAPI