How to use website search functionality with my LLM
Hi everyone,I’m a beginner who discovered Hugging Face a few days ago and I’m really impressed by what we can do here.I was wondering if it’s possible to replicate the “domain search” feature (like in HuggingChat) for my own custom chatbots, essentially using it as a RAG  approach.Is there a straightforward way to crawl or connect data from a website URL for that purpose? If so, could you please explain how or point me to any relevant tools or examples?Thanks a lot !nyriCapture d’écran 2025-01-11 à 12.03.05758×866 65.9 KB
2025-01-11T11:51:01Z
Hello. There are two broad methods. One is to process the results of a normal web search using a programming language such as Python and pass the results to LLM yourself. The other is a method calledFunction Calling, in which you instruct LLM to execute a search tool and return the results. (There are various names for this method.)In the case of the former, there are various useful libraries, so you should try searching for them. If you can use the latter, it is usually built into the LLM execution environment, so it is often found somewhere in the documentation.Pass the results of a normal web search to LLMhttps://pypi.org/project/duckduckgo-search/github.comGitHub - alby13/ollama-internet-search-tool: Ollama AI Internet Search Tool for Local AI. A...Ollama AI Internet Search Tool for Local AI. A Python program that uses DuckDuckGo search results and AI response.Function Callingollama.comTool support · Ollama BlogOllama now supports tool calling with popular models such as Llama 3.1. This enables a model to answer a given prompt using tool(s) it knows about, making it possible for models to perform more complex tasks or interact with the outside world.