Demo of Open Domain Long Form Question Answering
Hi,I just wanted to try the demohttps://huggingface.co/qa/, but it seems it is down. Could we have it again please? Thanks.
2020-07-09T13:33:43Z
The demo is back on, thanks for letting us know!
2020-07-11T12:32:22Z
Hi@yjernite, thanks for your great works and great blog post!At the moment, the system is down again, so would it be possible to have Colab or Kaggle notebook, so that we can use our own GPU?EDITIt’s back! So sometimes the server just down or some problem on GPU … But standalone notebook would be great anyway
2020-07-14T14:04:09Z
Hi@Jung, there is a notebook for it:https://github.com/huggingface/notebooks/blob/master/longform-qa/Long_Form_Question_Answering_with_ELI5_and_Wikipedia.ipynb. I followed the instructions on the notebook, and after several hours (donwloading, indexing, etc, and using pre-trained model) the system is ready to get question.
2020-07-14T14:55:55Z
Thanks@cahyafor the pointer!!HF team indeed has many great notebooks where I missedI will definitely try it out!
2020-12-09T03:26:07Z
Hi Yacine,@yjerniteI believe the demo has been down (I have waited for 24 hours),https://huggingface.co/qa/return 502 Bad Gateway
2020-12-11T10:26:49Z
Hi, I have dockerized the engine, and provide a rest api to use it, if you want to tryhttps://hub.docker.com/r/wirawan/eli5I will include later a front end in to the docker image
2020-12-15T00:07:58Z
@cahyaamazing job! Thanks for sharing.BTW, did you further finetune the model or use as is ?and which beam-number did you use in the telegram bot ?
2020-12-15T12:43:30Z
Thanks. I use the model as it is without fine tuning. I use beam number 16, with max_len of 256 to reduce the text generation time. I also use cpu for the faiss, so the whole service can fit to my 8GB GPU ram. Actually it needs now only 5GB gpu ram instead of 13GB when I used gpu for the faiss index.
2021-01-03T09:35:51Z
Hi@jung@cahya, just catching up nowThanks@cahyafor the work on the docker image!@Jungthe demo is supposed to restart when it crashes but that doesn’t always work, and then I have to restart it manuallyWe’ll eventually change it so it uses our inference API to make it more stable.I’d be super interested to know what you’re using it for in the meantime!
2021-01-04T10:11:27Z
Hi@yjernite, I am trying to use it with some books instead of wiki. I will write it here if it worksUntil now, I used it in a telegram chatbot for fun (https://github.com/cahya-wirawan/AskMeAnything-Chatbot).
2021-02-05T15:09:59Z
Hi@yjernite, I did some experiments with the demo. It seems that the Bart model trained for this demo doesn’t really take the retrieved passages as source for its answer. It likes to hallucinatefor example if I ask “what is cempedak fruit”, the answer doesn’t contain any information from the retrieved passages. I think it generates text mostly only from the question and its Barts parameter.
2021-02-08T13:42:19Z
So that’s a really interesting question: the model tends to rely on documents for some questions and hallucinate more on other - if you play around, you can see that it sometimes gets a good answer using supporting documents even for examples that aren’t in the training set, but it also needs to rely on its own knowledge from time to time which can lead to mistakes.Definitely an open area of investigation!
2021-02-08T15:37:49Z
Btw, you have to try the fruit, it’s really delicious. I think@Jungcan confirm it