Decision Criteria to select a Retrieval Augmented Q&A Model
Hi there, we would like to select a model to be used on premise by our short term insurance client to perform question and answer on their corporate policies and procedures. Our intention is to leverage an existing semantic search (Watson Discovery) to pass the context text plus the original end user question to the model. Obviously the size of the requisite infrastructure on premise will guide our selection what else should we consider i.e. do we just investigate the most popular Q&A models based on download and SQUAD scores and test those? In theory we should try select those models tested with datasets similar to ours which is tricky? Anyway we have “found” the following models as potentials are these appropriate? XLnet, bert-large-uncased-whole-word-masking-finetuned-squad, deepset/roberta-base-squad2, albert-xxlarge-v2, distilbert-base-cased-distilled-squad Many thanks Alessadro