A Comparative Survey of Recent Natural Language Interfaces for Databases
  Over the last few years natural language interfaces (NLI) for databases have
gained significant traction both in academia and industry. These systems use
very different approaches as described in recent survey papers. However, these
systems have not been systematically compared against a set of benchmark
questions in order to rigorously evaluate their functionalities and expressive
power.
  In this paper, we give an overview over 24 recently developed NLIs for
databases. Each of the systems is evaluated using a curated list of ten sample
questions to show their strengths and weaknesses. We categorize the NLIs into
four groups based on the methodology they are using: keyword-, pattern-,
parsing-, and grammar-based NLI. Overall, we learned that keyword-based systems
are enough to answer simple questions. To solve more complex questions
involving subqueries, the system needs to apply some sort of parsing to
identify structural dependencies. Grammar-based systems are overall the most
powerful ones, but are highly dependent on their manually designed rules. In
addition to providing a systematic analysis of the major systems, we derive
lessons learned that are vital for designing NLIs that can answer a wide range
of user questions.
