AiSocrates: Towards Answering Ethical Quandary Questions
Considerable advancements have been made in various NLP tasks based on the
impressive power of large pre-trained language models (LLMs). These results
have inspired efforts to understand the limits of LLMs so as to evaluate how
far we are from achieving human level general natural language understanding.
In this work, we challenge the capability of LLMs with the new task of Ethical
Quandary Generative Question Answering. Ethical quandary questions are more
challenging to address because multiple conflicting answers may exist to a
single quandary. We propose a system, AiSocrates, that provides an answer with
a deliberative exchange of different perspectives to an ethical quandary, in
the approach of Socratic philosophy, instead of providing a closed answer like
an oracle. AiSocrates searches for different ethical principles applicable to
the ethical quandary and generates an answer conditioned on the chosen
principles through prompt-based few-shot learning. We also address safety
concerns by providing a human controllability option in choosing ethical
principles. We show that AiSocrates generates promising answers to ethical
quandary questions with multiple perspectives, 6.92% more often than answers
written by human philosophers by one measure, but the system still needs
improvement to match the coherence of human philosophers fully. We argue that
AiSocrates is a promising step toward developing an NLP system that
incorporates human values explicitly by prompt instructions. We are releasing
the code for research purposes.