Large Language Models, and ChatGPT in particular, have recently grabbed the attention of the community and the media. Having reached high language proficiency, attention has been shifting toward its reasoning capabilities. It has been shown that ChatGPT can carry out some simple deductive reasoning steps when provided with a series of facts out of which it is tasked to draw some inferences. In this talk, I am going to argue for the need of models whose language generation is driven by an implicit reasoning process. To support my claim, I will present our evaluation of ChatGPT on the 20-Questions game, traditionally used within the Cognitive Science community to inspect the information seeking-strategy's development. This task requires a series of interconnected skills: asking informative questions, stepwise updating the hypothesis space by computing some simple deductive reasoning steps, and stopping asking questions when enough information has been collected. Thus, it is a perfect testbed to monitor the language and reasoning interplay in LLMs, shed lights on their strength and their weakness, and lay the ground for models that think while speaking.
Free University of Bozen-Bolzano (2002-20011), she worked on Natural Language Interfaces to Structured Data. In 2011, she started working on Distributional Semantics investigating its compositional properties and its integration with Computer Vision models. Since then she has mostly worked on Multimodal Models in interactive settings (e.g visual dialogues). She has recently been the EU representative within the ACL Sponsorship Board, and she is member of the ELLIS Trento unit.
Personal Website. http://disi.unitn.it/~bernardi/