Spoken language-based interfaces and dialogue systems are on the rise thanks to advances in Machine Learning. However, we are still far away from being able to develop and deploy social robots or virtual assistants that are capable of fluent, flexible and robust communication in cooperative tasks with human users. One main challenge that still remains is the modeling of face-to-face dialogue with embodied conversational agents, i.e. the orchestration of humanlike understanding and generation of multi-channel, multi-functional, and multi-modal communication embedded in a coherent and fluent dialogue. I will present work that pursues this goal by bringing together data-based and model-based approaches in a framework for „interaction-aware“ dialogue processing. Issues covered will include the realtime adaptation to communication partners at different levels of dialogue processing, the use of semantic and pragmatic functions of nonverbal communication, and the perception of such conversational agents by human users in different contexts of use.