In the past years, multi-modal human-machine-interaction has gained a lot of interest. The main reason is the fact that modern systems, though usually being very powerful and efficient, have substantial deficits concerning the intuitive operability by the actual user. That is because of the insufficient and little intuitive user guidance, caused by the flawed or inflexible dialog modeling and usually insufficient cognitive abilities of such systems, restricting the possibilities of situation-dependent reactions.

A special focus is set on the detailed analysis of the human interaction partner's face, because the cognition of a person and interpretation of its mimics play a crucial role in human-human communication, as well. There are existing methods that already yield good results under ideal conditions, but there is a strong need for research of the applicability under the broad spectrum of real world conditions.

Furthermore, multi-modal tracking algorithms are under development, in order to keep gapless contact to the interacting persons.