Postdoc position on Non-Verbal Behavior and Attention sensing in the wild F / M

Role Summary

The Perception and Activity Understandin g group seeks one motivated PostDoctoral researcher to join the project TUNASBE (Towards a Unified Understanding of Attention and Social Behaviors In-the-Wild). TUNASBE is a project funded by the SNSF aiming to advance AI’s understanding and interpretation of human states, activities, and behaviors in natural settings. Beyond recognizing basic actions, the focus is on discerning complex social behaviors by interpreting non-verbal cues, such as gaze, gestures, emotion, attention and intentions. These elements are central to social intelligence, enabling AI to predict others’ actions, infer mental states, and facilitate meaningful social interactions.

The project uses gaze following as an anchor, linking attention with broader social cues to explore human-centric behaviors across different scenes. Current computational models largely reduce gaze following to a machine-learning problem of predicting 2D gaze target locations. However, this project intends to design more holistic models that go beyond this narrow focus, incorporating contextual, semantic, and social dimensions of gaze, attention, and related behaviors.

The goal is to build on the team's expertise on non-verbal behaviors and attention sensing, and develop unified models capable of leveraging multiple social cues to improve representation and performance through a multi-task learning framework at both the subject level (head and facial behavior analysis) and the scene level. Challenges include refining gaze and attentiveness estimations, handling dynamic head gestures, integrating multiple cues, integrating audio and language models,  and creating in-the-wild datasets that encompass a range of social behavior,. Addressing these will push forward human-centric computer vision and open new possibilities for real-world applications.

T he project will be supported by two PhD students and one postdoc. They will work on the design of computational models for the project tasks, integrating existing datasets and devising new ones.

Your profile

The ideal candidate will hold a Ph.D. in Computer Science, Electrical Engineering, or a related field, with a strong emphasis on AI and computer vision. They should have extensive experience in designing, implementing, training, and validating computational models and algorithms, particularly within the machine learning and temporal modeling domains. Proficiency in Python is essential, along with expertise in deep learning frameworks like PyTorch. Familiarity with human processing and social behavior analysis is a plus. The candidate should excel in problem-solving and possess the ability to work effectively within a multidisciplinary team. Strong English communication skills, both written and verbal, are crucial for articulating complex technical concepts and research findings. Furthermore, as a postdoctoral researcher, you will have the opportunity to play a central role in co-supervising PhD students, and coordinating technical effort in the team, including data annotation, tool developments, as well as to handle coordination activities with external partners.

The postdoc position is for one year, renewable for two years depending on results.

How to apply

Please provide the following information, as two separate pdf files :

File 1 : (i) updated CV with list of publications; (ii) grades from Bachelor and Master.

File 2 : (i) a motivation letter (max 2 pages); (ii) a list of 3 relevant published works including links for access; (iii) reference letters (if available).

Contact details of 3 reference persons.

For information, you can contact Dr Jean-Marc Odobez

Application deadline

Screening of applications will begin on December 1, 2024, and will continue until the position is filled.

About Idiap

Idiap is a research institute of national importance that engages in fundamental research, education and technology transfer in artificial intelligence, machine learning and signal processing. Idiap offers competitive salaries and conditions in a young, high-quality, dynamic, and multicultural environment. Idiap is located in the town of Martigny in Valais, a scenic region in the south of Switzerland, surrounded by the highest mountains of Europe, and offering exceptional quality of life, exciting recreational activities, including hiking, climbing and skiing, as well as varied cultural activities. English is the official working language.

At Idiap we place great emphasis on diversity and we know diversity fosters creativity and innovation. We are committed to equality of opportunity, to being fair and inclusive, and to being a place where we all belong. Employment at Idiap is based solely on a person's merit and qualifications. Idiap does not discriminate against any employee or applicant because of race, color, religion, gender, sexual orientation, gender identity / expression, national origin, disability, age, marital status, pregnancy, or any other basis protected by law.