Incorporating Measures of Intermodal Coordinationin Automated Analysis of Infant-Mother Interaction
Published in ICMI 2020, 2020
Recommended citation: Klein, L., Ardulov, V., Hu, Y., Soleymani, M., Gharib, A., Thompson, B., ... & Matarić, M. J. (2020, October). Incorporating Measures of Intermodal Coordination in Automated Analysis of Infant-Mother Interaction. In Proceedings of the 2020 International Conference on Multimodal Interaction (pp. 287-295). https://dl.acm.org/doi/abs/10.1145/3382507.3418870
Abstract: Interactions between infants and their mothers can provide meaningful insight into the dyad’s health and well-being. Previous work has shown that infant-mother coordination, within a single modality, varies significantly with age and interaction quality. However, as infants are still developing their motor, language, and social skills, they may differ from their mothers in the modes they use to communicate. This work examines how infant-mother coordination across modalities can expand researchers’ abilities to observe meaningful trends in infant-mother interactions. Using automated feature extraction tools, we analyzed the head position, arm position, and vocal fundamental frequency of mothers and their infants during the Face-to-Face Still-Face (FFSF) procedure. A de-identified dataset including these features was made available online as a contribution of this work. Analysis of infant behavior over the course of the FFSF indicated that the amount and modality of infant behavior change evolves with age. Evaluating the interaction dynamics, we found that infant and mother behavioral signals are coordinated both within and across modalities, and that levels of both intramodal and intermodal coordination vary significantly with age and across stages of the FFSF. These results support the significance of intermodal coordination when assessing changes in infant-mother interaction across conditions.
Published in the Proceeding of Internation Conference of Multimodal Interaction