小柯机器人

荷兰马斯特里赫特大学Elia Formisano和法国CNRS 和艾克斯-马赛大学Bruno L. Giordano共同合作，近期取得重要工作进展。他们研究利用中间声学到语义表示将行为和神经反应与自然声音联系起来。相关研究成果2023年3月16日在线发表于《自然—神经科学》杂志上。

据介绍，识别声音涉及输入波形到语义表示的大脑转换。尽管过去的研究确定颞上回（STG）是关键的皮层区域，但这些大脑转换的计算指纹仍然很难表征。

研究人员利用模型比较框架，对比了声学、语义（连续和分类）和声音事件深度神经网络表示模型预测感知声音差异和7T人类听觉皮层功能性磁共振成像反应的能力。研究人员证实，频谱时间调制可以预测早期听觉皮层（黑索氏回）的反应，听觉维度（例如响度、周期性）可以预测STG反应和感知差异。声音事件深度神经网络预测黑索氏回反应类似于声学模型，但值得注意的是，它们在预测STG反应和感知差异方面优于所有竞争模型。

总之，这一研究结果表明，STG 需要中间声学到语义的声音表示，声学模型和语义模型都无法解释。 这些表示本质上是组合的，并且与行为相关。

附：英文原文

Title: Intermediate acoustic-to-semantic representations link behavioral and neural responses to natural sounds

Author: Giordano, Bruno L., Esposito, Michele, Valente, Giancarlo, Formisano, Elia

Issue&Volume: 2023-03-16

Abstract: Recognizing sounds implicates the cerebral transformation of input waveforms into semantic representations. Although past research identified the superior temporal gyrus (STG) as a crucial cortical region, the computational fingerprint of these cerebral transformations remains poorly characterized. Here, we exploit a model comparison framework and contrasted the ability of acoustic, semantic (continuous and categorical) and sound-to-event deep neural network representation models to predict perceived sound dissimilarity and 7T human auditory cortex functional magnetic resonance imaging responses. We confirm that spectrotemporal modulations predict early auditory cortex (Heschl’s gyrus) responses, and that auditory dimensions (for example, loudness, periodicity) predict STG responses and perceived dissimilarity. Sound-to-event deep neural networks predict Heschl’s gyrus responses similar to acoustic models but, notably, they outperform all competing models at predicting both STG responses and perceived dissimilarity. Our findings indicate that STG entails intermediate acoustic-to-semantic sound representations that neither acoustic nor semantic models can account for. These representations are compositional in nature and relevant to behavior.

DOI: 10.1038/s41593-023-01285-9

Source: https://www.nature.com/articles/s41593-023-01285-9

Nature Neuroscience：《自然—神经科学》，创刊于1998年。隶属于施普林格·自然出版集团，最新IF：28.771
官方网址：https://www.nature.com/neuro/
投稿链接：https://mts-nn.nature.com/cgi-bin/main.plex