During the International Conference on Learning Representations (ICLR) 2020 this week, which as a result of the pandemic took place virtually, Turing Award winner and director of the Montreal Institute for Learning Algorithms Yoshua Bengio provided a glimpse into the future of AI and machine learning techniques. He spoke in February at the AAAI Conference on Artificial Intelligence 2020 in New York alongside fellow Turing Award recipients Geoffrey Hinton and Yann LeCun. But in a lecture published Monday, Bengio expounded upon some of his earlier themes.
One of those was attention — in this context, the mechanism by which a person (or algorithm) focuses on a single element or a few elements at a time. It’s central both to machine learning model architectures like Google’s Transformer and to the bottleneck neuroscientific theory of consciousness, which suggests that people have limited attention resources, so information is distilled down in the brain to only its salient bits. Models with attention have already achieved state-of-the-art results in domains like natural language processing, and they could form the foundation of enterprise AI that assists employees in a range of cognitively demanding tasks.
链接地址：https://venturebeat.com/2020/04/28/yoshua-bengio-attention-is-a-core-ingredient-of-consciousness-ai/