Attention is the important ability to flexibly control limited computational resources. It has been studied in conjunction with many other topics in neuroscience and psychology including awareness, vigilance, saliency, executive control, and learning. It has also recently been applied in several domains in machine learning. The relationship between the study of biological attention and its use as a tool to enhance artificial neural networks is not always clear. This review starts by providing an overview of how attention is conceptualized in the neuroscience and psychology literature. It then covers several use cases of attention in machine learning, indicating their biological counterparts where they exist. Finally, the ways in which artificial attention can be further inspired by biology for the production of complex and integrative systems is explored.
1. Introduction 2. Attention in Neuroscience and Psychology 2.1. Attention as Arousal, Alertness, or Vigilance 2.2. Sensory Attention 2.3. Attention and Executive Control 2.4. Attention and Memory 3. Attention in Machine Learning 3.1. Attention for Natural Language Processing 3.2. Attention for Visual Tasks 3.3. Multi-Task Attention 3.4. Attention to Memory 4. Ideas for Future Interaction Between Artificial and Biological Attention 4.1. How to Enhance Performance 4.2. How to Deploy Attention 4.3. Attention and Learning 4.4. Limitations of Attention: Bugs or Features? 5. Conclusions 链接地址： https://www.frontiersin.org/articles/10.3389/fncom.2020.00029/full