来源：专知

Jane Wang是DeepMind神经科学团队的一名研究科学家，研究元强化学习和受神经科学启发的人工智能代理。她的背景是物理、复杂系统、计算和认知神经科学。

Kevin Miller是DeepMind神经科学团队的研究科学家，也是伦敦大学学院的博士后。他目前正在研究如何理解mice和机器的结构化强化学习。

Adam Marblestone是施密特期货创新公司(Schmidt Futures innovation)的研究员，曾是DeepMind的研究科学家，此前他获得了生物物理学博士学位，并在一家脑机接口公司工作。

Where Neuroscience Meets AI

地址：

https://sites.google.com/view/neurips-2020-tutorial-neurosci/home

大脑仍然是唯一已知的真正通用智能系统的例子。对人类和动物认知的研究已经揭晓了一些关键的见解，如并行分布式处理、生物视觉和从奖赏信号中学习的想法，这些都极大影响了人工学习系统的设计。许多人工智能研究人员继续将神经科学视为灵感和洞察力的来源。一个关键的困难是，神经科学是一个广泛的、异质的研究领域，包括一系列令人困惑的子领域。在本教程中，我们将从整体上对神经科学进行广泛的概述，同时重点关注两个领域——计算认知神经科学和电路学习的神经科学——我们认为这两个领域对今天的人工智能研究人员尤其相关。最后，我们将强调几项正在进行的工作，这些工作试图将神经科学领域的见解引入人工智能，反之亦然。

概要：

1. 概述 Introduction / background (15 min)

2. 认知神经科学 Cognitive neuroscience (30 min)

3. 学习电路与机制神经科学， Learning circuits and mechanistic neuroscience (30 min)

4. 交叉最新进展 Recent advancements at the interp (25 min)

https://sites.google.com/view/neurips-2020-tutorial-neurosci/home

参考文献：

Section 1 - Cognitive Neuroscience

Textbooks

• Gazzaniga, M., Ivry, R. B., & Mangun, G. R. (2018). Cognitive Neuroscience. W.W. Norton & Company.

• O’Reilly, R. C., & Munakata, Y. (2000). Computational Explorations in Cognitive Neuroscience. MIT Press.

• Abbot, L. F., & Dayan, P. (2001). Theoretical neuroscience. MIT Press.

Reviews: Innateness

• Zador, A. M. (2019). A critique of pure learning and what artificial neural networks can learn from animal brains. Nature Communications.

• Pinker, S. (2003). The Language Instinct: How the Mind Creates Language. Penguin UK.

• Hinton, G. E., & Nowlan, S. J. (1987). How learning can guide evolution. Complex Systems

Reviews: Vision

• Goodale, M. A., & Milner, A. D. (1992). Separate visual pathways for perception and action. Trends in Neurosciences,

• Andersen, R. A., & Buneo, C. A. (2002). Intentional Maps in Posterior Parietal Cortex. Annual Review of Neuroscience .

• Whitwell, R. L., Milner, A. D., & Goodale, M. A. (2014). The Two Visual Systems Hypothesis: New Challenges and Insights from Visual form Agnosic Patient DF. Frontiers in Neurology.

• DiCarlo, J. J., & Cox, D. D. (2007). Untangling invariant object recognition. Trends in Cognitive Sciences.

• Dehaene, S., & Cohen, L. (2011). The unique role of the visual word form area in reading. Trends in Cognitive Sciences.

• Kanwisher, N., & Yovel, G. (2006). The fusiform face area: a cortical region specialized for the perception of faces. Phil. Trans. Royal Society of London: B.

Reviews: Memory

• Squire, L. R. (2009). The legacy of patient HM for neuroscience. Neuron.

• McClelland, J. L., McNaughton, B. L., & O’Reilly, R. C. (1995). Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological Review.

• O’Reilly, R. C., & Norman, K. A. (2002). Hippocampal and neocortical contributions to memory: advances in the complementary learning systems framework. Trends in Cognitive Sciences.

Reviews: Planning

• Unterrainer, J. M., & Owen, A. M. (2006). Planning and problem solving: from neuropsychology to functional neuroimaging. Journal of Physiology.

• Dolan, R. J., & Dayan, P. (2013). Goals and habits in the brain. Neuron

• Miller, K. J., & Venditto, S. J. C. (2021). Multi-step planning in the brain. Current Opinion in Behavioral Sciences

Other works cited

• Akrami, A., Kopec, C. D., Diamond, M. E., & Brody, C. D. (2018). Posterior parietal cortex represents sensory history and mediates its effects on behaviour. Nature.

• Andersen, R. A., Andersen, K. N., Hwang, E. J., & Hauschild, M. (2014). Optic ataxia: from Balint’s syndrome to the parietal reach region. Neuron.

• Balaguer, J., Spiers, H., Hassabis, D., & Summerfield, C. (2016). Neural Mechanisms of Hierarchical Planning in a Virtual Subway Network. Neuron.

• Chomsky, N. (1976). Reflections on Language. Fontana Press.

• Cohen, N. J., & Squire, L. R. (1980). Preserved learning and retention of pattern-analyzing skill in amnesia: dissociation of knowing how and knowing that. Science.

• Dagher, A., Owen, A. M., Boecker, H., & Brooks, D. J. (1999). Mapping the network for planning: a correlational PET activation study with the Tower of London task. Brain.

• Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans’ choices and striatal prediction errors. Neuron.

• Doll, B. B., Duncan, K. D., Simon, D. A., Shohamy, D., & Daw, N. D. (2015). Model-based choices involve prospective neural activity. Nature Neuroscience.

• Downing, P. E., Jiang, Y., Shuman, M., & Kanwisher, N. (2001). A cortical area selective for visual processing of the human body. Science.

• Epstein, R., & Kanwisher, N. (1998). A cortical representation of the local visual environment. Nature.

• Fernando, C., Sygnowski, J., Osindero, S., Wang, J., Schaul, T., Teplyashin, D., Sprechmann, P., Pritzel, A., & Rusu, A. (2018). Meta-learning by the Baldwin effect. Proceedings of the Genetic and Evolutionary Computation Conference.

• Gabrieli, J. D., Corkin, S., Mickel, S. F., & Growdon, J. H. (1993). Intact acquisition and long-term retention of mirror-tracing skill in Alzheimer’s disease and in global amnesia. Behavioral Neuroscience.

• Goodale, M. A., Milner, A. D., Jakobson, L. S., & Carey, D. P. (1991). A neurological dissociation between perceiving objects and grasping them. Nature.

• Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T., Agapiou, J., Badia, A. P., Hermann, K. M., Zwols, Y., Ostrovski, G., Cain, A., King, H., Summerfield, C., Blunsom, P., Kavukcuoglu, K., & Hassabis, D. (2016). Hybrid computing using a neural network with dynamic external memory. Nature.

• Guo, Z. V., Li, N., Huber, D., Ophir, E., Gutnisky, D., Ting, J. T., Feng, G., & Svoboda, K. (2014). Flow of cortical activity underlying a tactile decision in mice. Neuron.

• Holmes, G. (1918). Dsturbances of visual orientation. British Journal of Ophthalmology

• Hopcraft, R., Holdo, R., Mwangomo, E., Mduma, S., Thirgood, S., Borner, M., Fryxell, J., Olff, J., & Sinclair, A. (2015). Why are wildebeest the most abundant herbivore in the Serengeti ecosystem? In A. Sinclair, K. Metzger, S. Mduma, & J. Fryxell (Eds.), Serengeti IV: Sustaining biodiversity in a coupled human-natural system. University of Chicago Press.

• James, T. W., Culham, J., Humphrey, G. K., Milner, A. D., & Goodale, M. A. (2003). Ventral occipital lesions impair object recognition but not object-directed grasping: an fMRI study. Brain

• Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: a module in human extrastriate cortex specialized for face perception. J. Neuroscience

• Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM.

• Kurth-Nelson, Z., Economides, M., Dolan, R. J., & Dayan, P. (2016). Fast Sequences of Non-spatial State Representations in Humans. Neuron.

• Miller, G. A., Galanter, E., & Pribram, K. H. (1960). Plans and the structure of behavior.

• Miller, K. J., Botvinick, M. M., & Brody, C. D. (2017). Dorsal hippocampus contributes to model-based planning. Nature Neuroscience.

• Milner, A. D., Perrett, D. I., Johnston, R. S., Benson, P. J., Jordan, T. R., Heeley, D. W., Bettucci, D., Mortara, F., Mutani, R., & Terazzi, E. (1991). Perception and action in “visual form agnosia.” Brain.

• Milner, B., Corkin, S., & Teuber, H.-L. (1968). Further analysis of the hippocampal amnesic syndrome: 14-year follow-up study of H.M. Neuropsychologia.

• Murata, A., Gallese, V., Luppino, G., Kaseda, M., & Sakata, H. (2000). Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP. J. Neurophysiology.

• Naselaris, T., Bassett, D. S., Fletcher, A. K., Kording, K., Kriegeskorte, N., Nienborg, H., Poldrack, R. A., Shohamy, D., & Kay, K. (2018). Cognitive Computational Neuroscience: A New Conference for an Emerging Discipline. Trends in Cognitive Sciences.

• Neisser, U. (2014). Cognitive Psychology

• Norman, K. A., & O’Reilly, R. C. (2003). Modeling hippocampal and neocortical contributions to recognition memory: a complementary-learning-systems approach. Psychological Review.

• Pflugshaupt, T., Gutbrod, K., Wurtz, P., von Wartburg, R., Nyffeler, T., de Haan, B., Karnath, H.-O., & Mueri, R. M. (2009). About the role of visual field defects in pure alexia. Brain.

• Pritzel, A., Uria, B., Srinivasan, S., Puigdomènech, A., Vinyals, O., Hassabis, D., Wierstra, D., & Blundell, C. (2017). Neural Episodic Control. arXiv [cs.LG]

• Russell, S., & Norvig, P. (2002). Artificial Intelligence: A Modern Approach

• Schapiro, A. C., Turk-Browne, N. B., Botvinick, M. M., & Norman, K. A. (2017). Complementary learning systems within the hippocampus: a neural network modelling approach to reconciling episodic memory with statistical learning. Phil. Trans. Royal Society of London: B.

• Shallice, T. (1982). Specific impairments of planning. Phil. Trans. Royal Society of London: B.

• Shen, E. H., Overly, C. C., & Jones, A. R. (2012). The Allen Human Brain Atlas: comprehensive gene expression mapping of the human brain. Trends in Neurosciences

• Snyder, L. H., Batista, A. P., & Andersen, R. A. (1997). Coding of intention in the posterior parietal cortex. Nature.

• Starck, J. M., Ricklefs, R. E., & Others. (1998). Avian growth and development: evolution within the altricial-precocial spectrum. Oxford University Press.

• Steinmetz, N. A., Zatka-Haas, P., Carandini, M., & Harris, K. D. (2019). Distributed coding of choice, action, and engagement across the mouse brain. Nature.

• Stringer, C., Pachitariu, M., Steinmetz, N., Carandini, M., & Harris, K. D. (2019). High-dimensional geometry of population responses in visual cortex. Nature.

• Szwed, M., Dehaene, S., Kleinschmidt, A., Eger, E., Valabrègue, R., Amadon, A., & Cohen, L. (2011). Specialization for written words over objects in the visual cortex. NeuroImage.

• Tsao, D. Y., Freiwald, W. A., Tootell, R. B. H., & Livingstone, M. S. (2006). A cortical region consisting entirely of face-selective cells. Science.

• van Opheusden, B., Galbiati, G., Z. Bnaya Li, Y., & Ma, W. J. (2017). Modeling Decision Tree Search in a Two-Player Game. Proceedings of the 39th Annual Meeting of the Cognitive Science Society.

• Vesia, M., & Crawford, J. D. (2012). Specialization of reach function in human posterior parietal cortex. Experimental Brain Research.

• Vikbladh, O., Meager, M. R., King, J., Blackmon, K., Devinsky, O., Shohamy, D., Burgess, N., & Daw, N. D. (2018). Two Sides of the Same Coin: The Hippocampus as a Common Neural Substrate for Model-Based Planning and Spatial Memory. Neuron

• Wada, Y., & Yamamoto, T. (2001). Selective impairment of facial recognition due to a haematoma restricted to the right fusiform and lateral occipital region. Journal of Neurology, Neurosurgery, and Psychiatry.

• Wallace, D. J., Greenberg, D. S., Sawinski, J., Rulla, S., Notaro, G., & Kerr, J. N. D. (2013). Rats maintain an overhead binocular field at the expense of constant fusion. Nature.

• Williams, S. C. P., & Deisseroth, K. (2013). Optogenetics. PNAS.

• Wimmer, R. D., Ian Schmitt, L., Davidson, T. J., Nakajima, M., Deisseroth, K., & Halassa, M. M. (2015). Thalamic control of sensory selection in divided attention. Nature.

• Zoccolan, D., Kouh, M., Poggio, T., & DiCarlo, J. J. (2007). Trade-off between object selectivity and tolerance in monkey inferotemporal cortex. J Neuroscience

• Zoccolan, D., Oertelt, N., DiCarlo, J. J., & Cox, D. D. (2009). A rodent model for the study of invariant visual object recognition. PNAS

Section 2 - Circuits and Mechanistic Neuroscience

• Shepherd, Gordon M. The synaptic organization of the brain. Oxford university press, 2004.

• Fee, Michale S., and Jesse H. Goldberg. A hypothesis for basal ganglia-dependent reinforcement learning in the songbird. Neuroscience 198 (2011): 152-170.

• Kornfeld, J., Januszewski, M., Schubert, P., Jain, V., Denk, W., & Fee, M. S. (2020). An anatomical substrate of credit assignment in reinforcement learning. BioRxiv.

• Murdoch, D., Chen, R. and Goldberg, J.H., 2018. Place preference and vocal learning rely on distinct reinforcers in songbirds. Scientific reports, 8(1), pp.1-9.

• Gielow, M.R. and Zaborszky, L., 2017. The input-output relationship of the cholinergic basal forebrain. Cell reports, 18(7), pp.1817-1830.

• Chubykin, A.A., Roach, E.B., Bear, M.F. and Shuler, M.G.H., 2013. A cholinergic mechanism for reward timing within primary visual cortex. Neuron, 77(4), pp.723-735.

• O'Reilly, R.C., Hazy, T.E., Mollick, J., Mackie, P. and Herd, S., 2014. Goal-driven cognition in the brain: a computational framework. arXiv preprint arXiv:1404.7591.

• O'Reilly, R.C., Wyatte, D. and Rohrlich, J., 2014. Learning through time in the thalamocortical loops. arXiv preprint arXiv:1407.3432.

• Guerguiev, J., Lillicrap, T.P. and Richards, B.A., 2017. Towards deep learning with segregated dendrites. ELife, 6, p.e22901.

• Sacramento, J., Costa, R.P., Bengio, Y. and Senn, W., 2018. Dendritic cortical microcircuits approximate the backpropagation algorithm. In Advances in neural information processing systems (pp. 8721-8732).

• Körding, K.P. and König, P., 2001. Supervised and unsupervised learning with two sites of synaptic integration. Journal of computational neuroscience, 11(3), pp.207-215.

• Whittington, J.C. and Bogacz, R., 2019. Theories of error back-propagation in the brain. Trends in cognitive sciences, 23(3), pp.235-250.

• Clancy, K.B., Koralek, A.C., Costa, R.M., Feldman, D.E. and Carmena, J.M., 2014. Volitional modulation of optically recorded calcium signals during neuroprosthetic learning. Nature neuroscience, 17(6), pp.807-809.

• George, D., Lazaro-Gredilla, M., Lehrach, W., Dedieu, A. and Zhou, G., 2020. A detailed mathematical theory of thalamic and cortical microcircuits based on inference in a generative vision model. bioRxiv.

• Heeger, D.J., 2017. Theory of cortical function. Proceedings of the National Academy of Sciences, 114(8), pp.1773-1782.

• Rolls, E.T., 1996. A theory of hippocampal function in memory. Hippocampus, 6(6), pp.601-620.

• Pfeiffer, B.E. and Foster, D.J., 2015. Autoassociative dynamics in the generation of sequences of hippocampal place cells. Science, 349(6244), pp.180-183.

• Carrillo-Reid, L., Yang, W., Bando, Y., Peterka, D.S. and Yuste, R., 2016. Imprinting and recalling cortical ensembles. Science, 353(6300), pp.691-694.

• Müller, M.G., Papadimitriou, C.H., Maass, W. and Legenstein, R., 2020. A model for structured information representation in neural networks of the brain. Eneuro, 7(3).

• Abbott, L.F., Bock, D.D., Callaway, E.M., Denk, W., Dulac, C., Fairhall, A.L., Fiete, I., Harris, K.M., Helmstaedter, M., Jain, V. and Kasthuri, N., 2020. The Mind of a Mouse. Cell, 182(6), pp.1372-1376.

• Turner, N.L., Macrina, T., Bae, J.A., Yang, R., Wilson, A.M., Schneider-Mizell, C., Lee, K., Lu, R., Wu, J., Bodor, A.L. and Bleckert, A.A., 2020. Multiscale and multimodal reconstruction of cortical structure and function. bioRxiv.

• George D, Lehrach W, Kansky K, Lázaro-Gredilla M, Laan C, Marthi B, Lou X, Meng Z, Liu Y, Wang H, Lavin A. A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs. Science. 2017 Dec 8;358(6368).

Section 3 - Recent advancements at the interp

• Yamins, D. L., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014). Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences, 111(23), 8619-8624.

• Yamins, D. L., & DiCarlo, J. J. (2016). Using goal-driven deep learning models to understand sensory cortex. Nature neuroscience, 19(3), 356-365.

• Mante, V., Sussillo, D., Shenoy, K. V., & Newsome, W. T. (2013). Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature, 503(7474), 78-84.

• Song, H. F., Yang, G. R., & Wang, X. J. (2017). Reward-based training of recurrent neural networks for cognitive and value-based tasks. Elife, 6, e21492.

• Yang, G. R., Joglekar, M. R., Song, H. F., Newsome, W. T., & Wang, X. J. (2019). Task representations in neural networks trained to perform many cognitive tasks. Nature neuroscience, 22(2), 297-306.

• Dezfouli, A., Morris, R., Ramos, F. T., Dayan, P., & Balleine, B. (2018). Integrated accounts of behavioral and neuroimaging data using flexible recurrent neural network models. In Advances in Neural Information Processing Systems (pp. 4228-4237).

• Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., ... & Botvinick, M. (2018). Prefrontal cortex as a meta-reinforcement learning system. Nature Neuroscience, 21(6), 860-868.

• Dabney, W., Kurth-Nelson, Z., Uchida, N., Starkweather, C. K., Hassabis, D., Munos, R., & Botvinick, M. (2020). A distributional code for value in dopamine-based reinforcement learning. Nature, 577(7792), 671-675.

• Akrout, M., Wilson, C., Humphreys, P., Lillicrap, T., & Tweed, D. B. (2019). Deep learning without weight transport. In Advances in neural information processing systems (pp. 976-984).

• Miconi, T. (2017). Biologically plausible learning in recurrent neural networks reproduces neural dynamics observed during cognitive tasks. Elife, 6, e20899.

• Bellec, G., Salaj, D., Subramoney, A., Legenstein, R., & Maass, W. (2018). Long short-term memory and learning-to-learn in networks of spiking neurons. In Advances in Neural Information Processing Systems (pp. 787-797).

• Merel, J., Aldarondo, D., Marshall, J., Tassa, Y., Wayne, G., & Ölveczky, B. (2019). Deep neuroethology of a virtual rodent. In International Conference on Learning Representations.

• Greydanus, S., Koul, A., Dodge, J., & Fern, A. (2018, July). Visualizing and understanding atari agents. In International Conference on Machine Learning (pp. 1792-1801). PMLR.

• Barrett, D. G., Morcos, A. S., & Macke, J. H. (2019). Analyzing biological and artificial neural networks: challenges with opportunities for synergy?. Current opinion in neurobiology, 55, 55-64.

• Morcos, A. S., Barrett, D. G., Rabinowitz, N. C., & Botvinick, M. (2018). On the importance of single directions for generalization. In International Conference on Learning Representations.

• Raghu, M., Gilmer, J., Yosinski, J., & Sohl-Dickstein, J. (2017). Svcca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. In Advances in Neural Information Processing Systems (pp. 6076-6085).

• Puri, N., Verma, S., Gupta, P., Kayastha, D., Deshmukh, S., Krishnamurthy, B., & Singh, S. (2019, September). Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attribution. In International Conference on Learning Representations.

• Sussillo, D., & Barak, O. (2013). Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks. Neural computation, 25(3), 626-649.

• Maheswaranathan, N., Williams, A., Golub, M., Ganguli, S., & Sussillo, D. (2019). Universality and individuality in neural dynamics across large populations of recurrent networks. In Advances in neural information processing systems (pp. 15629-15641).

未来智能实验室的主要工作包括：建立AI智能系统智商评测体系，开展世界人工智能智商评测；开展互联网（城市）云脑研究计划，构建互联网（城市）云脑技术和企业图谱，为提升企业，行业与城市的智能水平服务。

  如果您对实验室的研究感兴趣，欢迎加入未来智能实验室线上平台。扫描以下二维码或点击本文左下角“阅读原文”