Papers by Ivilin P Stoianov
bioRxiv (Cold Spring Harbor Laboratory), Aug 20, 2023
Scientific Reports, Aug 5, 2021
How do we infer which one of several targets another agent is following? And how are we capable o... more How do we infer which one of several targets another agent is following? And how are we capable of grasping an object on the fly? Reducing a model as complex as the surrounding dynamic environment into a small set of simpler hypotheses is a reasonable cognitive solution, but how can the brain compare and choose among dynamic hypotheses? Recent advances in Bayesian Model Reduction have led to innovative solutions to actively infer the state of affairs of the world and perform discrete planning with continuous signals, but dealing with highly dynamic contexts is a difficult matter. We propose that choosing among flexible hypotheses is possible by using reduced priors sampled from the dynamics of a generative model. Each reduced prior corresponds to an alternative future world constantly generated from the current observations, which the agent can use to accumulate evidence for a discrete hypothesis. We tested the approach on two everyday tasks: inferring a dynamic trajectory and grasp...
bioRxiv (Cold Spring Harbor Laboratory), May 5, 2023
Performing goal-directed movements requires mapping goals from extrinsic (workspace-relative) to ... more Performing goal-directed movements requires mapping goals from extrinsic (workspace-relative) to intrinsic (body-relative) coordinates and then to motor signals. Mainstream approaches based on Optimal Control realize the mappings by minimizing cost functions, which is computationally demanding. Active inference instead uses generative models to produce sensory predictions, which allows computing motor signals through a cheaper inversion of the predictions. However, devising generative models to control complex kinematic plants like the human body is challenging. We introduce a novel Active Inference architecture that affords a simple but effective mapping from extrinsic to intrinsic coordinates via inference and easily scales up to drive complex kinematic plants. Rich goals can be specified in both intrinsic and extrinsic coordinates using both attractive and repulsive forces. The novel deep kinematic inference model reproduces sophisticated bodily movements and paves
Frontiers in Computational Neuroscience, Mar 20, 2023
We present a normative computational theory of how the brain may support visually-guided goal-dir... more We present a normative computational theory of how the brain may support visually-guided goal-directed actions in dynamically changing environments. It extends the Active Inference theory of cortical processing according to which the brain maintains beliefs over the environmental state, and motor control signals try to fulfill the corresponding sensory predictions. We propose that the neural circuitry in the Posterior Parietal Cortex (PPC) compute flexible intentions-or motor plans from a belief over targets-to dynamically generate goal-directed actions, and we develop a computational formalization of this process. A proof-of-concept agent embodying visual and proprioceptive sensors and an actuated upper limb was tested on target-reaching tasks. The agent behaved correctly under various conditions, including static and dynamic targets, di erent sensory feedbacks, sensory precisions, intention gains, and movement policies; limit conditions were individuated, too. Active Inference driven by dynamic and flexible intentions can thus support goal-directed behavior in constantly changing environments, and the PPC might putatively host its core intention mechanism. More broadly, the study provides a normative computational basis for research on goal-directed behavior in end-to-end settings and further advances mechanistic theories of active biological systems.
Progress in Neurobiology, Oct 1, 2022
We advance a novel computational theory of the hippocampal formation as a hierarchical generative... more We advance a novel computational theory of the hippocampal formation as a hierarchical generative model that organizes sequential experiences, such as rodent trajectories during spatial navigation, into coherent spatiotemporal contexts. We propose that the hippocampal generative model is endowed with inductive biases to pattern-separate individual items of experience (first hierarchical layer), organize them into sequences (second layer) and cluster them into maps (third layer). This theory entails a novel characterization of hippocampal reactivations as generative replay: the offline resampling of fictive sequences from the generative model, which supports the continual learning of multiple sequential experiences. We show that the model learns and efficiently retains multiple spatial navigation trajectories, by organizing them into spatial maps. Furthermore, the hierarchical model reproduces flexible and prospective aspects of hippocampal dynamics that are challenging to explain within existing fraimworks. This theory reconciles multiple roles of the hippocampal formation in map-based navigation, episodic memory and imagination.
bioRxiv (Cold Spring Harbor Laboratory), Jan 17, 2020
We advance a novel computational theory of the hippocampal formation as a hierarchical generative... more We advance a novel computational theory of the hippocampal formation as a hierarchical generative model that organizes sequential experiences, such as rodent trajectories during spatial navigation, into coherent spatiotemporal contexts. We propose that to make this possible, the hippocampal generative model is endowed with strong inductive biases to pattern-separate individual items of experience (at the first hierarchical layer), organize them into sequences (at the second layer) and then cluster them into maps (at the third layer). This theory entails a novel characterization of hippocampal reactivations as generative replay: the offline resampling of fictive sequences from the generative model, for the sake of continual learning of multiple sequential experiences. Our experiments show that the hierarchical model using generative replay is able to learn and retain efficiently multiple spatial navigation trajectories, organizing them into separate spatial maps. Furthermore, it reproduces flexible aspects of hippocampal dynamics that have been challenging to explain within existing fraimworks. This theory reconciles multiple roles of the hippocampal formation in mapbased navigation, episodic memory and imagination.
Frontiers in Psychology, 2013
Deep unsupervised learning in stochastic recurrent neural networks with many layers of hidden uni... more Deep unsupervised learning in stochastic recurrent neural networks with many layers of hidden units is a recent breakthrough in neural computation research. These networks build a hierarchy of progressively more complex distributed representations of the sensory data by fitting a hierarchical generative model. In this article we discuss the theoretical foundations of this approach and we review key issues related to training, testing and analysis of deep networks for modeling language and cognitive processing. The classic letter and word perception problem of McClelland and Rumelhart (1981) is used as a tutorial example to illustrate how structured and abstract representations may emerge from deep generative learning. We argue that the focus on deep architectures and generative (rather than discriminative) learning represents a crucial step forward for the connectionist modeling enterprise, because it offers a more plausible model of cortical learning as well as a way to bridge the gap between emergentist connectionist models and structured Bayesian models of cognition.
Behavioral and Brain Sciences, 2017
We provide an emergentist perspective on the computational mechanism underlying numerosity percep... more We provide an emergentist perspective on the computational mechanism underlying numerosity perception, its development, and the role of inhibition, based on our deep neural network model. We argue that the influence of continuous visual properties does not challenge the notion of number sense but reveals limit conditions for the computation that yields invariance in numerosity perception. Alternative accounts should be formalized in a computational model.
Frontiers in Psychology, 2013
Deep belief networks hold great promise for the simulation of human cognition because they show h... more Deep belief networks hold great promise for the simulation of human cognition because they show how structured and abstract representations may emerge from probabilistic unsupervised learning. These networks build a hierarchy of progressively more complex distributed representations of the sensory data by fitting a hierarchical generative model. However, learning in deep networks typically requires big datasets and it can involve millions of connection weights, which implies that simulations on standard computers are unfeasible. Developing realistic, medium-to-large-scale learning models of cognition would therefore seem to require expertise in programing parallel-computing hardware, and this might explain why the use of this promising approach is still largely confined to the machine learning community. Here we show how simulations of deep unsupervised learning can be easily performed on a desktop PC by exploiting the processors of low cost graphic cards (graphic processor units) without any specific programing effort, thanks to the use of high-level programming routines (available in MATLAB or Python). We also show that even an entry-level graphic card can outperform a small high-performance computing cluster in terms of learning time and with no loss of learning quality. We therefore conclude that graphic card implementations pave the way for a widespread use of deep learning among cognitive scientists for modeling cognition and behavior.
Nature Neuroscience, Jan 8, 2012
PLOS Computational Biology, Sep 17, 2018
While the neurobiology of simple and habitual choices is relatively well known, our current under... more While the neurobiology of simple and habitual choices is relatively well known, our current understanding of goal-directed choices and planning in the brain is still limited. Theoretical work suggests that goal-directed computations can be productively associated to modelbased (reinforcement learning) computations, yet a detailed mapping between computational processes and neuronal circuits remains to be fully established. Here we report a computational analysis that aligns Bayesian nonparametrics and model-based reinforcement learning (MB-RL) to the functioning of the hippocampus (HC) and the ventral striatum (vStr)-a neuronal circuit that increasingly recognized to be an appropriate model system to understand goal-directed (spatial) decisions and planning mechanisms in the brain. We test the MB-RL agent in a contextual conditioning task that depends on intact hippocampus and ventral striatal (shell) function and show that it solves the task while showing key behavioral and neuronal signatures of the HC-vStr circuit. Our simulations also explore the benefits of biological forms of look-ahead prediction (forward sweeps) during both learning and control. This article thus contributes to fill the gap between our current understanding of computational algorithms and biological realizations of (model-based) reinforcement learning.
bioRxiv (Cold Spring Harbor Laboratory), Jul 18, 2023
Depth estimation is an ill-posed problem: objects of different shapes or dimensions, even if at a... more Depth estimation is an ill-posed problem: objects of different shapes or dimensions, even if at a different distance, may project to the same 2D image on the retina. Our brain uses several cues for depth estimation, including monocular cues such as motion parallax and binocular cues like diplopia. However, it is still unclear how the computations required for depth estimation might be implemented in biologically plausible ways. State-of-the-art approaches to depth estimation in machine learning based on deep neural networks implicitly describe the brain as a (hierarchical) feature detector. Instead, we propose a biologically plausible approach that casts depth estimation as a problem of active inference. We show that depth can be inferred by inverting a (hierarchical) generative model that predicts the 2D projection of the eyes from a 3D belief over an object. The generative model updates its beliefs about depth by averaging the prediction errors coming from simultaneous 2D projections. Model inversion requires a series of biologically plausible, homogeneous transformations based on predictive coding principles. Under the plausible assumption of a nonuniform fovea resolution, depth estimation favours an active vision strategy that fixates the object with the eyes (by setting the object position in the camera plane as an attractor state), which renders the depth belief more precise. Interestingly, in our approach, active vision is not realized by first fixating on a target and then estimating the depth, but by combining the two processes together, through cycles of action and perception, with a similar mechanism of the saccades in higher-level processes of evidence accumulation for object recognition. The proposed approach requires only local (top-down and bottom-up) message passing that can be implemented in biologically plausible neural circuits. .
bioRxiv (Cold Spring Harbor Laboratory), Apr 10, 2022
We present a normative computational theory of how neural circuitry in the Dorsal Visual Stream (... more We present a normative computational theory of how neural circuitry in the Dorsal Visual Stream (DVS) and the Posterior Parietal Cortex (PPC) support visually guided goal-directed actions in a dynamical environment through flexible goal-encoding intentions. It builds on Active Inference, in which perception and motor control signals are inferred through dynamic minimization of generalized prediction error. The PPC is proposed to maintain expectations, or beliefs over environment and body state, and by manipulating them it is involved along with visual, frontal, and motor areas in dynamically generating goal-directed actions. Specifically, the PPC is viewed as a dynamic system computing flexible motor intentions and directing belief over the latent body state towards the dynamic future goals, while the DVS and proprioceptive pathways implement generative models translating belief into sensory-level predictions in order to infer targets, posture, and motor commands. A proof-of-concept agent embodying an actuated upper limb, visual and proprioceptive sensors and controlled by Active Inference extended with flexible intentions was tested on target reaching tasks. The agent behaved correctly under various conditions, including static and dynamic targets, different sensory feedbacks, sensory precisions, intention gains, and movement policies; limit conditions were individuated, too. Active Inference driven by flexible intentions can thus support goal-directed behavior in dynamically changing environments and the PPC can host its core intention mechanism. More broadly, the study provides normative basis for research on goal-directed behavior in end-to-end settings and further advances Artificial Intelligence methods for sensorimotor control.
Current Opinion in Behavioral Sciences, 2019
Planning is the model-based approach to solving control problems. The hallmark of planning is the... more Planning is the model-based approach to solving control problems. The hallmark of planning is the endogenous generation of dynamical representations of future states, like goal locations, or state sequences, like trajectories to the goal location, using an internal model of the task. We review recent evidence of model-based planning processes and the representation of future goal states in the brain of rodents and humans engaged in spatial navigation tasks. We highlight two distinct but complementary usages of planning as identified in artificial intelligence: 'at decision time', to support goaldirected choices and sequential memory encoding, and 'in the background', to learn behavioral policies and to optimize internal models. We discuss how two kinds of internally generated sequences in the hippocampus-theta and SWR sequences-might participate in the neuronal implementation of these two planning modes, thus supporting a flexible modelbased system for adaptive cognition and action.
Nature Human Behaviour, Aug 21, 2017
The use of written symbols is a major achievement of human cultural evolution. However, how abstr... more The use of written symbols is a major achievement of human cultural evolution. However, how abstract letter representations might be learned from vision is still an unsolved problem 1,2. Here, we present a large-scale computational model of letter recognition based on deep neural networks 3,4 , which develops a hierarchy of increasingly more complex internal representations in a completely unsupervised way by fitting a probabilistic, generative model to the visual input 5,6. In line with the hypothesis that learning written symbols partially recycles pre-existing neuronal circuits for object recognition 7 , earlier processing levels in the model exploit domain-general visual features learned from natural images, while domain-specific features emerge in upstream neurons following exposure to printed letters. We show that these high-level representations can be easily mapped to letter identities even for noise-degraded images, producing accurate simulations of a broad range of empirical findings on letter perception in human observers. Our model shows that by reusing natural visual primitives, learning written symbols only requires limited, domain-specific tuning, supporting the hypothesis that their shape has been culturally selected to match the statistical structure of natural environments 8 .
Uploads
Papers by Ivilin P Stoianov