From beast machines to dreamachines
University of Sussex
Consciousness remains one of the central mysteries in science and philosophy. In this talk, I will illustrate how the framework of predictive processing can help bridge from mechanism to phenomenology in the science of consciousness – addressing not the ‘hard problem’, but the ‘real problem’. I will advance the view that predictive processing, precisely because it is not itself a theory of consciousness, offers a powerful approach for addressing the real problem. I will illustrate this view first by showing how conscious experiences of the world around us can be understood in terms of perceptual predictions, developing a version of computational (neuro)phenomenology. Then, turning the lens inwards, I’ll explore how the experience of being an embodied self can be understood in terms of control-oriented predictive (allostatic) regulation of the interior of the body. This implies a deep connection between mind and life, providing a new way to understand the subjective nature of consciousness as emerging from systems that care intrinsically about their own existence. Contrary to the old doctrine of Descartes, we are conscious because we are beast machines. I’ll finish by describing a recent art-science collaboration – the dreamachine – which involves mass stroboscopically-induced visual hallucinations and a large-scale online survey of ‘perceptual diversity’ – The Perception Census.
Novel approaches for understanding the neurocomputational basis of interoception and emotion-cognition interactions
Laureate Institute for Brain Research & University of Tulsa
How the brain detects and interprets signals from within the body – a process known as interoception – may play an important role in generating subjective feelings. While interoception has received growing attention from researchers in recent years, the precise computational mechanisms through which the brain processes interoceptive signals, and how these signals influence emotion and cognition, remain unclear. In this talk, I will present recent computational modelling studies we have performed to better characterize these mechanisms across three interoceptive channels: cardiac, gastrointestinal, and respiratory. First, I will describe results of modeling heartbeat perception as Bayesian inference, which suggest that subjective estimates of the reliability (precision) of cardiac signals may be less flexible in multiple psychiatric patient samples (depression, anxiety, substance use, and eating disorders) relative to healthy participants. Second, I will describe results of modeling gastrointestinal perception as Bayesian inference during EEG recording, which show that individual differences in subjective signal precision and prior expectations have excitatory and inhibitory influences on neural responses, as hypothesized within predictive processing models. Finally, I will present results of a study examining how respiratory interoception, and associated anxiety, influence neurocomputational mechanisms of planning and decision-making on reward-learning tasks, and how this differs between healthy and transdiagnostic psychiatric samples (anxiety and substance use disorders). Overall, these results provide evidence for neurocomputational mechanisms of brain-body interactions across multiple interoceptive channels and how they contribute to subjective feelings and cognition. They may also highlight novel mechanistic treatment targets that could be evaluated in future clinical studies.
Interoceptive feelings of emotion, memory and pain in clinical populations
A growing body of recent studies suggests that sensations or feelings inside the body influence various domains of cognition. In this talk, I will pick up our clinical and experimental studies related to this topic and present the recent findings to lead possible discussions. Previous neuroimaging studies revealed that the insular cortex functions as one of the centres for emotion recognition, but most studies have shown correlational findings. I will first present the results of the awake surgery studies for patients with insular glioma, providing causal evidence that the insular cortex has critical roles for emotion recognition. Next, I will present our psychophysiological evidence suggesting that interoceptive signals could trigger timing-based prospective remembering as a form of prospection. Finally, I will focus on the migraine and its underlying anticipatory component for causing pain. Patients with migraine or several related clinical disorders are highly susceptible to the changes in weather conditions. They occasionally complain of pain and physical discomfort even in conjunction with a typhoon at a far distance. However, such sensing mechanisms remain unresolved and shrouded in mystery. I will show the findings of recent studies suggesting the possible means of sensing the changes in atmospheric pressure to explain the exacerbating pain in migraine through vestibular interoception and psychogenic factors.
Developmental origins of interoceptive processing
A recent theory proposes that the constitution of the self depends upon the mentalization of the body, particularly its homeostatic needs, which are fundamentally shaped by embodied interactions with other people in early infancy and beyond. The ability to perceive one’s own internal sensations, called interoception, supports homeostatic control and allostatic adaptation. According to the theory, interoceptive processing has fundamentally social origins. However, we have yet to understand how social information impact interoceptive processing. Moreover, we know very little about the early development of interoceptive ability – when and how infants exhibit interoceptive sensitivity. In this talk, I will briefly present a series of studies that we have recently performed to investigate how social context modulates interoceptive processing. Then I will describe our recent study that focused on early development of interoceptive processing. Specifically, I will present a developmental change in the sensitivity to cardio-audiovisual synchrony in the first year of life and discuss the possible mechanisms underlying the behavioral changes. Overall, the results provide evidence for the developmental origins of interceptive processing and shed light on the nature of the self.
Decoding of neural representations involved in subjective bodily awareness
Functional magnetic resonance imaging (fMRI) is one of the most common tools of non-invasive human brain imaging, and multivoxel pattern analysis (MVPA) of fMRI has recently become popular. Beyond aconventional mass-univariate analysis, MVPA can be used to examine the spatial patterns of brain activity and have made it possible to decode information from brain activity to reveal the specific neural representations. For example, the similarity of neural representations can be tested by measuring distances between distinct brain activity patterns (representational similarity) or the generalization ability of the decoders between different conditions (cross-classification). In this talk, we will introduce our recent studies that have investigated the neural representations related to subjective sensorimotor or bodily awareness. These include supramodel sense of body ownership in the parieto-premotor cortices and kinesthetic motor imagery of finger movements in the primary motor cortex, as well as the cardiac and gastric interoceptive awareness in the insular cortex.
Prediction and back-projection in self-body representation
The neural representation of the self-body involves multisensory, including vision, proprioception, and tactile sensation, as well as sensorimotor integration with respect to the body. Interestingly, under certain conditions, the self-body representation is extended to an external object other than the self-body (rubber hand, VR avatar, other person, etc.). In this talk, I will introduce the phenomenon of back-projection, in which changes and characteristics of the embodied object (i.e., the rubber hand) are projected back onto the self. For example, after experiencing a sense of ownership of a fake rubber hand through visuotactile stimulation, the subject’s hand movement is triggered by the unexpected movement of the fake hand. Recently, it has also been reported that the characteristics of a VR avatar are reflected onto the self when the full-body illusion occurs (also known as the Proteus effect). We will argue that these are functions of the "self-body" representation established between the self and a fake body (part), which leads to predictive self-body processing, and when a phenomenon contrary to the prediction occurs, the self regulates itself to reduce the prediction error between the self-body representation and the fake body state.
Why does cognition feel effortful?
Okinawa Institute of Science and Technology Graduate University
There is a feeling of effort associated with cognition. For example, when we strain to see the details of a scientific figure, when we focus our attention while listening to a difficult lecture, or when we think hard about making a delicate decision, our cognitive activity involves effort that is experienced as such: a feeling of effort. Phenomenologically, this feeling of cognitive effort is sufficiently like the feeling of physical effort that, arguably, we can treat them as belonging to the same kind of phenomenon. This unification of cognitive and physical effort may be surprising, especially given that there is no overt involvement of bodily exertion in cognitive effort. On the other hand, it points to the possibility of a deeper unification of efforts in the context of a theory of agency, especially if that theory can make it intelligible how the efficacy of mental actions can have its own specific energy cost.
Physiological mechanisms to process interoceptive information from peripheral organs in rodents
The insular cortex serves as a hub cortical region that is bidirectionally connecting to an extensive cortical and subcortical brain areas and has been shown to modulate emotional behavior both in humans and rodent models, including fear and facial expressions, anxiety, and depression. In addition, accumulating evidence demonstrates that the insular cortex plays a crucial role in interoception. However, neurophysiological mechanisms and insights supporting this hypothesis remain to be clarified. To address this issue, we performed simultaneous recordings of multiunit spike patterns and local field potential signals from the insular cortex, an electrocardiogram signal, and a peripheral blood glucose concentration from a freely moving rodent animal. Recordings were daily obtained for seven hours. At single-cell levels, a subset of insular cortical neurons increased or decreased their spike rates in response to changes in peripheral organ states. These results highlight insular cortical neurons as a detector of temporal changes in interoceptive signals, which may induce various emotional valence and, in turn, regulate these organ signals through efferent controls in the brain-body axis.
Towards Cognitive Robots: Designing Considerate Agents for Effective Human Interaction
Italian Institute of Technology
Typically, machines are created to surpass humans: their sensors strive for utmost accuracy, their bodies are built to handle tasks effortlessly, and their minds aim to objectively record everything. However, this pursuit of perfection has not resulted in the development of agents capable of effectively interacting with us. At best, we have highly advanced tools that skilled users can learn to control, but they lack the ability to truly collaborate with us. Instead, we propose the design of cognitive robots that prioritize consideration for humans. While these robots may not resemble humans in their appearance, they should possess the capability to understand human perception, action, and cognition, including the biases and distortions inherent in human nature. Furthermore, they should possess the capacity to learn and adapt, and they should be aware of the crucial role of the affective dimension in human cognition. By developing considerate robots we are not only creating advanced technological tools, but we can gain invaluable insights into the mechanisms that drive our cognitive processes. In this presentation, I will offer concrete examples of this approach from our research conducted with the humanoid robot iCub.
From Explicit Mirror Neuron Modeling to Emergent Mechanisms
Ozyegin University & Osaka University
Mirror Neurons, originally discovered in the premotor cortex of macaque monkeys, appear to represent actions or action related information in multiple modalities irrespective of the agent of the action.Since their discovery, besides the vast number of neuroscientific research studies, several computational models have been proposed. The models were constructed such that the desired functionality can be obtained in simulation settings. In particular, learning based on self-observation during action execution has been a fruitful approach for training of networks to generate mirror-like behavior. However, to work these systems included a preprocessing step, i.e. 'feature engineering', to embed self-vs-other equivalence in the networks. With the recent developments in deep learning, it is now possible to ask whether this equivalence can naturally emerge from self-observation without feature engineering. Our recent work shows some signs of a positive answer to this question, where the motor parameters and the vision of the hand in action is employed in an end-to-end learning fashion with a novel deep neural network architecture. After training, the network is found to exhibit responses that can be associated with egocentric action recall (mirror-like response) as well as object based action recall depending on the amount and fidelity of the input it is allowed to see. This emergent dual behavior suggests that in the brain such circuits might have evolved, which are then, by additional circuitry, modulated allowing voluntary mode selection for imitation and action understanding.
Learning predictive models for intention estimation and high-level reasoning in robots
Predicting the consequences of one’s actions is an essential requirement for cognition and decision-making in biological and artificial systems. Neurophysiological data suggests that the human brain benefits from internal forward models that continuously predict the outcomes of the generated motor commands for planning and control. Inferring others’ actions and executing one’s actions are closely intertwined processes. The reuse of cortical circuits for both movement generation and action estimation seems to be a fundamental principle in sensorimotor organization in primates. For complex object manipulation in an environment, an organism extracts object properties relevant for manipulation, detects object affordances, and makes motor plans based on them to fulfill a desired change in the environment. In this talk, I will talk about computational models that enable the generation of high-level plans to achieve different goals based on learned predictive models and the emergence of intention estimation and altruistic behavior using predictive models and object affordances.
Talks by CREST Members
Enhancing Wellbeing through the Promotion of a Stable Sense of Self: A Qualitative Study on the Impact of the Tojisha-Kenkyu Program
The University of Tokyo
This research investigates the relationship between stability in various dimensions of the self and cognitive feelings, focusing on material, spiritual, and social aspects, as well as the sense of agency and autobiographical memory. Previous studies have highlighted the positive correlation between a stable sense of self and improved wellbeing. Moreover, the integration of interception into one's self-perception has been associated with a heightened sense of stability.
The present study aims to explore the effects of the Tojisha-Kenkyu program, a specialized intervention targeting the stability of the sense of self in relation to cognitive feelings. Employing a qualitative research design, this investigation delves into the subjective experiences and outcomes of individuals participating in the program. Through interviews, we seek to gain a comprehensive understanding of the program's impact on participants' cognitive feelings and overall wellbeing. Findings from this research are expected to contribute to the growing body of knowledge on the significance of cultivating a stable sense of self. The outcomes will shed light on the potential benefits of the Tojisha-Kenkyu program and offer valuable insights for researchers, practitioners, and individuals interested in enhancing their own wellbeing through the cultivation of a stable sense of self.
Using extended reality to study the experience of presence
This talk explores the cognitive phenomenon of the Feeling of Presence (FoP) – the sense of being here and now in either the real or virtual world. Utilizing Extended Reality (XR) technology, our immersive experiences can significantly augment FoP. On the other hand, certain clinical conditions are known to dampen this feeling such as Depersonalisation/Derealisation disorders. While XR research has illuminated many elements influencing FoP, much remains undiscovered about its intricate phenomenology and the neurocognitive processes that underpin it. My talk will dive into our comprehensive study series that explore the cognitive mechanisms behind FoP, employing XR technologies as our research instrument. Initially, I will introduce a set of experiments involving XR that scrutinize the impact of sensorimotor contingencies and affordances on a particular form of presence, specifically relating to the perception of an object’s existence in the world. Moving beyond perceptual presence, I will further delve into the concept of the 'conviction about reality.' I will present our unique Substitutional Reality setup designed to probe this aspect of FoP, thereby providing valuable insights into how we perceive and interact with our world, whether real or virtually constructed.
Altered hierarchical predictive processing: Exploring psychiatric and neurodevelopmental disorders through a neurorobotics approach
National Center of Neurology and Psychiatry
Adaptive behavior and cognition in humans are believed to be driven by hierarchical predictive processes within generative models, a concept commonly referred to as predictive processing. This perspective has been increasingly utilized to understand the computational principles of the brain. However, impairments in the development or maintenance of signaling within neural systems have been postulated to contribute to the manifestation of symptoms in psychiatric and developmental disorders.
In this presentation, I will introduce a series of experiments conducted using a 'neurorobotics approach,' in which behavioral control mechanisms with hierarchical predictive processes were implemented through the physical actions of a humanoid robot driven by a hierarchical recurrent neural network. The results of these experiments revealed that altered hierarchical predictive processes, such as functional disconnections between levels of the hierarchical network and altered intrinsic neuronal excitabilities, play a crucial role in the emergence of symptoms observed in psychiatric and developmental disorders. These symptoms include aberrant neural activities, atypical behavior, and distorted estimation of sensory precision. These findings suggest that the neurorobotics approach paves the way for new avenues of investigation into the mechanisms underlying psychiatric and developmental disorders.
[P-1] Co-regulation of the Day-Long Autonomic Nervous System in Children and Caregivers - Analysis Using (Cross) Recurrence Plot
Jiarui Li (1), Michiko Matsunaga (2), Masako Myowa (3), Yukie Nagai (1)
(1) The University of Tokyo, (2) Osaka University, (3) Kyoto University
Recent neurodevelopmental studies have suggested that interpersonal neural synchrony (INS) between child and their caregiver in daily interaction could be a sensitive biomarker for parenting problems such as childcare stress. One critical step for solving this problem is investigating the dynamics of child-caregiver co-regulation of their interoceptive states, e.g., the Autonomic Nervous System (ANS) activities. The ANS has a direct role in the physical response to stress. The wealth of information rich in ANS data, especially long-timescale data, is hard to quantify adequately by simple statistical measurements. Therefore, this study introduces the (Cross) Recurrence Plot ((C)RP) to visualize the circadian rhythm of the day-long ANS data and mutual physiological regulation between child and caregiver. Furthermore, we utilize the Self-Organizing Map (SOM) model to verify whether the quantified ANS complexity can predict the mothers’ parenting stress. Our results found that the mental stress of a subject, e.g., the mother, can be predicted by the interactive partner, e.g., her child, using RPs. Moreover, physiological synchrony between the two individuals quantified by CRPs is also important information to predict subjects’ mental stress.
[P-2] Estimation of mental state via physiological measurement
Sarah Cosentino, Yukie Nagai
University of Tokyo
Neurodivergent (ND) people, e.g. people on the Autism Spectrum, have difficulties in adapting and conforming to neurotypical (NT) behavioral expectations, for example, to established NT patterns of communication. To build a more inclusive society, it is thus important to close the gap between ND and NT behavioral differences. An effective solution would be using technology to build a "behavioral translator" between NDs and NTs. On this line, a psychophysiological approach seems very promising. Psychophysiology studies multiple aspects of behavior, emotions being the most common example. It has long been recognized that changes in mental states are associated with physiological responses. Measuring an individual's physiological responses would then allow us to better understand and predict emotional responses. Multimodality, the integration and analysis of several different measurements, will allow to build a more complete picture of the overall individual behavioral patterns.
[P-3] Preliminary experiment: the effect of subjective choices on task continuity for a simple task
Ayumi Ikeda (1), Takuya Kamimura (2), Akira Karasudani (2), Keisuke Suzuki (1), Hiroyuki Iizuka (1)
(1) Hokkaido University, (2) Fujitsu Limited
The connection between one's behavior and the consequences of one's behavior is crucial in daily life. However, there are psychological phenomena such as choice blindness in which the difference between a subjective choice and a consequence of the choice is undetectable. In this study, we examined whether the participants' perception that they had subjectively chosen the difficulty level of the task influenced their efforts to perform the task, without actually changing the difficulty level of the task. The task consisted of 30 trials x 30 blocks of a simple two-digit addition task. The experiment consisted of two conditions: the choice condition allowing participants to input the task's difficulty level, and the no-choice condition giving no choice to the participant. The results of a comparison between the conditions for the within-participant factor indicated that the number of breaks that could be taken voluntarily during the intervals of the block was significantly less in the choice condition. The subjective choice may influence the continuity of the task, even if it is not followed by the consequences of the choice.
[P-4] Reflective Artificial Agents for Sustainability
Aishwaryaprajna, Peter R. Lewis
Ontario Tech University
Recent literature has argued that the rationality of AI agents regarding utility functions or goals is not enough to bring cooperation in socially situated multi-agent systems in the absence of social norms. Intentional cooperation and coordination in social systems may be achieved by social self-awareness with capabilities of perception and reasoning about others in the system and the environment. Our work builds on this notion and targets the building of reflective cognitive agents that are capable of reasoning about their immediate actions and the corresponding long-term impact on their environment. Cooperation has been widely studied in multi-agent foraging tasks. However, the consequences of agent-environment interactions in the longer term and the achievement of sustainability have been largely unexplored in this context. We illustrate the impact of a reflective governance loop in the architecture of foraging agents that supports the long-term sustainability of the environment and hence, also of themselves, due to the consideration of the higher level of goals at each step of learning and decision-making.
[P-5] Development of the Japanese version of the Phenomenological Control Scale
Shu Imaizumi (1), Keisuke Suzuki (2)
(1) Ochanomizu University, (2) Hokkaido University
People possess a capacity for phenomenological control, enabling them to align their perceptual experiences with their intentions and goals. However, this capacity varies among individuals. To measure this trait, the Phenomenological Control Scale (PCS) has been developed. In this study, we developed and validated the Japanese version of the PCS (PCS-J) based on a preregistered online survey (n = 261). The PCS-J demonstrated sufficient internal consistency and test-retest reliability (retest n = 152). Given the association between hypnotizable suggestibility and positive schizotypy, the PCS-J's convergent validity was supported by a weak positive correlation with positive schizotypy. Discriminant validity of the PCS-J was demonstrated by the absence of correlation with negative schizotypy. This scale could be useful for future research on perception, cognitive feelings, and their individual differences.
[P-6] Learning 3D object-centric representation through prediction: examining insights from infants
Tushar Arora, John M Day, Jirui Liu, Li Erran Li, Ming Bo Cai
University of Tokyo
As part of human core knowledge, the representation of objects is the building block of mental representation that supports high-level concepts and symbolic reasoning. Infants develop the notion of objects situated in 3D environments without supervision. Towards understanding the minimal set of assumptions needed to learn object perception, we investigate a predictive learning approach to achieve the abilities of inferring objects' locations in 3D, segmenting objects from images and perceiving depth with similar constraints faced by infants, namely, no direct supervision or pre-training from computer vision datasets or utilizing supplementary signals not immediately available to the brain. We treat objects as latent causes of scenes, inferred by the brain. Such latent causes allow efficient prediction of the coherent motion of all parts of an object both in 3D and on retina. We conjecture that prediction error of future sensory input may be a major teaching signal for the brain to learn to infer objects. Accordingly, we develop a framework to extract object-centric representations from single 2D images by learning to predict future scenes through combination of two approaches: 1) optical flow-based warping of current visual input based on inferred object segmentation, motion and depth, and 2) implicitly 'imagining' the next scene based on statistical regularity. Importantly, the model only requires the presence of object motion and information of self-motion during training (which are natural for the brain). The work demonstrates a new approach to learning symbolic representation grounded in sensation.
[P-7] Tuning the Kuramoto Model to the output of a Neural Mass Model
University of Sussex
In this study, we aimed to synchronize two Whole Brain Network Models of coupled oscillators with time delays into a unified operational state, expanding upon the fundamental Kuramoto model formula. We employed genetic algorithms and Continuous-Time Recurrent Neural Networks (CTRNN) to optimize parameters, achieving synchronization with a Neural Mass model, specifically the Jansen-Rit model. The relevance of this research stems from the distinct insights provided by the Kuramoto and Neural Mass models: while the former offers a macroscopic perspective on overall system coherence, the latter simulates cortical dynamics. By attaining simultaneous states in both models, we were able to observe a broader spectrum of states in human brain network communication. Consequently, our findings lend considerable support to the theory of Communication Through Coherence (CTC), reinforcing its importance as a critical mechanism for brain information sharing and perceptual binding.
[P-8] The mathematics of systems with goals
Agents are generally understood as systems autonomously acting with a purpose, to achieve certain goals within an environment. Over the years, agents and goals have been studied in several different fields, including control theory and reinforcement learning, biology and origins of life, cognitive science and philosophy, artificial intelligence and artificial life. Their role is central in various disciplines, but their definition is taken for granted and in most cases unique to a certain research field. In this work we formalise some features we consider necessary for a unified understanding of agency. We do so using the language of category theory, which in recent years has been exploited in a number of applications, including quantum physics, game theory and machine learning. This work can be seen within the framework recently named “categorical cybernetics”, bridging the gap between cybernetic treatments of systems theory and purpose, and modern frameworks developed in (abstract) mathematics.
[P-9] Understanding central tendency effects in space and time perception in autism using deep neural networks
Jie Mei (1), Shiyun (Iris) Dong (2), Yukie Nagai (1)
(1) The University of Tokyo, (2) Newcastle University
Research has demonstrated that in perception, especially when facing uncertainty, individuals tend to rely on prior knowledge. This cognitive process exhibits flexibility, as the influence of prior representations becomes more prominent when sensory estimates are less precise, necessitating greater integration of past knowledge to alleviate the effects of noise.
This phenomenon, commonly referred to as central tendency, has been observed in various perceptual domains. In the meantime, studies have revealed that people with autism spectrum disorders (ASD) exhibit altered central tendency effects, often demonstrating a lower degree of flexibility. However, mixed results have been reported in tasks such as time processing.
In this study, we examine reliance on prior sensory information over developmental stages by employing a predictive-coding inspired recurrent neural network. Specifically, we modeled patterns of development in space and time perception tasks, including: (1) a length reproduction task, measuring the extent to which the perception of length is influenced by the previous presentations, (2) a visual stimulus-based interval reproduction task, measuring central tendency effects in time processing. We have reproduced results in published studies and confirmed that people with ASD may use past information to compensate for sensory inaccuracies and provided an estimation of reliance on prior in both ASD and non-ASD individuals across developmental stages.
[P-10] Developmental Simulation of Atypical Flexibility and Hierarchy using Predictive-coding-inspired Neural Network Model
Soda Takafumi (1,2), Ahmadi Ahmadreza (3), Tani Jun (4), Honda Manabu (1), Hanakawa Takashi (5), Yamashita Yuichi (1)
(1) National Center of Neurology and Psychiatry, (2) Tokyo Medical and Dental University, (3) Geobotica, (4) Okinawa Institute of Science and Technology Graduate University, (5) Kyoto University
Neurodevelopmental disorders are complex due to the interaction of multiple factors such as neural systems, behavior, environment, and learning. Computational approaches have improved our understanding of these disorders, but they lack a developmental learning perspective. In this study, we conducted simulation experiments using a predictive-coding-inspired variational Bayesian recurrent neural network model to examine the acquisition of hierarchical representations and its failures. Specifically, we investigated whether manipulating neural stochasticity and noise levels in external environments during the learning process can lead to altered acquisition of hierarchical Bayesian representation and reduced cognitive flexibility. The results showed that networks with normal neural stochasticity acquired hierarchical representations reflecting the underlying probabilistic structures in the environment, including higher-order representation, and exhibited good flexibility. When neural stochasticity was high during learning, top-down generation using higher-order representation became atypical, though flexibility remained similar to normal settings. Conversely, when neural stochasticity was low, networks demonstrated reduced flexibility and altered hierarchical representation. Importantly, increasing the level of noise in external stimuli ameliorated this altered higher-order representation and flexibility. These findings demonstrate that our proposed method facilitates modeling of neurodevelopmental disorders by integrating multiple factors.
[P-11] Cultural transmission promotes the emergence of hierarchical structure
The University of Tokyo
Human language is characterized by complex structural features, such as the hierarchical combination of words to form sentences. Previous studies have suggested that cultural transmission plays a key role in the emergence of structural features in human languages. While Cornish et al. (2013) demonstrated the emergence of hierarchical structures in non-linguistic systems, we argue that their laboratory study may have overestimated the role of cultural transmission due to a lack of appropriate controls and analyses. To directly test the effect of cultural transmission, we replicated Cornish et al.'s (2013) transmission experiment (transmission condition) and an experiment with no cultural transmission as a control (individual condition). We found that sequences became more structured as the number of generations increased; however, those produced in the transmission condition were more structured than those in the individual condition. These results suggest that cultural transmission plays an important role in the emergence of hierarchical structures that cannot be explained by increased learnability alone. The emergence of complex structural properties in human culture, such as language, technology, and music, may have resulted from information transmission processes between different individuals.
[P-12] The change of interoceptive processes accompanying the awareness of mind-wandering
Kazushi Shinagawa, Yuri Terasawa, Yuto Tanaka, Satoshi Umeda
Ｗe spend much time on thoughts unrelated to the current situation, such as imagining events from the future or past. Such phenomena are called mind-wandering. When our minds drift away from here and now, we can always return to the present. We examined how our minds return from mind-wandering through the information process before awareness. In our experiments, the participants were asked to report mind-wandering immediately after realizing their minds had drifted away from the simple reaction task, which consisted of sound or breath focus conditions with recording EEG, ECG, and respiratory. We examined noticing one's state by going back in time from the report and focused on P3, time-frequency, and heartbeat-evoked potentials (HEP), an EEG response time-locked to the R-peak of the Heartbeat. These indices reflect the allocation of attention to external stimuli, processing of interoceptive information, and whole-brain state, respectively. Through these indices, we examined information processing during concentration on sound and breathing, MW, and awareness.
[P-13] Developmental change of muscular sensorimotor information flow in spontaneous infantile movements
The University of Tokyo
Experimental animal studies revealed that the distributions of spontaneous neuronal activity patterns reflect and contribute to neuronal maturation. Based on a Bayesian approach, this process is interpreted as the construction of an internal probabilistic model, thereby suggesting statistical regularization through environmental interactions shapes the spontaneous activity reflecting prior expectations. Here, we gained insights into the structuration process of the sensorimotor interaction during spontaneous movements in human neonates and 3-mo-old infants. By quantifying muscular sensorimotor information flow, we identified developmental change suggesting the sensorimotor explorations, which may be a consequence of the process with an optimizing prediction error and conditional expectations. Moreover, our results indicated the contribution of early spontaneous movements to the self-organized spatiotemporal structuring of sensorimotor interactions, providing conceptual insight that links early whole-body movements to spontaneous neuronal activity maturing in an autonomous fashion. Therefore, the sensorimotor interaction is to the structure of embodiment what functional connectivity is to the brain’s anatomical connectivity.
[P-14] Thought transitions modulated by interoception
Mai Sakuragi, Kazushi Shinagawa, Yuri Terasawa, Satoshi Umeda
Mind wandering (MW) is a phenomenon of thought shifting to matters unrelated to the task at hand. The relationship between the thought shifting associated with MW and bodily response has not been directly examined. This study examined the influence of cardiovascular reactivity and interoception—the sensing of an internal bodily state—on the shifting of thought states. Participants (N = 100, 70 women) completed two tasks: the heartbeat counting task (HCT) and the vigilance task (VT). We assessed participants’ interoceptive accuracy by their performance on the HCT. In the VT, participants pressed a key when the target stimuli appeared. During this task, we asked participants to report the content of their thoughts. In half of the VT, we presented subliminal vibration stimuli, to induce alteration in the heart rate (i.e., vibration block). Results showed that participants with higher interoceptive accuracy reported higher contemplation of self-referential thoughts (past episodes and future plans about themselves ) during the vibration block than those with lower interoceptive accuracy. The results suggest that individuals with higher interoceptive accuracy are more susceptible to subliminal physical reactions, resulting in a divergence of attention from the task and a stronger draw toward self-referential thought.
[P-15] Multiple Timescale Recurrent State-Space Model for Learning Long-Horizon Tasks
Kentaro Fujii, Shingo Murata
Cognitive agents, including humans and autonomous robots, face difficulties in dealing with temporal uncertainty and temporal dependency when performing long-horizon tasks. While world models have shown promise in various benchmark tasks, they often struggle to manage long contexts due to the limited representation capabilities of their latent dynamics model. To overcome this limitation, we propose a novel hierarchical latent dynamics model that incorporates multiple-timescale dynamics. Our model, named the “multiple timescale recurrent state-space model” (MTRSSM), consists of a higher level with slow dynamics and a lower level with fast dynamics, each utilizing deterministic and stochastic latent states. Through quantitative and qualitative analysis, we demonstrate that our proposed MTRSSM-based world model outperforms other baselines in generating video predictions through latent imagination for long-horizon robotic object-manipulation tasks. Importantly, we highlight the crucial role of the higher level in effectively managing temporal uncertainty and temporal dependency in such tasks. These findings indicate that the proposed MTRSSM facilitates learning and planning abilities for long-horizon tasks by achieving a better understanding of the environment and generating more accurate predictions.
[P-16] Being mimicked for choices increases perception of warmth, not competence
Paula Wicher, Eva Krumhuber, Antonia Hamilton
University College London
It is widely believed that being mimicked makes us like the person more (Chartrand and Bargh, 1999). Can we get different benefits depending on what we copy –physical movements or something more abstract like preferences? Here, we compared the social consequences of copying choices and copying hand movements in the context of making art choices. Participants completed an in-lab mimicry task with 3 different ‘confederates’ who either mimicked their hand movements, art choices or did 50/50 of both. They believed the confederates were real people on a Zoom call, when in fact they were pre-recorded videos. Then they completed measures of perceived warmth and competence to assess first impressions. The results showed that participants liked ‘confederates’ who mimicked their choices more than the ones who mimicked their hand movements. Moreover, copying preferences increased social perceptions of warmth and copying hand movements increased competence scores. These results suggest copying choices seem to be a stronger driving factor in likability judgments than copying motor movements.
[P-17] Interoceptive behavior switching facilitates deep homeostatic reinforcement learning
Naoto Yoshida (1), Hoshinori Kanazawa (1,2), Yasuo Kuniyoshi (1,2)
(1) UTokyo, (2) Next Generation Artificial Intelligence Research Center
Stabilizing the internal state of the body is important for the survival of biological agents. Homeostatic Reinforcement Learning (Homeostatic RL) provides the normative framework for adaptive behavior, including reward definitions. Deep homeostatic reinforcement learning is homeostatic RL extended by deep reinforcement learning and allows us to handle high-dimensional continuous motor control and observations. In our previous work, the agent demonstrated that its behaviors were clearly separated by the internal state of the body in the multi-resource environment. From this observation, we suggest that the specific neural network architecture would facilitate learning in the homeostatic RL domain. In this study, we propose two neural network architectures: the Interoceptive Behavior Switching (IBS) and the Interoceptive Mixture of Experts (IMoE). IBS is the query-key-value attention architecture where the query is encoded from the interoception of the agent. IMoE uses the mixture of experts architecture and the mixture weights of the policy modules are encoded from the interoception. Both architectures include modular policies and a switching mechanism. In the computational experiment with four homeostatic RL benchmark environments, we confirmed the advantage of IBS and IMoE over the fully-connected architecture in these tasks.
[P-18] Surviving Uncertainty: Exploring Open-ended Allostatic Planning in a Homeostatic Variational Recurrent Neural Network
Hayato Idei (1,2), Jun Tani (3), Tetsuya Ogata (2), Yuichi Yamashita (1)
(1) National Center of Neurology and Psychiatry, (2) Waseda University, (3) Okinawa Institute of Science and Technology
To remain alive, cognitive agents need to resist a natural tendency to disorder (i.e., the law of entropy increase in physics) by minimizing the entropy (uncertainty) of their sensory states. Previous computational theories, grounded on the free-energy principle (FEP), posited homeostatic processing through goal-directed active inference by assuming an explicit set-point (goal) for the interoceptive state. However, these theories have yet to explore predictive modulation of sensorimotor goal states in changing environments, a concept known as allostasis. We propose a multimodal homeostatic variational recurrent neural network model, grounded on FEP. In our model, reduction of future sensory uncertainty serves as an allostatic 'meta-goal.' This 'meta-goal' modulates future prior beliefs or 'goals' about hierarchical sensorimotor sequences, including the interoceptive state. We tested this allostatic planning model in a simulation experiment of survival task, in which food’s position and nutritional value change dynamically and the neural-agent needs to appropriately maintain its energy state by simultaneously performing perception, planning, and action via free-energy minimization from past to future. We show that the proposed model significantly outperformed a set-point-driven model in terms of the survival period. Our results suggest that the uncertainty-driven allostatic planning could explain flexible, robust maintenance of homeostasis in dynamic environments.
[P-19] Color Enhances the Categorization of Scene in Peripheral Vision
Lana Okubo (1,2), Kazuhiko Yokosawa (3)
(1) The University of Tokyo, (2) JSPS, (3) Tsukuba Gakuin University
It is known that observers more correctly categorize the chromatic natural scenes than the achromatic natural scenes. However, when the scene’s color is inverted, this effect disappears. These facts suggest that the visual system uses the typical color of scenes recalled from memory for scene recognition. We examined in which visual field this effect is observed. We manipulated the presentation area of scenes: a central circular area with a radius of 5°(central vision condition)/an external area of a circular area with a radius of 5° (peripheral vision 5° condition)/ peripheral vision 10° condition/ peripheral vision 15° condition. We also manipulated the scene's color: normal color/ inverted color/ monochrome. There were two types of scenes: natural/ artificial. In the central vision and peripheral vision 5° conditions, natural scenes with normal colors produced higher categorization performance than achromatic natural scenes. In other peripheral vision conditions, normal color helped categorize natural and artificial scenes. Our results showed that the chromatic cue helps scene recognition even in the peripheral visual field, which is thought to have poor color sensitivity. In larger eccentricities, typical color contributes to categorical judgments of scenes, regardless of whether the scene is natural or artificial.
[P-20] What does the experience of “Tojisha-Kenkyu” bring to participants: an interview study
Katsuya, N., Kumagaya, S., Ayaya, S., Mukaiyachi, I., Hashimoto, K., Okuda, K., Suzuki, W.
The University of Tokyo, Health Sciences University of Hokkaido
“Tojisha-Kenkyu” (self-directed research) is a practice born in Urakawa, Hokkaido, Japan, and is led by people with mental illness. It is a practice in which people with auditory hallucinations and other symptoms consider the mechanisms and methods of dealing with their problems through dialogue with peers. How do these experiences change one's perception of oneself and one's meaning to the practice of "Tojisha-Kenkyu"? In this study, we examined what the experience of “Tojisha-Kenkyu” brought to them and how they made sense of the practice. Thirty-seven participants (with mental illness, developmental disabilities, or hearing voices) were interviewed. The interviewees were asked about their experiences with “Tojisha-Kenkyu” and overall summaries of the experiences. Specifically, we asked about (1) What is different about myself before and after working on “Tojisha-Kenkyu”, (2) what was good about encountering “Tojisha-Kenkyu”, and (3) what was good and difficult about working on “Tojisha-Kenkyu”. Regarding the differences from their previous selves, the participants responded that their perception of symptoms such as auditory hallucinations had changed, that they were able to utilize their own weaknesses, and that they were able to be objective about their own situation. The findings are discussed in terms of personal recovery and cognitive feeling.
[P-21] The intrinsic value of autonomous machines: how human-robot interaction enhances consequence satisfaction
Kiichiro Suemitsu (1), Midori Ban (1), Hideyuki Takahashi (2), Tomohiro Ishizu (3), Hiroshi Ishiguro (1)
(1) Osaka University, (2) ATR, (3) Kansai University
With the advancement of technology, we have had increasing opportunities to interact with artificial agents. Many of these agents are designed to resemble humans in their appearance and movements. However, aside from these superficial elements, the current relationship between humans and agents seems to be predominantly one-way. Existing research has pointed out that when considering relationships between humans, it is important to build partnerships that are equal and mutually interactive, rather than one-way relationships. Furthermore, these principles have been shown to be inherently important even in the context of interactions between humans and machines. Based on these considerations, we conducted experiments to explore the importance of reciprocity in interactions between humans and artificial entities such as agents. We examined whether consequences achieved in collaboration with an agent are more highly valued by humans than consequences achieved by a human alone. The study compared conditions where participants engaged in co-creation with a robot and conditions where participants worked alone but received only approval from a robot. The experimental results confirmed that engaging in co-creation with a robot resulted in a more positive impression of the robot and the collaborative work.
[P-22] How do Prefrontal Cortical Circuits Process Noxious Information for Adjusting Future Behaviors?
The University of Tokyo
An essential component of learning and decision-making is the ability to learn reward/punishment structures of the environment which an animal interacts with and to swiftly adjust its future behaviors in response to changes in the environment. Prefrontal cortical circuits are thought to play critical roles in such behavioral flexibility. However, the exact algorithms and computations that the prefrontal cortex use for processing reward and punishment information remains unclear. In this poster, (1) I will present an experimental evidence regarding how anterior cingulate cortex (ACC) in rats adjust their decision-making behaviors based on reward and punishment; (2) I will also present data obtained from computational modeling and discuss how reinforcement learning (RL) frameworks help dissecting the decision algorithms that ACC implements; (3) I will then discuss how these experimental and computational studies can possibly be extended to address how prefrontal cortical circuits predict and evaluate pain for adjusting future behaviors.
[P-23] Reliability of Representational Space from Similarity Judgements Across Multiple Dimensions and Languages
Johan A. Gamba, Grey Johnson, Brian Odegaard
University of Florida
Comparing and assessing the similarity between objects is one of the most fundamental cognitive processes for human beings. The raise of multivariate techniques such as Representational Similarity Analysis (RSA), allowed us to understand better the neural activations patterns underlying those cognitive comparisons in terms of specific regions, moments, or conceptual distances. However, the exploration of similarity judgments about different properties across multiple categories remains to be elucidated. In this study, we aimed to investigate the relationship between multiple psychological similarity dimensions and fMRI voxel-based RSA Matrices in regions of the ventral pathway and prefrontal areas (dorsolateral PFC). Specifically, by asking about similarities on different properties of an object (shape, color, category, etc.) online, we want to understand how reliable the cognitive evaluation is when comparing it with the neural matrices of specialized areas. Our current behavioral results have shown that the representational space across similarity judgments is highly conserved across different languages and individual items of a certain category. Moreover, we have also seen a strong behavioral differentiation in some categories like animacy. We currently hypothesize that the correlation between behavioral and fMRI similarity matrices will be larger for specialized regions that involve certain objective features.
[P-24] Top-Down and Bottom-Up Information Flow in the Predictive Processing of Natural Images
Hiroki Kojima, Keisuke Suzuki, Yuichi Yamashita
NCNP, Hokkaido University
Predictive processing is assumed to underpin perception, yet most computational models are designed for low-dimensional inputs. It is unclear whether these mechanisms are also applicable to high-dimensional inputs, such as natural images. In this study, we used PredNet, a deep neural network for video prediction that is designed as a hierarchical predictive system. We investigated whether the information processing of PredNet for high-dimensional inputs can be interpreted within the context of hierarchical predictive processing, with particular emphasis on top-down and bottom-up information flow and precision metrics. We evaluated the relative significance of top-down and bottom-up information, which aligns with precision in Laplace approximations, using the Hilbert-Schmidt Independence Criterion (HSIC). Our results showed that increasing the weight of loss in higher layers leads to a prediction process more influenced by priors based on training data. Additionally, we observed an increase in top-down information flow as quantified by HSIC. These findings suggest that the information processing in PredNet can be partially understood through the principles of predictive processing.
[P-25] A Deep Generative Model for Extracting Shared and Private Latent Representations from Multimodal Data
Kaito Kusumoto, Shingo Murata
Integrative learning of multimodal data has the potential for various applications, such as robotics and medicine. This study aims to develop a computational model that can learn multimodal data. Each modality is considered to have low-dimensional latent representations; however, these representations are not always fully shared with another modality. Therefore, we assume that latent representations of multimodal data consist of shared representations across all the modalities and private representations unique to each modality. Under this assumption, we propose a deep generative model that can learn to extract these different latent representations from both non-time-series and time-series data in an end-to-end manner. We conducted simulation experiments to evaluate our proposed model with an artificial multimodal dataset of image and stroke handwriting data with shared and private information. Experimental results demonstrate the ability of our model to extract shared and private representations, cross-modal reconstruction, and joint-modal reconstruction.
[P-26] Heartbeat-Evoked Potential Mediated by the Cardiac Cycle Phases
Yuto Tanaka, Yuri Terasawa, Satoshi Umeda
Interoception refers to the perception of the internal bodily state, including the perception of heartbeat and respiration. Perception of the stimuli is affected by the phase of the cardiac cycle, systole or diastole. This tendency is thought to be related to the perception and processing of interoception. In this study, we investigated how the perception of nonaffective auditory tone is associated with the cardiac cycle phase and whether the difference of cardiac cycle phases is reflected in the psychophysiological measure, heartbeat-evoked potential (HEP). We conducted an oddball task in which pure tones were presented at 10 ms (late diastole condition), 200 ms (systole condition), or 500 ms after the R wave (diastole condition). Greater HEP amplitudes were observed when the tone was presented during diastole than during systole or late diastole. These results indicate that the HEP reflects differences in perception and processing of interoception between the cardiac cycle phases.
[P-27] Interoceptive attention modulates cognitive feeling in visual perception
Yusuke Haruki, Keisuke Suzuki, Kenji Ogawa
Our perceptions are accompanied by subjective confidence, a one form of cognitive feeling. Previous studies found that such confidence judgments do not depend solely on the accuracy of the perception by suggesting the importance of ongoing bodily signals (i.e., interoception). Here, we conducted two experiments to measure perceptual accuracy and confidence while manipulating the precision of interoception via attentional focus. Specifically, a total of 29 participants performed a dot motion direction discrimination task while providing trial-by-trial confidence ratings. Prior to each trial, they directed their attention either towards interoception (heartbeat) or exteroception (tone sound for auditory and vertical line for visual sensation). The results showed that after interoceptive attention trial, supposed to enhance the relative precision of interoception, participants exhibited a more conservative subjective confidence rating when responding erroneously in the visual discrimination. This effect was not observed in trials where participants had correct responses. Notably, there were no statistically significant differences between conditions in terms of perceptual accuracy and metacognitive efficiency (i.e., accurate monitoring of one's own perceptual performance). These findings suggest that interoceptive signals may jointly shape subjective confidence in the perception of the external world.
[P-28] Relationship between physiological synchrony and emotional cognition and performance in collaborative tasks
Aiko Murata, Shiro Kumano, Kazuaki Honda, Naoki Saijo
NTT Communication Science Laboratories
Physiological interpersonal synchrony, as measured by autonomic nervous system activity, has been shown to be related to social effects such as a sense of togetherness and group cohesion. However, the relationship between physiological synchrony and subjective feelings and group performance has not been fully investigated. We conducted several experiments to investigate how people's subjective feelings and performance are related to physiological synchrony. In these experiments, we simultaneously recorded the heart rates of several participants in situations that required cooperation or coordination among the participants. The results suggest that the intensity of physiological synchrony during face-to-face interactions is related to the intensity of participants' subjective arousal. Furthermore, physiological synchrony among members was found to increase gradually as group performance improved. These findings may suggest that people's subjective emotions and overall group performance may change when physiological states are shared by multiple people. To better understand the psychophysiological mechanisms underlying larger group phenomena such as crowd frenzy and group cooperation, attention may also need to focus on the physical and physiological connections of the group.
[P-29] Temporal Integration
Fernando Rodriguez, Anindya Ghosh, Oluwaseyi Jesusanmi
University of Sussex
While different approaches from cognitive science and related fields share the view that a temporal aspect is fundamental to our phenomenological experience, our perception of time seems in direct tension with the physically grounded notion of it. We present a formal framework centered on the idea that sensory incompleteness translates into temporally dense constructions of the perceptual present.
[P-30] Gaze crossing in virtual space: examining the influence of two-way contingency in infants’ real-time interactions with adults
Youtao Lu, Megumi Ishikawa, Maria Gohlke, Tomoko Isomura, Leonardo Zapata Fonseca, Tom Froese, Sho Tsuji
The University of Tokyo, Tsuda University, Nagoya University, Okinawa Institute of Science and Technology Graduate University
A huge body of research has shown that social contingency facilitates infants’ development in various domains, but the mechanisms behind this facilitatory effect remain to be elucidated. In many existing studies, infants were only observing contingent actions coming from an individual or entity, instead of actively engaging in continuous exchanges of contingent actions. Examination of such uni-directional contingency may be insufficient to understand the underlying mechanisms, as contingency is almost always bi-directional in real-time interactions. To address this limitation, we modified the perceptual crossing paradigm to allow real-time interactions between infants and adults in a minimalist setting. In this new paradigm, an infant uses their eye gaze to explore a 2-D virtual space reciprocally with an adult. Our preliminary result shows that the encounters of eye gaze occurred more often and were maintained longer in contingent interactions. This finding is comparable to those reported in perceptual crossing studies of adults and adolescents. It suggests that enhanced (joint) attention may play an important role in explaining the facilitatory effect of contingency. In addition, the enhanced attention can be sufficiently explained by the dynamics of real-time interactions, with no agency attribution presupposed.
[P-31] Imitation Between Robots with Different Morphologies through DMBN
H. Emre Aktas (1), Yukie Nagai (2), Erhan Oztop (3,4), Emre Ugur (1)
(1) Bogazici University, (2) The University of Tokyo, (3) Osaka University, (4) Ozyegin University
In this study, We showed that the previously introduced Deep Modality Blending Networks method can be utilized to achieve imitation behavior between robots with different morphologies. Using that method we created a common latent space by blending the representations constructed from the observations gathered from different robot modalities. To showcase this ability, we used 2 morphologically very different robots namely a differential drive mobile robot and a manipulator with 6 degrees of freedom in our experiments. We also showed that, our method can be used both for learning a policy and achieving goal imitation between them. Furthermore, to show how the blending between the representations of different modalities occur, we analyzed the states of the internal neurons of the network as training progresses.