I am excited to read the whole interesting article of yours. I just skimmed it for now, reading the parts of most interest to me.

As you asked for input, some questions came to mind (sorry in advance, if I am asking something already addressed in your paper) :

1. Can your model also address the "temporal continuity" of the subjective experience, exclusive to humans and certain animals? I would doubt that, as I recall this important feature of the subjective experience depends on the prefrontal cortex. This is one of the reasons why animals with smaller or no prefrontal cortices seem to have disrupted or no intentionality (a component of the subjective experience), despite having fully powered hippocampi.

2. Imaging studies show that there is a quantitative (and to some degree qualitative) difference between the activation of brain areas during remembering episodic memories and experiencing the subjective experience, first hand. In the latter, cortical (and subcortical) areas responsible for the processing of the experience light up vividly, while in remembering the same memories, the areas light up faintly and partially. Thus, I am under the impression that besides the hippocampus, maybe certain cortical areas are needed for the subjective experience to occur.

3. Highly hypnotizable individuals under deep hypnosis can experience (suggested) imaginations as well as episodic memories [or even false episodic memories] like "first-hand" subjective experiences. Neural correlates of hypnotic experiences again point to certain areas such as the dorsolateral PFC, ventrolateral PFC, ACC, and (a great degree of activation of) sensory cortical areas. This again suggests that besides the hippocampus, perhaps other brain areas and networks are needed for the subjective experience to be experienced the way we "believe in it" rather than only seeming to be remembered.

4. The primary sensory areas (like V1 and A1) might activate when either the person is experiencing something for real or he (under hypnosis or some other conditions) believes that what he is perceiving is the real first-hand experience. Is it possible to address the activation of primary sensory areas with hippocampal-only networks?

5. I understand your model is for neurotypical subjects. But lets remember the famous patient HM whose hippocampi had been destroyed by a viral infection. He had disrupted episodic memory. However, his subjective experience together with his past musical skills and memories were all still there (minus his anterograde amnesia). This may suggest that the subjective experience depends on a multitude of brain networks, and not only hippocampus.

I see you have already discussed his case. But despite probable qualitative differences between HM and neurotypical people, HM had the subjective experience, right? Plus who can say the subjective experience of each single person is not unique qualitatively? (Of course, if it exists.)

Again apologies for commenting before reading your paper thoroughly.

A most fascinating review/opinion paper! IMHO, the section about the Hpc as the experience simulator can be lengthened. As the Hpc is a core temporal component of the default mode network, its activation can also signify mind-wandering, non-attention, or susceptibility to distraction. Some fMRI data from my end shows this, and I think it's good to point this out - that higher magnitudes of Hpc activation does not always equate with greater prospective thinking or memory-related processes. Under certain task conditions, Hpc deactivation, co-occurring with prefrontal DMN deactivations, may represent attention and executive processing of extrinsic or perceptual information. Some discussion about these ideas/potential phenomena would make this paper even more interesting.

Jimmy Y. Zhong Thank you very much for your comments, and of course, for taking time to read the paper.

I agree; the mental simulation part could be lengthened. In fact, I think the theory really should be lengthened to a book eventually. But I was getting push-back from publishers for an already long article, and had to leave a lot out. I mostly wanted to get across the idea that the DMN + the HPC could generate the simulations that we perceive as 'mind'.

Hippocampal activity is a bit hard to measure, because the HPC is almost always in use, even during sleep. While awake, we are constantly forming new memories, just in case something happens that we need to remember. At night, the HPC processes episodic memories during slow-wave sleep, and helps the neocortex practice semantic and procedural tasks during REM sleep. Since measurement is against a baseline, and the baseline is usually quite active, we usually only measure an increase during specific memory-heavy or simulation-heavy tasks.

As you saw in my paper, I don't include the hippocampal complex in the default mode network (although I know it's often included in the literature), because I think it is its own network. It works hand-in-glove with both the extrinsic neocortex (senses + motor), generating memories, and with the DMN, forming mental simulations. Thus, I think of it as the 'movie-making' and 'movie-playback' network, constructing and re-constructing episodic memory movies and mental simulations.

I think about the sensory+motor neocortex as the 'extrinsic' or world-oriented network, tasked to handle immediate interaction between the body and its environment. And of course, I think of the DMN as the 'intrinsic' network, tasked to handle planning, extrapolating beyond the current scene, and other jobs of the imagination.

This, I think, is why we can drive a car, without forming any memory of that drive, because if the DMN busies the HPC, creating a daydream or social rehearsal fantasy or similar simulation, then the information about the drive never makes it into the HPC. The extrinsic network, meanwhile, is conducting the drive as usual, without danger, because it does need memory in order to handle its stereotypical driving functions. I go into the idea of 'Driving Mind' more thoroughly in this short explainer video:

https://vimeo.com/98785998

I think that 'flow states' are an example when the extrinsic network is working at full volume, but the DMN is almost inactive, and the HPC is only forming memories, just in case something goes wrong. When we are 'lost in reverie' or deep in a daydream, then the DMN is turned up, and the extrinsic network turns down. In both cases, the HPC is still active, but not necessarily above the already active baseline.

It's also worth considering the distinct functional roles of hippocampal field CA1 vs. CA3. According to rat studies, CA3 is usually trying to predict what the next moment of episodic memory will look like, by comparing the current moment's input with past memories. Whereas, CA1 switches within each theta cycle between 'broadcasting' first CA3's prediction, and then a more straight-forward expression of data arriving directly from the entorhinal cortex. I think CA3, because of its heightened simulational ability, is likely to be more associated with the DMN, and CA1 more with the extrinsic neocortex.

You mention that HPC activity is heightened even sometimes, when neither memory nor simulation tasks are happening. I'm not aware of that exception to the rule. It would be interesting to hear more about that.

Thanks again for your time and response!

Vahid Rakhshan Thanks for your comments! I will break up my answers by bullet point, to try to keep each answer succinct.

#1. Yes, I think that the episodic memory system provides an excellent way of understanding 'temporal continuity'. The 'stream of consciousness' is really a 'chain of memories'. Continuity is created precisely by the hippocampal complex (HC) keeping us connected to our past, and helping the DMN imagine our future.

In fact, episodic memory provides the only possible mechanism of psychological continuity. If you ever witness someone with a damaged episodic memory system, they sound like broken records, repeating the same scripts over and over again.

You couldn't follow a movie plot or read a book without episodic memory, because EM keeps the current moment connected to what came before. You would always be completely geographically lost without episodic memory, unless you stayed within well-known environments that are familiar to your neocortex.

Anatomically, the story is like this: time cells in the HC are the temporal version of the HC's 'place cells'. Just as place cells help the brain map where in a space a memory was generated, so too the time cells help the brain map where in a sequence a memory was generated. So, if I drive on a brand new route, my memory system is trying to connect the various views of the drive to a sequence, so that I can find my way on that route again later.

Also the HC is simultaneously generating a memory of now, but also looking backward and looking forward. Field CA3 of the HC, in particular, is tasked with trying to predict what the next moment's memory will look like. It does this by comparing the current moment's experiential data with memory of previous episodes. And CA3 is also tasked with invoking old episodic memories, wholesale. So the very system that is creating subjective experience (i.e. brand new memories) is also constantly looking backward and projecting forward to create temporal continuity, and keep us on our goals.

The actual goal- or intention-setting, by contrast, is set up primarily by the DMN (at least goals that extend more than a couple seconds into the future), but the DMN uses the HC's simulational powers to help visualize potential futures and the steps that they require. The DMN uses the HC as a previz tool in order to facilitate its planning functions, and it uses its memory of those plans to keep on task.

Is 'intentionality' a feature of subjective experience? I think it's rather an illusion of it.

Subjective experience (i.e. a brand new episodic memory) includes a lot of upstream brain processing in it, including the experience of 'will', but these are only representations, for the sake of memory. It's important for the sake of memory that my organism can remember who did what, so it confabulates a unitary agentive self that seems to be willing things. But that self only exists within the memory; the actual intention is set by coalitions of neurons, primarily in the prefrontal cortex. For example, Libet's timing experiment makes sense within our model, because the 'will' to enact the behavior happened from upstream nodes , and took a portion of a second in order to make it into the new memory.

Vahid Rakhshan # 2. Imaging studies show that there is a quantitative (and to some degree qualitative) difference between the activation of brain areas during remembering episodic memories and experiencing the subjective experience, first hand. In the latter, cortical (and subcortical) areas responsible for the processing of the experience light up vividly, while in remembering the same memories, the areas light up faintly and partially. Thus, I am under the impression that besides the hippocampus, maybe certain cortical areas are needed for the subjective experience to occur.

Absolutely agreed! An episodic memory engram can be analogized to a video camera's digital file (but less accurate). It is just code, that would appear meaningless, unless played back on the right bit of hardware. And that hardware is the neocortex for senses and limbic system for feelings. Just as a video code needs a video and auditory processors and pixels + speakers to re-construct a video movie, so too the movie of memory needs the other cortices for playback. External input plays from bottom-up through the sensory systems, and the new memory plays from top-down, reactivating many of the same cortices that contributed to the memory. This serves partially as priming for the neocortex, and as an efficient pattern-matching system for memory. It helps inform the whole brain with a unified story about 'what just happened' and it also creates a full multimodal input into the DMN.

The DMN is charged with making sense of old memories, but it also reviews the brand new one, to extrapolate beyond the current scene (for example, imagining what someone else is thinking). Since the episodic memory code is sparse pointers (as per our 'player piano roll' analogy in the paper), the memory engram is not itself the experience; it is the code to re-activate the senses and feelings that made up the memory. It is that global multimodal activation which is the experience of memory (including the experience of 'now'), and that activation informs the DMN's extrapolation and future-planning process.

Of course, an old memory is usually faint and story-like, with only tiny bits of salient qualia left. But not so with 'flashbulb memories'. Many of us still have vivid memories of 9/11, for example. And very recent salient memories, like something dramatic that happened a few seconds ago, can easily be replayed with full technicolor vividness. That is still episodic memory, even though it just happened.

Of course, the brand new memory (i.e. subjective experience) needs to be very full of detail, because the brain does not yet know which of those details will be important to remember. So the brain constantly sends details (especially dynamic details, like vision and hearing) to the HC for encoding, but most of those details are soon thereafter discarded, because nothing particularly memorable happened during that time. If the HC did not process all that originally vivid qualia, then none of it would be available later, as HM teaches us.

During slow wave sleep, the HC consolidates the episodic memory engrams, presumably for the sake of data management, discarding unnecessary details so that only the important parts of the memory are left. When that memory is recalled, it no longer has all those details attached to it, and so it only stirs up faint neocortical activity.

But also, I think we humans probably experience recalled episodic memory in a fainter and more story-like way than other hippocampal animals. This is because of language. Language makes an excellent mnemonic, and can shorten a whole movie of memory into a simple narrative. That narrative, because it relies on language, only needs a few shreds of qualia attached to it, in order to make sense. With non-language animals, by contrast, the movie (and the mental maps that are attached to it) are all they have to go on. So I think it's likely that non-language animals tend to have more vivid recall.

Vahid Rakhshan 3. Highly hypnotizable individuals under deep hypnosis can experience (suggested) imaginations as well as episodic memories [or even false episodic memories] like "first-hand" subjective experiences. Neural correlates of hypnotic experiences again point to certain areas such as the dorsolateral PFC, ventrolateral PFC, ACC, and (a great degree of activation of) sensory cortical areas. This again suggests that besides the hippocampus, perhaps other brain areas and networks are needed for the subjective experience to be experienced the way we "believe in it" rather than only seeming to be remembered.

Honestly, I don't know a lot about hypnosis, although I think it's quite interesting. But I think that you're describing a guided confabulation.

In amnesiacs with intact hippocampi but memory deficits due to prefrontal damage, it's very easy to suggest memories to them. Often, all you have to do is ask them about something they don't remember, and they will make up an experience, and believe it with full confidence. Or they will cobble together details that someone else told them, and present it as a memory that happened to them.

Interestingly, amnesiacs with complete bilateral hippocampal damage do not seem to confabulate, suggesting that the hippocampus is necessary to create memory-like constructs, even faked ones.

The brain regions you're describing are mostly DMN nodes (with the sensory nodes 'borrowed' in order to create the vividness of the implanted memories, as per #2, above). The dlPFC, of course, seems to be at the top of both the DMN and extrinsic neocortex hierarchies, and the ventrolateral PFC is in charge of reviewing memories. The DMN, as I described in my response to Jimmy Zhong, above, uses the HC to create vivid simulations, so I think that's what's happening here. Under hypnosis, the DMN may be guided in creating a vivid simulation, which piggy-backs on the same circuits as moment-to-moment episodic memory, so the simulation may be recalled as if it were an actual memory.

Also worthy of note is that dreams seem to happen in the HC, as well. During REM sleep, the HC stays active in order to provide sequence and timing information to help the neocortex practice new semantic and procedural skills. The near-random information from the neocortex still arrives at the HC, which confabulates a realistic-seeming experience out of it. We evolved to mostly forget our dreams, so they don't get confused with real memories.

What's really important about that last paragraph is that even moment-to-moment experiencing is like a form of dreaming, albeit constrained by the information coming in from the neocortex. If you take off that constraint, then the HC continues on its dream. Ralf-Peter Behrendt, a UK psychiatrist who has published a very similar model to ours, sees schizophrenic perception as a problem of an under-constrained hippocampal field CA3. Under hypnosis and in schizophrenia, the experience is probably confabulated very much like a dream, mixing real-world and imaginary stimuli.

Vahid Rakhshan 4. The primary sensory areas (like V1 and A1) might activate when either the person is experiencing something for real or he (under hypnosis or some other conditions) believes that what he is perceiving is the real first-hand experience. Is it possible to address the activation of primary sensory areas with hippocampal-only networks?

I'm not quite sure I understand this question. Maybe I addressed it in #3?

Vahid Rakhshan 5. I understand your model is for neurotypical subjects. But lets remember the famous patient HM whose hippocampi had been destroyed by a viral infection. He had disrupted episodic memory. However, his subjective experience together with his past musical skills and memories were all still there (minus his anterograde amnesia). This may suggest that the subjective experience depends on a multitude of brain networks, and not only hippocampus.

I see you have already discussed his case. But despite probable qualitative differences between HM and neurotypical people, HM had the subjective experience, right? Plus who can say the subjective experience of each single person is not unique qualitatively? (Of course, if it exists.)

As you mention, I dedicated a section of the paper to HM and other patients like him. And yes, those patients are why I specified my explanandum as 'neurotypical subjective experience'.

Was HM a philosophical zombie? Honestly, I don't know. I have scoured the literature for someone asking the appropriate questions of any of these patients, but I haven't found it. I think it's entirely possible no one thought to ask. I'm trying to connect to Eleanor Maguire, a UK researcher with a whole population of patients with complete bilateral hippocampal damage, to see if she can probe them about this and related questions, but she hasn't replied yet.

The model doesn't really have anything to say about non-hippocampal experience, if such a thing exists. It seems clear that those of us with working hippocampi do not remember or own non-hippocampal experience, or everything would feel like an echo, of neocortical followed by hippocampal experiences.

But it also seems likely that episodic memory, by having the 'newscast' of subjective experience, obscures whatever upstream processing came before it, since HM and similar patients still had access to that upstream processing. HM saw and heard, but maybe in the way that only a p-zombie would. We neurotypicals also use that upstream processing to drive immediate behavior, because the data is much more robust and also, the trip through the hippocampus is too slow. But, according to our model, we don't actually perceive any of that neocortical processing directly. Rather, the processing happens, the nodes send reports to the HC, and the HC generates 'perception' (i.e. the new memory). That memory is created with a simulation of the pre-hippocampal processes included, so we perceive the memory version of the process, not the process itself. It's like in a dream; we feel like we're performing processes, but it's just part of the simulation.

Everything that we call 'perception' is exactly and only what is included in the new memory, which is why so much of our pre-memory processing is not included in our perceptions. We only remember/perceive the newsworthy parts, the dynamic, novel, or surprising stimuli, not the stereotypical parts, like most of our brain's processing, or the feeling of our clothes, a persistent odor, an air-conditioning hum, or the view of our own noses. None of that is newsworthy for memory, so none of that is perceived.

This includes (and this is the hardest part to embrace): all perception of the self, including the mind. 'I' only exist within my memory. There is an organism, that others interact with, and who calls himself Matt, but the 'I' that I experience from within, is just a simulation, a confabulation, a character in the movie of memory.

I've been thinking a lot about the Hard Problem lately, because it usually comes up whenever I introduce the theory to someone. And I must admit, the more I think about it, the less likely it seems that HM had subjective experience. Yes, he saw, yes he heard, but he may very well have been a philosophical zombie, no inner experience at all. It feels icky and odd to believe that, but theoretically, I think it's the most elegant solution, that experience is exactly and only a product of the episodic memory system (obviously in collaboration with other networks). Without the hippocampus, there's nothing to bind the data of experience together; there's no broadcast of a unified story, there's no movie.

But I can't speak authoritatively one way or the other, without access to the appropriate patient population.

Thanks again for taking the time. I look forward to hearing your further thoughts, once you've completed the paper.

best,

matt faw

Matt Faw As regards to the paradoxical activity of the Hpc I mentioned, "deactivations" (relative to a baseline, that is) cannot be seen as a standalone phenomenon but would need to be interpreted with respect to DMN regions and regions of the extrinsic neocortex you mentioned (e.g., in the midline and posterior sections: e.g., postcentral gyrus, angular gyrus/TPJ, precuneus, RSC, hMT+, to name a few).

Suppose that you obtain findings showing that lower activations (or "deactivations") in the frontal DMN/intrinsic network regions AND the Hpc co-occur with better (more accurate) goal-directed performance; and that this better pefrormance covary/correlate with higher activation in the extrinsic network regions simultaneously, then something is telling you that your task-at-hand engages the extrinsic network to a greater extent. In this case, with regard to the perceptual or cognitive demands imposed by this task, it would not be irrational to claim that Hpc functioning would be more attuned or susceptible to the influences of the DMN than to the effects of the extrinsic network. Findings like this could suggest a prominence or prevalence of bottom-up processing in which Hpc involvement need not be essential or mandatory for accurate behavioral performance.

Jimmy Y. Zhong Good points, thank you. And I do agree that the Hpc is probably less necessary for many extrinsic tasks than one would think, given its role in our mnemonic life. Memory is formed in parallel to, and as result of, the extrinsic sensing/behavior; it is not a key component of immediate behavior. We could never hit a fast-ball, for example, if we had to rely on our relatively slow memory/perception, rather than on the quicker pre-memory sensory/motor chains. And the fact that we can drive a car while having a conversation, and yet have most of our memory be about the conversation, shows that memory is a poor conduit of information for immediate behavior.

That said, I do think that the history of imaging studies has included some mis-steps, when too much inference was drawn. For example, of course, the discovery of the DMN was late in being discovered, because studies were looking for a delta from the baseline, but the DMN was itself the at-rest baseline. And some nodes may be active in a given task, and therefore show up on the fMRI, but they may not be necessary for the task (e.g. they may just handle aspects of the task, but the task can go on without them). And there's plenty of redundancy, plasticity, and alternate wiring in the brain. Etc. So I'm guessing that there are going to be more rude surprises in the future, about precisely what increases and decreases in activation means for a node's role in a task.

I think the Hpc's baseline is probably more active than we've treated it, because memory formation seems to be a near-constant feature of waking life (even if most of it quickly fades). And I think the Hpc probably evolved first to bind together extrinsic data into episodic memories (and the DMN probably evolved first to make sense of memories), and they both probably co-evolved the further simulational powers later, piggy-backing on the architecture of memory. So it wouldn't surprise me if active simulation of imaginary scenes takes more blood flow to the Hpc, if the Hpc is less efficient at doing that task. Imagining things can actually be some work. So can memorizing things. But just having normal memory processes happening in the background, just in case, doesn't feel like work at all. That may reflect the hundreds of millions of years of the Hpc evolving as a memory machine, and only recent evolution as part of a cognitive process. So it's possible we may be somewhat misled by imaging.

Thanks again for your feedback and discussion. Please let me know if you have further thoughts or questions about the model!

best,

matt faw

Matt Faw Thanks for your comprehensive replies, and sorry for my delay. I was very busy. I learned a good deal by reading your responses, but I will limit my replies to the points of conflict.

You said: "

Vahid Rakhshan Thanks for your comments! I will break up my answers by bullet point, to try to keep each answer succinct. #1. Yes, I think that the episodic memory system provides an excellent way of understanding 'temporal continuity'. The 'stream of consciousness' is really a 'chain of memories'. Continuity is created precisely by the hippocampal complex (HC) keeping us connected to our past, and helping the DMN imagine our future. In fact, episodic memory provides the only possible mechanism of psychological continuity. If you ever witness someone with a damaged episodic memory system, they sound like broken records, repeating the same scripts over and over again. You couldn't follow a movie plot or read a book without episodic memory, because EM keeps the current moment connected to what came before. You would always be completely geographically lost without episodic memory, unless you stayed within well-known environments that are familiar to your neocortex. Anatomically, the story is like this: time cells in the HC are the temporal version of the HC's 'place cells'. Just as place cells help the brain map where in a space a memory was generated, so too the time cells help the brain map where in a sequence a memory was generated. So, if I drive on a brand new route, my memory system is trying to connect the various views of the drive to a sequence, so that I can find my way on that route again later. Also the HC is simultaneously generating a memory of now, but also looking backward and looking forward. Field CA3 of the HC, in particular, is tasked with trying to predict what the next moment's memory will look like. It does this by comparing the current moment's experiential data with memory of previous episodes. And CA3 is also tasked with invoking old episodic memories, wholesale. So the very system that is creating subjective experience (i.e. brand new memories) is also constantly looking backward and projecting forward to create temporal continuity, and keep us on our goals. The actual goal- or intention-setting, by contrast, is set up primarily by the DMN (at least goals that extend more than a couple seconds into the future), but the DMN uses the HC's simulational powers to help visualize potential futures and the steps that they require. The DMN uses the HC as a previz tool in order to facilitate its planning functions, and it uses its memory of those plans to keep on task. Is 'intentionality' a feature of subjective experience? I think it's rather an illusion of it. Subjective experience (i.e. a brand new episodic memory) includes a lot of upstream brain processing in it, including the experience of 'will', but these are only representations, for the sake of memory. It's important for the sake of memory that my organism can remember who did what, so it confabulates a unitary agentive self that seems to be willing things. But that self only exists within the memory; the actual intention is set by coalitions of neurons, primarily in the prefrontal cortex. For example, Libet's timing experiment makes sense within our model, because the 'will' to enact the behavior happened from upstream nodes , and took a portion of a second in order to make it into the new memory.

5 days ago

Matt Faw Added a reply

Vahid Rakhshan # 2. Imaging studies show that there is a quantitative (and to some degree qualitative) difference between the activation of brain areas during remembering episodic memories and experiencing the subjective experience, first hand. In the latter, cortical (and subcortical) areas responsible for the processing of the experience light up vividly, while in remembering the same memories, the areas light up faintly and partially. Thus, I am under the impression that besides the hippocampus, maybe certain cortical areas are needed for the subjective experience to occur. Absolutely agreed! An episodic memory engram can be analogized to a video camera's digital file (but less accurate). It is just code, that would appear meaningless, unless played back on the right bit of hardware. And that hardware is the neocortex for senses and limbic system for feelings. Just as a video code needs a video and auditory processors and pixels + speakers to re-construct a video movie, so too the movie of memory needs the other cortices for playback. External input plays from bottom-up through the sensory systems, and the new memory plays from top-down, reactivating many of the same cortices that contributed to the memory. This serves partially as priming for the neocortex, and as an efficient pattern-matching system for memory. It helps inform the whole brain with a unified story about 'what just happened' and it also creates a full multimodal input into the DMN. The DMN is charged with making sense of old memories, but it also reviews the brand new one, to extrapolate beyond the current scene (for example, imagining what someone else is thinking). Since the episodic memory code is sparse pointers (as per our 'player piano roll' analogy in the paper), the memory engram is not itself the experience; it is the code to re-activate the senses and feelings that made up the memory. It is that global multimodal activation which is the experience of memory (including the experience of 'now'), and that activation informs the DMN's extrapolation and future-planning process. Of course, an old memory is usually faint and story-like, with only tiny bits of salient qualia left. But not so with 'flashbulb memories'. Many of us still have vivid memories of 9/11, for example. And very recent salient memories, like something dramatic that happened a few seconds ago, can easily be replayed with full technicolor vividness. That is still episodic memory, even though it just happened. Of course, the brand new memory (i.e. subjective experience) needs to be very full of detail, because the brain does not yet know which of those details will be important to remember. So the brain constantly sends details (especially dynamic details, like vision and hearing) to the HC for encoding, but most of those details are soon thereafter discarded, because nothing particularly memorable happened during that time. If the HC did not process all that originally vivid qualia, then none of it would be available later, as HM teaches us. During slow wave sleep, the HC consolidates the episodic memory engrams, presumably for the sake of data management, discarding unnecessary details so that only the important parts of the memory are left. When that memory is recalled, it no longer has all those details attached to it, and so it only stirs up faint neocortical activity. But also, I think we humans probably experience recalled episodic memory in a fainter and more story-like way than other hippocampal animals. This is because of language. Language makes an excellent mnemonic, and can shorten a whole movie of memory into a simple narrative. That narrative, because it relies on language, only needs a few shreds of qualia attached to it, in order to make sense. With non-language animals, by contrast, the movie (and the mental maps that are attached to it) are all they have to go on. So I think it's likely that non-language animals tend to have more vivid recall.

5 days ago

Matt Faw Added a reply

Vahid Rakhshan 3. Highly hypnotizable individuals under deep hypnosis can experience (suggested) imaginations as well as episodic memories [or even false episodic memories] like "first-hand" subjective experiences. Neural correlates of hypnotic experiences again point to certain areas such as the dorsolateral PFC, ventrolateral PFC, ACC, and (a great degree of activation of) sensory cortical areas. This again suggests that besides the hippocampus, perhaps other brain areas and networks are needed for the subjective experience to be experienced the way we "believe in it" rather than only seeming to be remembered. Honestly, I don't know a lot about hypnosis, although I think it's quite interesting. But I think that you're describing a guided confabulation. In amnesiacs with intact hippocampi but memory deficits due to prefrontal damage, it's very easy to suggest memories to them. Often, all you have to do is ask them about something they don't remember, and they will make up an experience, and believe it with full confidence. Or they will cobble together details that someone else told them, and present it as a memory that happened to them. Interestingly, amnesiacs with complete bilateral hippocampal damage do not seem to confabulate, suggesting that the hippocampus is necessary to create memory-like constructs, even faked ones. The brain regions you're describing are mostly DMN nodes (with the sensory nodes 'borrowed' in order to create the vividness of the implanted memories, as per #2, above). The dlPFC, of course, seems to be at the top of both the DMN and extrinsic neocortex hierarchies, and the ventrolateral PFC is in charge of reviewing memories. The DMN, as I described in my response to Jimmy Zhong, above, uses the HC to create vivid simulations, so I think that's what's happening here. Under hypnosis, the DMN may be guided in creating a vivid simulation, which piggy-backs on the same circuits as moment-to-moment episodic memory, so the simulation may be recalled as if it were an actual memory. Also worthy of note is that dreams seem to happen in the HC, as well. During REM sleep, the HC stays active in order to provide sequence and timing information to help the neocortex practice new semantic and procedural skills. The near-random information from the neocortex still arrives at the HC, which confabulates a realistic-seeming experience out of it. We evolved to mostly forget our dreams, so they don't get confused with real memories. What's really important about that last paragraph is that even moment-to-moment experiencing is like a form of dreaming, albeit constrained by the information coming in from the neocortex. If you take off that constraint, then the HC continues on its dream. Ralf-Peter Behrendt, a UK psychiatrist who has published a very similar model to ours, sees schizophrenic perception as a problem of an under-constrained hippocampal field CA3. Under hypnosis and in schizophrenia, the experience is probably confabulated very much like a dream, mixing real-world and imaginary stimuli.

5 days ago

Matt Faw Added a reply

Vahid Rakhshan 4. The primary sensory areas (like V1 and A1) might activate when either the person is experiencing something for real or he (under hypnosis or some other conditions) believes that what he is perceiving is the real first-hand experience. Is it possible to address the activation of primary sensory areas with hippocampal-only networks? I'm not quite sure I understand this question. Maybe I addressed it in #3?

5 days ago

Matt Faw Added a reply

Vahid Rakhshan 5. I understand your model is for neurotypical subjects. But lets remember the famous patient HM whose hippocampi had been destroyed by a viral infection. He had disrupted episodic memory. However, his subjective experience together with his past musical skills and memories were all still there (minus his anterograde amnesia). This may suggest that the subjective experience depends on a multitude of brain networks, and not only hippocampus. I see you have already discussed his case. But despite probable qualitative differences between HM and neurotypical people, HM had the subjective experience, right? Plus who can say the subjective experience of each single person is not unique qualitatively? (Of course, if it exists.) As you mention, I dedicated a section of the paper to HM and other patients like him. And yes, those patients are why I specified my explanandum as 'neurotypical subjective experience'. Was HM a philosophical zombie? Honestly, I don't know. I have scoured the literature for someone asking the appropriate questions of any of these patients, but I haven't found it. I think it's entirely possible no one thought to ask. I'm trying to connect to Eleanor Maguire, a UK researcher with a whole population of patients with complete bilateral hippocampal damage, to see if she can probe them about this and related questions, but she hasn't replied yet. The model doesn't really have anything to say about non-hippocampal experience, if such a thing exists. It seems clear that those of us with working hippocampi do not remember or own non-hippocampal experience, or everything would feel like an echo, of neocortical followed by hippocampal experiences. But it also seems likely that episodic memory, by having the 'newscast' of subjective experience, obscures whatever upstream processing came before it, since HM and similar patients still had access to that upstream processing. HM saw and heard, but maybe in the way that only a p-zombie would. We neurotypicals also use that upstream processing to drive immediate behavior, because the data is much more robust and also, the trip through the hippocampus is too slow. But, according to our model, we don't actually perceive any of that neocortical processing directly. Rather, the processing happens, the nodes send reports to the HC, and the HC generates 'perception' (i.e. the new memory). That memory is created with a simulation of the pre-hippocampal processes included, so we perceive the memory version of the process, not the process itself. It's like in a dream; we feel like we're performing processes, but it's just part of the simulation. Everything that we call 'perception' is exactly and only what is included in the new memory, which is why so much of our pre-memory processing is not included in our perceptions. We only remember/perceive the newsworthy parts, the dynamic, novel, or surprising stimuli, not the stereotypical parts, like most of our brain's processing, or the feeling of our clothes, a persistent odor, an air-conditioning hum, or the view of our own noses. None of that is newsworthy for memory, so none of that is perceived. This includes (and this is the hardest part to embrace): all perception of the self, including the mind. 'I' only exist within my memory. There is an organism, that others interact with, and who calls himself Matt, but the 'I' that I experience from within, is just a simulation, a confabulation, a character in the movie of memory. I've been thinking a lot about the Hard Problem lately, because it usually comes up whenever I introduce the theory to someone. And I must admit, the more I think about it, the less likely it seems that HM had subjective experience. Yes, he saw, yes he heard, but he may very well have been a philosophical zombie, no inner experience at all. It feels icky and odd to believe that, but theoretically, I think it's the most elegant solution, that experience is exactly and only a product of the episodic memory system (obviously in collaboration with other networks). Without the hippocampus, there's nothing to bind the data of experience together; there's no broadcast of a unified story, there's no movie. But I can't speak authoritatively one way or the other, without access to the appropriate patient population. Thanks again for taking the time. I look forward to hearing your further thoughts, once you've completed the paper. best, matt faw"

Regarding your #1 response to me:

I agree that everything (including the self and intentionality) is in the memory. But I was saying something else: that the very illusion of intentionality stems from regions outside HC.

Regarding the very important sense of "temporal continuity", I think it is mostly done by the prefrontal cortex, and not the HC. The temporal continuity depends on the working memory, which is manipulated and maintained by the PFC.

Regarding the sequence of events. Well I think that is not very much related to "temporal continuity". The stream of thoughts is about our illusion of "the present time" (about few seconds into the past), while the temporal sequencing is about every (or most of) episodic memories. And besides, the order of sequences is mostly monitored by the PFC again (not the HC). So if we are looking for "time cells", the PFC would be a better place to look.

Responses 2 and 3: I need to articulate your next responses more, before being able to answer properly. For example, (#3) the hypnotic experience I was telling is not similar at all to confabulation. Confabulation is a false memory, while hypnosis is direct first-hand experience. It can be a true or a false memory, or it can be an imagination. The content apart, it feels like our everyday "experience". Or in the case of hallucinations or lucid dreaming (which are quite similar to hypnotic experience), these are first-hand experiences. However, confabulation is the illusion of remembering a false memory. And in the hypnotic experience, the DL-PFC and ACC are turned off, so the justifications you made about them being a part of DMN and acting to create a vivid simulation cannot hold. (#2) Regarding the level of activity of cortical areas during a first-hand experience versus a remembered memory: You bring about interesting notions such as "shreds of qualia" and link them to the activity of cortical networks. And you consider these cortical networks to be the DMN. However, (1) we do not know what is "shreds of qualia", if anything like it exists. (2) The activities I was talking about (that differ in intensity between a first-hand experience and its remembering) do not occur in the DMN. They majorly occur in the primary and other sensory areas. Regarding the vivid memories, unfortunately their neural correlates are not understood yet. But that does not mean that having vivid reliving memories is necessarily a hippocampus-only mechanism. I think such vivid memories are actually some versions of "hypnotic experience", in which the person is reliving the traumatized or significant memory. And the neural correlates of hypnosis are far beyond the HC alone. (#4) Thanks, but I think my questions 3 and 4 were not addressed completely. :) (#5) Again, I think the model is confusing the working memory with episodic memory. Although HM was like a broken record repeating himself over and over, it was quite noteworthy that in every repeat, he did experience his perceptual input quite healthily. His sense of temporal continuity was intact, and he did experience the world and his thoughts. This is because despite the disruption of his episodic memory, his working memory had remained intact, and that (and not the episodic memory) is, I believe, what needed for a first-hand "experience". Reading through your quick responses, I am under the impression that we are not on the same page. For instance, this model apparently ignores the role of working memory in experiencing "the experience" and confuses working memory with episodic memory. It overlooks some important forms of conscious experience (such as hypnosis), and cannot address properly the differences in the brain imaging between living something or remembering it.

Vahid Rakhshan Thank you for your reply.

It sounds like our major point of disagreement is about working memory, so I'll focus on that. I understand why you say that working memory is more cortical than hippocampal, because the classic model of memory (long term, short term, etc.) sets it up that way. But I think that model has already been overturned by hippocampal researchers.

My explanation is related to my response to Jimmy Zhong, above, about how imaging may mislead us, because imaging is based on delta from a baseline. When researchers search for the neural correlates of working memory, the hippocampus did not show up as being more active than baseline. This was, however, because the awake hippocampus is almost always processing cortical data into new episodic memories. It is active all the time, and so hasn't always been included in tasks in which it plays an active role.

In working memory imaging studies, imaging the delta from baseline reveals only the cortical areas that are involved in the specific task at hand. But they don't reveal the hippocampus, which is also involved in the task, because its baseline doesn't change. It is only in the last few years that hippocampus studies, including single cell recordings, show that the hippocampus is fully involved in most working memory tasks, as well.

references:

Baddeley A. The episodic buffer: a new component of working memory? Trends Cogn Sci 2000, 4:417–423.

Soto D, Silvanto J. Reappraising the relationship between working memory and conscious awareness. Trends Cogn Sci 2014, 18:520–525. doi:10.1016/j. tics.2014.06.005.

Rissman J, Wagner AD. Distributed representations in memory: insights from functional brain imaging. Annu Rev Psychol 2012, 63:101–128. doi:10.1146/ annurev-psych-120710-100344.

What I'm arguing is that, yes, there is 'working memory' at the cortical level, and that is what HM used to process information. But that working memory is not directly experienced, at least not by we neurotypicals. Because the working memory cortical activity is upstream of the HC, its contents neurotypically may end up in memory/experience. In our model, however, everything we neurotypicals experience is only and completely from the hippocampal loop. All that upstream activity is never directly experienced, at least in the way that we neurotypicals think about subjective experience.

And yes, the PFC is involved in not only the cortical loops of pre-episodic 'working memory', but it is also involved in hippocampal memory. The ventrolateral PFC, in particular, interacts dynamically with the HC in memory formation and reconstruction, and the orbital frontal and ventromedial PFC both extrapolate information from the 'experience' that the HC output activates. So, it is entirely plausible that a working memory study can show activation in the cortices, especially in the PFC, and yet still fully involve the HC.

I do hope you get the chance to read the entire paper, because it is much more carefully written, all claims are fully referenced, and the threads are more carefully drawn out and inspected.

best,

matt faw

Dear Matt Faw Thanks for your reply. Sure I will gladly read your interesting paper once I can find some spare time.

You said " And yes, the PFC is involved in not only the cortical loops of pre-episodic 'working memory', but it is also involved in hippocampal memory. The ventrolateral PFC, in particular, interacts dynamically with the HC in memory formation and reconstruction, and the orbital frontal and ventromedial PFC both extrapolate information from the 'experience' that the HC output activates. So, it is entirely plausible that a working memory study can show activation in the cortices, especially in the PFC, and yet still fully involve the HC. ".

There is a misunderstanding here. I never argued against the potential involvement of the HC in the experience. I always argued in all my posts that unlike what your model proposes (that the subjective experience is carried out single-handedly by the hippocampal complex), I believe that there are other important areas and networks beyond the HC that are necessary for the subjective experience to emerge too.

However, cases like HM (who despite damaged hipoocampi did have subjective experiences) suggest that the HC and the episodic memory might not be even needed for the subjective experience to happen. The PFC and its working memory probably are.

I would gladly read your valuable article and your references. If you don't mind, I would like to write a letter to editor on it. Please let me know your opinion.

Hi Vahid Rakhshan .

You wrote: "I believe that there are other important areas and networks beyond the HC that are necessary for the subjective experience to emerge too."

You'll find no disagreement from me here. I think that's completely right. (I agreed with this above, too, in my response to #2).

Episodic memory is, of course, dependent upon upstream nodes to send reports of their newsworthy processing to the HC. Without those reports, the memory is formed with that data just missing. I think this is best exemplified by various forms of perceptual neglect. If the left half of sensory information doesn't make it to the HC, then perception is just of the right half.

And the experiencing of hippocampal output is dependent upon downstream nodes, as well. The vividness of recalled memory and/or mental simulations is directly correlated to the amount of cortical activation from the hippocampal 'broadcast'.

Subjective experience, I argue in the paper, is due to the loop between extrinsic neocortex, DMN, and HC. All three networks are completely involved. My only argument about the hippocampus is that it binds together the newsworthy reports from upstream nodes, and unifies them into one master report of a mind+body in the world.

In the paper, I use the metaphor of a 'player piano roll' to illustrate how the hippocampal output triggers neurotypical subjective experience (NSE). A 'player piano' (i.e. self-playing piano) has as its mnemonic device a metal cylinder, with little grooves and bumps on it. This cylinder does not, of course, have the song recorded on it; instead it has instructions for the player piano to re-construct the song.

Same thing with a recalled memory, or with NSE. The output from the HC is just a code of downstream sites to be activated. It is the specific constellation of site activations which I argue gives rise to NSE.

This is actually my major point of disagreement with Ralf-Peter Behrendt, a UK psychiatrist who published a similar model to ours. He asserts that experience occurs within field CA3 of the hippocampus, but that seems frankly implausible to me. It doesn't explain the vividness of experience, and it doesn't explain the anatomical data of activations by the HC output. He and I have been arguing about this for the last 5 years.

My guess is that you and I don't disagree on that much; it's just the nature of internet discussion that makes it sound like we do. Again, that's why I encourage reading the paper, because I can't possibly do justice to the entire argument here, and my word choice in the paper is much more careful.

Thanks again for the discussion!

best,

matt faw

Vahid Rakhshan Also, you mentioned: "HM (who despite damaged hippocampi did have subjective experiences)"

That's interesting. I have scoured the literature on HM and similar patients, and found no references to their subjective experience, one way or the other. My guess had been that the researchers who worked with these patients used standard measures and tests, but did not probe the issue of SE, because they saw no compelling reason to do so. AFAIK, there is currently no standard test or measure for SE, except self-report.

Eleanor Maguire's lab in the UK did probe whether these patients could form vivid mental simulations, and she and her colleague Demis Hassabis found that they could not. But when I interviewed Hassabis in 2013 for my documentary on the subject, he had not yet probed the question of SE with these patients.

As I wrote above, our model is essentially mute on the subject of HM's SE, because it only describes neurotypical SE. I do make an argument in the paper that it's plausible that HM may not have had SE, or at least that, if he did, it should be very different than neurotypical SE, but the model doesn't make any firm predictions about HM, so I remain agnostic.

If you know of good source material that suggests that HM did indeed have subjective experience, I'd love to hear it. It would be very useful to study.

Thanks!

matt faw

Dear Matt Faw

Thanks for your comprehensive replies, as well as your interesting article. I would (hopefully soon) read it and come back to you.