What does it mean to be conscious? It’s the state of being awake, being aware, being alive. It’s that thing that human beings have that rocks don’t, even though both are composed of the same raw materials. But it’s more than just being alive, isn’t it? Consciousness is our self, our soul. It’s the ghost in the machine, the feeling that we’re more than just our bodies, the conviction that there’s a subjective “I” with thoughts and feelings and memories, and it’s this that makes us unique, makes us us, in fact. But what is it, this consciousness, and where does it come from?

I’m interested in this because I am a writer of fiction. When I write, I’m attempting to render on the page the personal experience of people who don’t exist. The words I choose should create a doorway to another self, they should allow the reader to briefly put their own consciousness to one side and experience the world through that of another.

Yet how to do that? We are muscles and bones and flesh, and advances in biology and neuroscience have led to the difficult but inescapable truth: there is no ghost in the machine. Our consciousness, our memory, our “self” exists only in the brain. Our identity is encoded in its billions of neurons and the infinity of connections between them; we are nothing more than electrical signals zipping through cells. But still the question is the same. How? How does the brain, this three-pound blob of pink mush, contain the richness of a life? And what happens when it malfunctions?

These are among the questions posed by States of Mind, a collection of exhibits, writings and art installations showing at the Wellcome Collection, London. It makes for a fascinating visit. There are pieces examining the injured brain, coma and persistent vegetative state. But for me, given the theme of my first book Before I Go to Sleep, the most interesting part of the exhibition looks at memories, at what they are, how they’re formed, and what happens when they’re destroyed or inaccurate.

Memories define us. They are our autobiography, they form our identity, our sense of who we are. Without them it’s impossible to both make sense of change and to imagine a future in any concrete way. Yet most of us take them for granted. Remembering happens automatically, we think of our memories as a repository, a sort of video recording of our lives to date. We believe that almost any scene is available for recall, to be reviewed and re-experienced, if only we could find it. But how can this be? How can decades of experience be stored in this one organ, and how faithful is the recording?

It’s now believed that memory is not a single process. We have short-term memories that fade within 20 seconds or so, and a long-term store where memories are coded for future recall and accessible months or even years hence. Both result from changes in the neurons of the brain, but in different ways. Short-term memories involve only temporary enhancements in the synaptic connections between cells, whereas long-term memories arise from physical changes in those same connections. When we store a memory long-term, the neurons “wire together”, and the more times we recall a particular memory the stronger is this bond. All this happens in various circuits across the brain, mostly in an area called the medial temporal lobe and focused around a structure known as the hippocampus.

So if our memories – the stories through which we understand ourselves and make sense of existence – are encoded physically in the billions of nerve cells in our brain, what happens when something goes wrong? Amnesia can be the loss of existing memories and/or the inability to form new ones and, perhaps because it cuts to the heart of identity, it has long been of interest to writers, artists and film-makers. Both my own Before I Go to Sleep and the Christopher Nolan film Memento are easy examples, but there are many others – and one, H.M., is being shown as an installation in the exhibition from later this month.

H.M. is the work of Kerry Tribe, a Los Angeles-based visual artist. Its title refers to Henry Molaison, arguably the most famous amnesiac in history. In 1953, at the age of 27, Molaison underwent “frankly experimental” brain surgery in an attempt to alleviate the debilitating epilepsy with which he suffered. Most of the structures in the medial temporal lobes were removed, including the hippocampus. When Henry woke he was confused. He did not recognise his caregivers or remember the routines or layout of the hospital, and after a couple of weeks it was realised that his condition was not improving. Henry had catastrophic amnesia, and it was permanent. For the rest of his life he was unable to retain any new autobiographical memories and retained only vague details of the first 25 years of his life. He died in 2008 having lived permanently in the present for over half a century.

Tribe’s film is not a straightforward telling of Molaison’s story, however. It’s a single film that plays through two adjacent projectors with a 20-second delay between them, so that the viewer sees simultaneous projections of two different parts of the same reel. Much like Memento – also inspired by Molaison and in which the narrative is told backwards, with each scene ending where the previous one began – it recreates the cognitive dissonance of amnesia. The effect is haunting and disconcerting.

Given our shared interest in Molaison’s story – he inspired Before I Go to Sleep, though the book is not about him – I wanted to talk to Tribe about her film. She told me that she has been making work about memory for a number of years, but became interested in Molaison while making Near Miss – a film examining the subjectivity of memories. She says she has always loved work with “a strong conceptual basis … where the form and content dovetail”. When she learned that Molaison had about 20 seconds of recall but nothing else, she says she realised “20 seconds could be the lag between my two projectors, and HM’s experience – or some semblance of it – could be materialised for the viewer, conceptually but also viscerally, sensorially. I wanted to achieve an embodied sense of what memory feels like for the viewer and make a scientifically accurate experimental film.”

She began work on the film in 2006, two years before Molaison’s death, and worked closely with Professor Suzanne Corkin, the neuroscientist who studied and looked after him for over 50 years. “I travelled anywhere I could that I knew had been significant to Henry – his childhood neighbourhood and high school, lakes he may have swum in as a boy, the department where he was studied at MIT – with a Bolex camera. The Bolex is unique in that it is powered by a hand crank and records for about 20 seconds before it stops.” The resulting film is a montage of haunting images, narrated by Corkin and intercut with scenes of Molaison talking to his doctors or undergoing assessment, all with that 20-second delay.

At the end of the film the voiceover tells us that Henry was never filmed, photographed or videotaped. The moment is jarring, as the viewer feels they’ve just spent 15 minutes in his company. I asked Tribe how she made the film. “The re-enactments that feature HM are staged with actors on a set and shot on digital video. Everything said is ‘real’ in the sense that all of HM’s dialogue comes directly from transcripts, and Professor Corkin plays herself. And those funny-looking tests you see HM performing in my film are the real objects he used in testing back in the 1960s. But the hands belong to an actor.” Molaison was in a nursing home by the time she started working on the project, she adds, and no longer verbal. Plus, “his privacy had been carefully guarded for decades and I didn’t want to compromise that. But Darryl Sandeen does a fantastic performance as HM – nearly everyone assumes it’s him.”

The film was first exhibited in 2009, after Henry’s death, and though inspired by his story it asks the viewer to think about memories more generally. Amnesia is a terrifying prospect, but another installation at States of Mind looks not at the lack of memories, but at their inaccuracy. False Memory Archive: Crudely Erased Adults (Lost in the Mall) is a series of red tinted images of a shopping centre, in which every adult figure has been erased leaving only lone and isolated children. It was created by AR Hopwood and refers to an experiment performed in the early 90s by the psychologist Elizabeth Loftus. She found 25% of people would recount in great detail the untrue story of being lost in a shopping mall as a child, a false memory that had been successfully implanted during a previous testing session. Loftus has spent decades studying false memories. “Many people believe that memory works like a recording device,” she said in her TEDGlobal talk in 2013. “(But) memory works more like a Wikipedia page: you can go in there and change it, but so can other people.”

This has frightening implications. Loftus has talked about people who have been found guilty of crimes they didn’t commit on the basis of eyewitness evidence that was later revealed to be a false memory. In one famous example, witnesses to the shooting of Jean Charles de Menezes reported that he leaped over the ticket barrier at Stockwell underground station before running down the escalator, despite CCTV footage showing him passing through the barrier normally. Those studying the phenomenon would suggest that it was probably a police officer who jumped the barrier, yet the charged atmosphere around the shooting, and subsequent events, caused the confused memory to become embedded in the mind of eyewitnesses as truth. I asked Kerry Tribe whether she though memory might be more malleable than we might like to think. “Yeah. There are so many interesting and troubling things we discover when we enter the word of neuropsychology. I recently learned that memories aren’t produced suddenly when something happens. It’s a slow process of solidifying, like pudding, over time. If stress hormones are present during the solidifying process, the strength of the event’s memory is increased, along with the associated feelings of stress.”

So memories are not recordings. They are stored in the brain as patterns of activity, fragile, subjective, and often false. It’s likely that every time we remember something we potentially alter it before recoding the new version, an ongoing process of editing and revision that we’re not even aware we’re doing. It turns out the stories we tell ourselves to make sense of our life might be at least partly fiction after all. So where does that leave the question of identity, consciousness and the self? As Tribe told me: “It never ceases to amaze me, the simple fact that our consciousness is embedded in a physical organ – essentially a piece of meat.” Through studying Henry Molaison we’re now much closer to understanding how the brain forms and stores memories. Yet even that is only part of the story. We’re a long way from understanding the processes by which the billions of neurons in the brain link and combine and weave together to make us who we are, but it’s a fascinating journey.

• Kerry Tribe’s H.M. is at the Wellcome Collection, London, from 26 April-24 July as part of States of Mind: Tracing the Edges of Consciousness.
  SJ Watson’s new novel Second Life is published by Random House.