Waking up each morning, when you have Alzheimer's, is like a scene from an amnesiac horror movie, Andrea Gillies suggests in her powerful and disturbing account of two years spent caring for her mother-in-law, Nancy, in the grip of galloping dementia.

Nancy wakes "to find that she has aged 50 years overnight, that her parents have disappeared, that she doesn't know the woman in the mirror, nor the people who claim to be her husband and children, and has never seen the series of rooms and furnishings that everyone around her claims insistently is her home," Gillies writes in Keeper.

The transition from sanity to Alzheimer's-induced oblivion is like stepping through Alice's looking glass, Gillies explains, and the most distressing period is when you are moving from one side to the other.

"She had one foot through the looking glass and she couldn't make those two worlds – dementia reality, and normal life – gel at all," Gillies says. "It was really quite science fiction in a way. Complete strangers coming into the house and saying 'How are you today? Shall we go and have a walk?' Obviously your reaction is going to be 'What are you talking about? I've no idea who you are.' She would awake in the morning not knowing who the man in the next bed was and she was afraid. Strangers would come into her bedroom and hand her clothes she had never seen before."

The raw horror of Alzheimer's is laid out by Gillies in this sharp examination of the comprehensive disintegration of her mother-in-law's brain and sense of self. "If I had to pick one catch-all descriptor for Nancy's life in the last few years it would be misery. Profound misery, unceasing and insoluble. She knows that something is wrong, very wrong, but what is it?" she writes. "Every day for her is spent in an ongoing quest to put things right."

But this is not another guide to be added to the depressing pile by the bedside for those who are confronting the decline of a relative. It is as much an exploration of memory, its loss and the subsequent erosion of personality, as a chronicle of the destructive chaos that the onset of Alzheimer's unleashes on the extended family. It is also a chronicle of Gillies' own unravelling, as the exhausting, thankless realities of a carer's role overwhelm her.

Her book is both scientific and political: a combination reflected in the two prizes it has won in the last six months – the Wellcome prize, in recognition of its dispassionate analysis of the havoc the disease wreaks on the brain, and the George Orwell prize, registering the author's anger at a medical and social care system that is failing to support sufferers and their carers.

Somehow, despite the territory, Gillies manages to steer the book away from misery lit and beneath the profoundly bleak narrative runs a stream of grim humour. Most powerful, however, is Nancy's own voice, carefully recorded by Gillies in nightly diary entries, a voice that is at times cantankerous, bewildered and defiant. Reading these monologues, we get very close to understanding what it feels like to experience this illness.

"I don't understand at all what is going on here," she says.

"None of it makes sense. What's happening here? How did I get here? What's happening? Please. Please."

Gillies and her husband begin with a very clear moral position on caring for the elderly. Children have a responsibility to look after their parents, she argues. "First the mothers mother and then the mothers are mothered in turn. That's how the world should work. We can't just abandon our mothers. We shouldn't discard the elderly; we should find ways to cherish them."

What makes this book so unexpected is the honesty with which Gillies records the catastrophic consequences of this well-intentioned act. When independent life becomes impossible for Nancy and her husband Morris (both names changed by the author), they move from a flat in Edinburgh to a bungalow near the house in the Scottish highlands where Gillies lives with her husband and children. But it swiftly becomes clear that they cannot look after themselves in their new home, so Gillies and her husband decide to sell up their own family home and buy a big Victorian house on a remote Scottish peninsula, moving their 10-old-year son, Jack, and his two teenage sisters into new schools and creating two separate establishments in one home, connected by a communal kitchen.

There is no self-congratulation about the decision, just a detailed account of how this wholesale ripping up of their own lives to accommodate the older generation sets the family on a course that turns out to be disastrous for everyone, with Nancy and Morris eventually being moved into a care home.

Two years later, Gillies' kind-hearted optimism has dwindled to desperation and she and her husband find they can no longer cope. Nancy has deteriorated from a benignly forgetful doting grandmother to a raging harpy, who slaps her granddaughters, calls her beloved grandson variously a bastard, arsehole and bitch, cannot remember how to use cutlery, prefers not to wash, tucks used bits of loo paper up her sleeve, squats on the floor to pee and hides her own turds behind paperbacks in the bookshelves ("the kind of discovery that's unexpected late at night when you're looking for something to read"). Most critically, she no longer recognises her husband.

Gillies believes now that their determination to do the right thing by Nancy meant in fact that they did the worst possible thing, moving her away from the home she knew well to three new and unfamiliar environments in the space of just a few years.

"In general, when people become carers, they have no idea what they're doing. It's about obligation and love and duty – old-fashioned ideas," she says, over lunch in St Andrews, where she has since moved with her family after surrendering care of Nancy to professionals. "I assumed, rather arrogantly, that a lot of the ill-behaviour associated with dementia – the hostility, the anger, the violence, the unpleasant side – were to do with external factors, because people weren't being looked after well. We thought, if we took Nancy into our home and exposed her to family love 24/7, caring grandchildren, that we would be able to somehow cushion her descent. This was completely misguided."

The decision to sell up and move to the remote peninsular was, Gillies says now, "whimsical". The mortgage was to be partly funded by running the house as a bed and breakfast, a business model that became increasingly unworkable as Nancy's dementia worsened. (On one occasion recounted in the book, Nancy surprises Australian guests over breakfast when she marches through the dining room wearing nothing more than a large pair of lilac briefs; they are too polite to complain.)

"I hadn't thought about the practicalities. I had this rather rosy idea that we would go for walks together on the beach, a three-generational family, that she would be amusingly dotty and that she would benefit from being with the children and this incredible location," Gillies says.

Now she knows that has more in common with the airbrushed Hollywood version of grandmothers that we wish to see. Movie grannies with their tea dresses and crochet cardigans "who are pliant and hygienic, who dispense old-world wisdom to the children of the house, and are amusingly direct" have a lot to answer for, she writes.

"Movie grannies don't refuse point-blank to clean their teeth. They don't yell obscenities at their grandchildren, or tell outsiders they're being kept a prisoner. They never pull down their trousers and touch their toes and ask you if their bottoms are clean, or get sent home early from the Thursday club for disruption."

But Gillies was also poorly advised by medical staff. If anyone were to ask her advice now on how best to care for someone with Alzheimer's, she would recommend that they continue supporting them in the home where they are already living for as long as possible, before moving them to a care home.

"It would have been better to leave them in their flat as long as possible and then make one move, to some kind of Alzheimer's unit. That sounds monstrously selfish – as if I was trying to avoid the caring responsibility, but it would have been much better for Nancy," she says.

Worse than simply destroying their recollections of the past, the memory loss produced by Alzheimer's prevents those who have the disease from making new memories. "She couldn't map her new home. Every morning she woke up thinking she had just arrived and she didn't know where the kettle was or how to get to the bathroom. This made her angry and afraid," Gillies says.

Her own role as a carer is scrutinised without sentimentality. She engages Nancy with gardening projects, cooking activities, encourages her to draw, or help with the cleaning, makes her endless cups of tea, meals, plates of cake, washes her, dresses her, helps her to the loo and encourages her to talk about the past and confront her illness. Initially, this exhaustive care is met with gratitude, but gradually Nancy turns against her, begins mumbling acerbic criticisms into her hand when she sees her, muttering darkly about the cruel people who make her do all the work and just want to steal her money. On one occasion she attacks her with the coal shovel. Gillies charts her own decline into depression, noting how Nancy succeeds "in sucking the optimism out" of her.

The transformation of her mother-in-law's personality sets Gillies on an investigation into the way the brain controls personality, concluding that identity is biological. "Selfhood isn't something separate from your biological self," she writes. "The brain is where your self is made."

Doctors prescribed Nancy drugs designed to stay the loss of memory, but it's not clear that these had much effect. They experimented with sedatives to calm her anxieties, but these left her restless in the night and more disturbed in the days that followed.

Gillies had to explain to her 10-year-old son that the woman walking around in his grandmother's body was "not really your granny any more" but was "someone who has been invaded by this disease". "Those are terrible words to have to say. But that's the only way to explain why a previously doting granny has started to hit her grandson and call him a bastard."

The middle to final stages of dementia are the most distressing, as Nancy's ability to remember her husband flickers. The anterior cingulate is the part of the brain controlling storage and retrieval of long-term memory, Gillies explains. "When it is damaged by dementia, its malfunction means that only parts of the long-term memory are accessible, and these bits can't come together in a meaningful way. Perhaps in terms of emotional impact, it's worse to have parts than none at all. If the anterior cingulate's disabled entirely, the access to the past is completely blocked off. It isn't in Nancy's case. Not yet. Instead, it fluctuates like bad radio reception, going on and off station."

When they tell Nancy that the strange man in the bed next to her is her husband, she tells them: "You're all liars." "You're all mad. You're all mistaken. That isn't Morris. You think I don't know who Morris is? Well, I do and that isn't him."

By the time Gillies and her family decide they are no longer able to cope, Nancy "was engaged daily in a very protracted slow-motion form of panic", a state that was painful for everyone around her, particularly her husband, to witness.

Sensitive to the charge that she is squeezing literary material from family disaster, Gillies makes her position clear in an angry introduction to the book. Dementia is ill-understood and remains so stigmatised that the government fails to treat it with the same urgency as other illnesses such as cancer, she argues. Cancer, because it is something that the young and glamorous can suffer from, commands a greater respect as a condition. Dementia is shabbier, associated with the distasteful process of ageing, something that we would all much rather forget about.

The figures, however, suggest that we shouldn't forget, and this was part of Gillies' motivation for writing the book – a project that distracted her from novel-writing. "There are more than 820,000 people in the UK with dementia, two-thirds of them are women, and the figure is rising sharply," she writes. "For UK citizens reaching 65 this year, the risk of developing dementia is one in three."

Most forcefully, she challenges what she says is a widespread misconception that dementia is a good way to go, a kind state of absentness that blinds the sufferer to the relentless decline towards death. On the contrary, dementia takes away everything, she concludes, "every last thing we reassure ourselves that nothing could take away from us."

Nancy's disease has progressed to a kinder stage, where her forgetfulness is soothingly comprehensive. Her husband is now dead, and she is untroubled by his absence. The family visit quarterly, to reassure themselves that she is being properly cared for, but their visits have no impact on Nancy, who accepts the flowers and chocolates, but has no recollection of ever having met them before.

"People find it hard to grasp that we don't see her more often, they see it as neglect. But it makes literally no difference to her," Gillies says. She still occasionally feels guilty at having abandoned the attempt to care for Nancy, even though she is clear that it was the best solution.

"For the six months before she went to the care home I think she was very unhappy because she was trying to grapple with who we all were. She half-recognised the situation and that was frightening," she says.

Now Nancy is no longer struggling "because she no longer has the remnants of her past to grapple with. It's all gone. All gone. All understanding of her life, all autobiography gone. To use the looking glass metaphor, she is a much happier person now because she is completely through to the other side of the looking glass. She lives in dementia reality and has no connection with the normal world."

• Keeper is published by Short Books. To order a copy for £7.99 (including UK mainland p&p) go to theguardian.com/bookshop or call 0330 333 6846