Four days a week, the psychoanalyst and writer Adam Phillips listens to his patients' problems in the light-filled and book-stuffed front room of a top-floor flat overlooking a row of Notting Hill boutiques. Each of them comes up the staircase once or twice a week and stays for 50 minutes, sitting or lying on the leather couch from where Phillips can survey them and offer his interpretations of what they say.
Although something of an outsider in the world of contemporary psychoanalysis, and often sharply critical of the mainstream, Phillips believes the basic rules established by Freud and his followers more than 100 years ago are "good and right". So he follows them: "sessions have a set time, you don't socialise with patients, it's entirely spoken. I believe all the basics. I think the method of psychoanalysis is inspiring and inspired."
On Wednesdays, in a smaller room at the back, he writes books, the latest of which is published next week. Missing Out: In Praise of the Unlived Life is Phillips's 17th book – he has edited many others, including a new set of translations of Freud – and is a characteristic blend of literary criticism and philosophical reflection packaged around a central idea. The theme here is missed opportunities, roads not taken, alternative versions of our lives and ourselves, all of which, Phillips argues, exert a powerful hold over our imaginations. Using a series of examples and close readings of authors including Philip Larkin and Shakespeare, the book suggests that a broader understanding of life's inevitable disappointments and thwarted desires can enable us to live fuller, richer lives. Good things come to those who wait.
Does he see himself as a champion of frustration? "I'm not on the side of frustration exactly, so much as the idea that one has to be able to bear frustration in order for satisfaction to be realistic. I'm interested in how the culture of consumer capitalism depends on the idea that we can't bear frustration, so that every time we feel a bit restless or bored or irritable, we eat, say, or we shop.
"It's only in an initial state of privation that you can begin to have thoughts about what it is you might want, to really imagine or picture it. It's very difficult to know what we're frustrated by. In making the case for frustration I want to make it more interesting, such that people can talk or think about it in different ways."
For Freud, the primary source of frustration from which later development stem is the child's thwarted desire to have sex with its parents. Later child analysts shifted their attention to the infant's earliest experiences of its mother, to whom it looked for comfort and food. Phillips's account draws on both these streams of thought, and on the work of the British paediatrician Donald Winnicott, his favourite psychoanalytic writer. In the 1940s it was Winnicott who began to describe the "good-enough mother" as the one who fulfils her baby's needs to the extent that the baby is able to cope when she doesn't.
"To fall in love is to be reminded of a frustration that you didn't you know had (of one's formative frustrations, and of one's attempted self-cures for them)," Phillips writes. "You wanted someone, you felt deprived of something, and then it seems to be there. And what is renewed in that experience is an intensity of frustration, and an intensity of satisfaction. It is as if, oddly, you were waiting for someone but you didn't know who they were until they arrived."
Phillips does not believe in including clinical examples in his books, believing that such conversations should remain entirely private. So case histories are few and far between. One or two are cited, such as the art teacher in a school for maladjusted children (this was the 1970s) who came and sat in the therapy room in silence for 10 minutes before saying: "If you look after me, who will look after you?"
But most of Phillips's observations take more abstract form. Either he writes about literature, about ideas expressed by literary critics or other psychoanalysts, or he addresses the reader directly, in the second person ("you wanted someone", "you were waiting for someone") and offers thoughtful generalisations about experiences he believes are common to many of us. If you recognise the experiences he is describing, or something he says has some bearing on your own emotional state, then reading Phillips can feel like an approximation of what it might be like to be in analysis. Is this, then, a form of self-help? "I'm not at all averse to self-help," he says. "I just think there are certain things a self-help book can't do for you, but anything it can do is good. For me, all literature is self-help – or that's one of the things it is – so I don't at all mind if my books are read like that."
What a self-help book can't do is listen or talk back. And more than any particular theories psychoanalysts have about relationships, about sex or anxiety or anything else that troubles us, it is the idea of psychoanalysis as a series of exploratory conversations that Phillips has always been keen to promote. He wrote his first book for Faber, the teasingly titled On Kissing, Tickling and Being Bored, almost 20 years ago, against a backdrop of challenges to Freudianism from two directions. Prozac had arrived in the US in the late 1980s, and the new fashion in medical psychiatry was for drug rather than talking treatments, as mental illnesses were reconceived as diseases of the brain. At the same time a culture war was under way as a group dominated by the literary scholar Frederick Crews attacked Freud's character and reputation, highlighting his personal drug use and perceived ethical shortcomings as evidence that the whole psychoanalytic edifice was a sham.
Phillips's response was arch, witty and equivocal. Yes, it's true that we mustn't put analysts on a pedestal, he agreed. But don't get so worked up – maybe analysis just isn't for you. Two decades on, his position hasn't really changed: "I would say to people, if you're curious, try it out. There are plenty of other treatments in the culture and something else may work for you, it may be aromatherapy, but this is what psychoanalysis is like, give it a go. But that's all. To make too much of a case for it beforehand is to make a false promise. Some people find it wonderful. Some people find it absolutely pointless. Some people find it exploitative. It's only for the people who are moved by it, amused by it, interested by it, comforted by it. People who find it fraudulent, diminishing and absurd shouldn't do it. That's fine."
He discovered psychoanalysis as an adolescent, when he picked up Winnicott's Playing and Reality in a bookshop in Bristol, where he went to Clifton College. An enthusiast for tropical birds as a boy in Cardiff, he took a biology A level. Put off by scientific language and drawn to poetry and philosophy, however, he ended up reading English at Oxford. He began his psychoanalytic training in his 20s, undergoing four years of analysis with Masud Khan, who had himself been analysed by Winnicott (in his later career Khan was a highly controversial figure, who had inappropriate relationships with patients and a problem with alcohol). Phillips began at the Institute of Child Psychology, and then at the Hampstead and Tavistock clinics – preserves of the two rival schools of child analysis led by Freud's daughter Anna and Melanie Klein – while working in the child psychiatry department of the Middlesex hospital. For 17 years he worked in the NHS, finally losing patience with a structure that offered less and less space. "When I started seeing children I could see them for as long as the treatment took. By the end people were saying, 'we'll pay for three sessions'. For me the project was not to meet the criteria but to ditch the criteria and do something else." He went into private practice.
As a young man Phillips was a great Darwin enthusiast – he first read him as a teenager – and when I ask what he feels he has missed out on in life – having just written a whole book on the subject – he says the thing he regrets most is not having made a trip to the Galápagos Islands after leaving university: "That was the place I wanted to go, to see the backdrop for The Origin of Species, before it became a theme park … It feels to me as though Darwin told us a very important truth about who we are." Darwin's Worms (1999), a book that is more direct and solemn than most of Phillips's writing and is among his favourites, juxtaposed essays on Darwin and Freud on the theme of death, twinning the two men as pioneers of a godless new world. In an elegant close reading of Darwin's final book on earthworms, alongside family letters, Phillips highlighted what the great evolutionist called his "slow & silent side attacks" on sceptics.
But he is dismissive of efforts to develop an evidential basis for psychoanalytic treatments, though most argue this is necessary if they are to remain widely available. "I am personally not at all interested in research," he says. "I think there is something compliant and servile about believing you have to meet the dominant criteria, and I don't think psychoanalysts should have bought into the scientific model with such eagerness. I don't think psychoanalysis is a science or should aspire to be one. I don't think it should be a deliberately misleading mystification either, but I don't think these kind of empirical criteria are the only criteria of value. Nobody is going to do empirical research on Wallace Stevens."
His more recent work has turned away from biology towards literature, and once a term he makes a visit to York University's English department. He is a supple critic, adept at etymology (no one is more alert to hidden meanings than a psychoanalyst) and at drawing new sense from often-familiar works. In Missing Out he looks at Larkin as well as King Lear and Othello, picking a thought-provoking path through "This Be the Verse" (better known by its first line, "They fuck you up, your mum and dad") and drawing too on the work of celebrated critics.
"Well-written close reading is one of the most interesting things going," he says. But in his view literary criticism, like psychoanalysis, took a wrong turn. "It's very strange that psychoanalysis is really about the most ordinary things in the world, but it's written about in a very boring or esoteric way. I don't think psychoanalysis should be a mandarin or elitist form of intelligence. Anybody who has worked in a child guidance clinic knows that all sorts of people across the generations and classes can use it, it's not at all a university-educated middle-class thing, and I think ordinary language is perfectly adequate."
Phillips says frankly that he "didn't understand" some of the highly theoretical post-structuralist criticism that was around when he started out. But in a sense this turn to a highly specialised and technical language by academics suited him well, because it created a niche for a talented writer with a broad knowledge of human nature and culture and a wish to write about them. Today he is one of a tiny handful of psychoanalysts with a trade publisher – his new book is dedicated to his editor, Simon Prosser – and a media profile. For many years he was in a relationship with another intellectual star, Jacqueline Rose, with whom he has an adopted Chinese daughter. He has two younger children with his current partner, the fashion curator Judith Clark, and is a frequent contributor to the London Review of Books.
Phillips says he isn't lonely, but he is intellectually isolated. Psychoanalytic books cannot bounce and spark off each other in the way that, say, books about Darwinism or the environment do. When I ask whether one of the insights in his new book is original, he seems unsure: "It's very hard to know what one's ideas are. I'm not sure I've read it, but I could have. I really don't know."
For him, psychoanalysis is a set of stories that we tell ourselves and each other, a way of redescribing our experiences. "To begin with, one needs to understand," he says, "but I think the final project is to relieve oneself of the need for self-knowledge. It's not that it's useless – in some areas of life it's very useful – but there are lots of areas in which it isn't, and in some areas it's actually pre-emptive and defensive, and this is where psychoanalysis potentially fails people, by assuming there is an infinite project and that the best thing you can do in life is to know yourself. Well, I don't think that's true."
His new book builds a case for the the limits of self-knowledge, and argues that as adults we face a choice between being understood, as we hoped to be as children, and fully exploring other aspects of ourselves, including our sexuality. Provocatively, he suggests that "the wish to be understood may be our most vengeful demand, may be the way we hang on, as adults, to our grudge against our mothers; the way we never let our mothers off the hook for not meeting our every need."
Impatient with the Kleinian conviction that our innate aggression and envy (initially directed at our mothers, and their breasts) are forces of destruction we must guard against, he speaks instead of "energies that are paradoxical. They can be destructive and they can not be. Sometimes they can be both at the same time. I think we should stick with that rather than be too certain about the difference between good and bad. The acid test, when you meet analysts, is are they people you would like to be like? Do they represent some version of a good or interesting life? Do they strike you as having whatever your values are? Are they kind, are they intelligent, are they funny?"
Phillips clearly hopes he is all these things, as do most of the rest of us – and this, he believes, is what we miss out on if we close our minds to psychoanalysis in all its strangeness: it makes us more interesting people.
He feels that psychoanalysis has lost the cachet it once had. "I started writing at a time when psychoanalysis was still interesting, just about, but in my lifetime it's gradually fallen off the cultural map, for all sorts of reasons." He doesn't believe, however, that this is the end of the story: "There's the science behind it, the neuro-anatomy and so on, that I think people are just beginning to get a sense of. There are proliferating other sorts of analysis – it's never been more pluralistic and that's a good thing. I don't think it's remotely all over. Psychoanalysis is just beginning."