'Know thyself," said the Oracle at Delphi, which sounds a good prescription for living wisely and well. (A bit vague, sure, but that's oracles for you - always with the vagueness.) It wasn't until the past century that psychologists brought their pesky experiments to bear on the Oracle's advice, whereupon they ran into a problem: we are, it seems, incapable of knowing our own minds. We think we know what's happening inside our brains - what makes us happy, or why we hold certain beliefs - but in truth, we don't: we're strangers to ourselves. Take physical attraction: in one recent experiment, participants were shown two pictures of women and told to pick the more attractive. Then they were given the picture and asked to explain their choice. Yet even when the researchers used sleight of hand, and started surreptitiously handing over the wrong picture, many still tried to justify their "choice". They'd refer, say, to a pretty pair of earrings, even though the woman they originally chose hadn't been wearing any.

It seems so stupidly obvious that we should have a superior grasp of our own minds - we've got the perfect vantage point inside our own skulls, haven't we? - that it's painful to face the reality: our relationship to ourselves is more like that of an amateur psychologist, guessing at what's happening in there, with little more accuracy than if we were guessing about someone else. In 1977, when academics Richard Nisbett and Timothy Wilson trawled the archives, they found this demonstrated in studies time and again. One classic case: stop people in a shopping centre and ask them to choose between four "different" products that are really identical, and they're much more likely to choose the one on the farthest right, or the one they see last. Suggest that such factors influenced their choice, however, and they'll scoff.

Think about this too much and it's liable to get depressing: it sounds like an argument for giving up on managing our internal lives, or making certain choices over other ones in the quest for a fulfilling existence. If we don't know what we want, or why, then why make choices at all? Why not let yourself be buffeted by circumstance, carried along by the current of events, if it's as likely or unlikely as anything else to prove enjoyable?

Or perhaps a healthy appreciation for the limits of self-knowledge - for knowing what we don't know about ourselves, if you'll pardon the Rumsfeldianness of the phrase - isn't such a defeatist attitude. Perhaps it could loosen us up. After all, once our fixed beliefs about what we need in order to be happy (in a relationship, say, or our work) are revealed to have such shaky foundations, isn't that a recipe not for resignation, but for experimentation - for doing things we'd written off as "not really us" and discovering new depths to ourselves in the process? Oscar Wilde, who knew a thing or two about destabilising fixed beliefs, had no truck with the Oracle. "Only the shallow know themselves," he retorted.

oliver.burkeman@theguardian.com