It's a common observation, among residents and visitors alike, that to walk through London at rush hour is to encounter a staggeringly large number of rude people, angry people and aggressive people, many of whom are also stupid. The observation is so commonplace, in fact, that it presents a problem: if you've ever had cause to criticise London's obnoxious hordes, you've almost certainly been dismissed by someone else, at some point, as being one of the obnoxious types yourself. You're not, of course. You're a reasonable, warm-spirited person. You sometimes get irritable, after a stressful day, when it's raining and you forgot your umbrella, but who wouldn't? That doesn't make you an unpleasant person. It's others who are irredeemably, intrinsically awful. (See also Sartre's famous remark that "hell is other people, especially the ones who linger pointlessly at the cash machine for 45 seconds after withdrawing their money.")

We think this way because we're hypocrites, certainly - but also thanks to one of the most important phenomena in social psychology, the fundamental attribution error (FAE). In accounting for others' behaviour, we chronically overvalue personality-based explanations, while undervaluing situational ones. "When we see someone else kick a vending machine for no visible reason, we assume they are 'an angry person'," writes Eliezer Yudkowsky at
overcomingbias.com. "But when you kick the machine, it's because the bus was late, your report is overdue, and now the damned machine has eaten your lunch money."

The bias runs deep. Few of us, surely, think of ourselves as having a fixed, monochrome personality: we're happy or sad, stressed or relaxed, depending on circumstances. Yet we stubbornly resist the notion that others might be similarly circumstance-dependent. In a classic 60s study, people were shown two essays, one arguing in favour of Castro's Cuba and one against. Even when it was explained that the authors had been ordered to adopt each position, based on a coin-toss - that their situation, in other words, had forced their hand - readers still concluded that the "pro-Castro" author must be, deep down, pro-Castro, and the other anti. (Intriguingly, this personality-trumps-situation bias seems less prevalent in more collectivist cultures, such as Japan, than individualist ones, such as America.)

Self-help gurus love to dispense personality-based counsel when it comes to others: there's advice for dealing with "toxic people", "energy vampires", the neurotic, negative, self-absorbed. Personality disorders exist, to be sure, but the FAE suggests we should err on the side of cutting people some slack: that almost everyone believes that what they do, at any given moment, is a natural response to their circumstances.

There are broader ramifications, too. The FAE undermines arguments based on the idea of "a few bad apples", whether that's torturers at Abu Ghraib or the asbo-wielding mini-criminals demonised in the Daily Mail. This isn't to excuse bad acts; it's just being clear-eyed. "While a few bad apples might spoil the barrel, a vinegar barrel will always transform sweet cucumbers into sour pickles," notes Philip Zimbardo, whose Stanford Prison Experiment showed how powerful situational influences can be. "So does it make more sense to spend resources to identify, isolate and destroy bad apples, or to understand how vinegar works?"

oliver.burkeman@theguardian.com