When you walk into a supermarket, what do you see? Walls of highly calorific, intensely processed food, tweaked by chemicals for maximum "mouth feel" and "repeat appeal" (addictiveness). This is what most people in Britain actually eat. Pure science on a plate. The food, in short, that is making the planet fat.

And next to this? Row upon row of low-fat, light, lean, diet, zero, low-carb, low-cal, sugar-free, "healthy" options, marketed to the very people made fat by the previous aisle and now desperate to lose weight. We think of obesity and dieting as polar opposites, but in fact, there is a deep, symbiotic relationship between the two.

In the UK, 60% of us are overweight, yet the "fat" (and I include myself in this category, with a BMI of 27, slap-bang average for the overweight British male) are not lazy and complacent about our condition, but ashamed and desperate to do something about it. Many of those classed as "overweight" are on a near-perpetual diet, and the same even goes for half of the British population, many of whom don't even need to lose an ounce.

When obesity as a global health issue first came on the radar, the food industry sat up and took notice. But not exactly in the way you might imagine. Some of the world's food giants opted to do something both extraordinary and stunningly obvious: they decided to make money from obesity, by buying into the diet industry.

Weight Watchers, created by New York housewife Jean Nidetch in the early 1960s, was bought by Heinz in 1978, who in turn sold the company in 1999 to investment firm Artal for $735m. The next in line was Slimfast, a liquid meal replacement invented by chemist and entrepreneur Danny Abraham, which was bought in 2000 by Unilever, which also owns the Ben & Jerry brand and Wall's sausages. The US diet phenomenon Jenny Craig was bought by Swiss multinational Nestlé, which also sells chocolate and ice-cream. In 2011, Nestlé was listed in Fortune's Global 500 as the world's most profitable company.

These multinationals were easing carefully into a multibillion pound weight-loss market encompassing gyms, home fitness, fad diets and crash diets, and the kind of magazines that feature celebs on yo-yo diets or pushing fitness DVDs promising an "all new you" in just three weeks.

You would think there might be a problem here: the food industry has one ostensible objective – and that's to sell food. But by creating the ultimate oxymoron of diet food – something you eat to lose weight – it squared a seemingly impossible circle. And we bought it. Highly processed diet meals emerged, often with more sugar in them than the originals, but marketed for weight loss, and here is the key get-out clause, "as part of a calorie-controlled diet". You can even buy a diet Black Forest gateau if want.

So what you see when you walk into a supermarket in 2013 is the entire 360 degrees of obesity in a single glance. The whole panorama of fattening you up and slimming you down, owned by conglomerates which have analysed every angle and money-making opportunity. The very food companies charged with making us fat in the first place are now also making money from the obesity epidemic.

How did this happen? Let me sketch two alternative scenarios. This is the first: in the late 1970s, food companies made tasty new food. People started to get fat. By the 1990s, NHS costs related to obesity were ballooning. Government, health experts and, surprisingly, the food industry were brought in to consult on what was to be done. They agreed that the blame lay with the consumer – fat people needed to go on diets and exercise. The plan didn't work. In the 21st century, people are getting fatter than ever.

OK, here's scenario two. Food companies made tasty new food. People started to get fat. By the 1990s, food companies and, more to the point, the pharmaceutical industry, looked at the escalating obesity crisis, and realised there was a huge amount of money to be made.

But, seen purely in terms of profit, the biggest market wasn't just the clinically obese (those people with a BMI of 30-plus), whose condition creates genuine health concerns, but the billions of ordinary people worldwide who are just a little overweight, and do not consider their weight to be a significant health problem.

That was all about to change. A key turning point was 3 June 1997. On this date the World Health Organisation (WHO) convened an expert consultation in Geneva that formed the basis for a report that defined obesity not merely as a coming social catastrophe, but as an "epidemic".

The word "epidemic" is crucial when it comes to making money out of obesity, because once it is an epidemic, it is a medical catastrophe. And if it is medical, someone can supply a "cure".

The author of the report was one of the world's leading obesity experts, Professor Philip James, who, having started out as a doctor, had been one of the first to spot obesity rising in his patients in the mid-1970s. In 1995 he set up a body called the International Obesity Task Force (IOTF), which reported on rising obesity levels across the globe and on health policy proposals for how the problem could be addressed.

It is widely accepted that James put fat on the map, and thus it was appropriate that the IOTF should draft the WHO report of the late 90s that would define global obesity. The report painted an apocalyptic picture of obesity going off the scale across the globe.

The devil was in the detail – and the detail lay in where you drew the line between "normal" and "overweight". Several colleagues questioned the group's decision to lower the cut-off point for being "overweight" – from a BMI of 27 to 25. Overnight, millions of people around the globe would shift from the "normal" to the "overweight" category.

Professor Judith Stern, vice president of the American Obesity Association, was critical, and suspicious. "There are certain risks associated with being obese … but in the 25-to-27 area it's low-risk. When you get over 27 the risk becomes higher. So why would you take a whole category and make this category related to risk when it isn't?"

Why indeed. Why were millions of people previously considered "normal" now overweight? Why were they being tarred with the same brush of mortality, as James's critics would argue, as those who are genuinely obese?

I asked James where the science for moving the cut-off to BMI 25 had come from. He said: "The death rates went up in America at 25 and they went up in Britain at 25 and it all fits the idea that BMI 25 is the reasonable pragmatic cut-off point across the world. So we changed global policy on obesity."

James says he based this hugely significant decision, one that would define our global understanding of obesity, partly on prewar data provided by US insurance company Met Life. But this data remains questionable, according to Joel Guerin, a US author who has examined the work produced by Met Life's chief statistician Louis Dublin.

"It wasn't based on any kind of scientific evidence at all," according to Guerin. "Dublin essentially looked at his data and just arbitrarily decided that he would take the desirable weight for people who were aged 25 and apply it to everyone."

I was interested in who stood to gain from his report and asked James where the funding for the IOTF report came from. "Oh, that's very important. The people who funded the IOTF were drugs companies." And how much was he paid? "They used to give me cheques for about 200,000 a time. And I think I had a million or more." And did they ever ask him to push any specific agenda? "Not at all."

James says he was not influenced by the drug companies that funded his work but there's no doubt that, overnight, his report reclassified millions of people as overweight and massively expanded the customer base for the weight-loss industry.

James rightly points out that he needed the muscle of drugs companies to press home the urgency of the unfolding obesity problem as a global public health issue, but didn't he see the money-making potential for the drug companies in defining obesity as an "epidemic"?

"Oh, let us be very clear," he says. "If you have a drug that drops your weight and doesn't do you any other harm in terms of side-effects, that is a multibillion megabuck drug."

I asked Gustav Ando, a director at IHS Healthcare Group, how important this decision to define obesity as a medical epidemic was for the industry. "It really turned a lot of heads," he said. "Defining it as an epidemic has been hugely important in changing the market perception." The drugs companies could now provide, Ando explained, "the magic bullet".

Paul Campos, a legal expert with a special interest in the politics of obesity, saw the decision to shift the BMI downwards as crucial not just in making a giant new customer base for diet drugs but in stigmatising the overweight. "What had been a relatively minor concern from a public health perspective suddenly was turned into this kind of global panic," he told me. "I think when you look at this issue what you see is a combination of economic interests with cultural prejudice which led to a toxic brew of social panic over weight in our culture."

But guess what? The drugs wheeled out to clean up the "epidemic" didn't turn into the blockbusters the industry had hoped for.

Since the 1950s, the great dirty secret of weight loss was amphetamines, prescribed to millions of British housewives who wanted to lose pounds. In the 1970s, they were banned for being highly addictive and for contributing to heart attacks and strokes. Now drugs were once more on the agenda – in particular, appetite-suppressants called fenfluramines. After trials in Europe, the US drugs giant Wyeth developed Redux, which was approved by the Food and Drug Administration (FDA) in spite of evidence of women developing pulmonary hypertension while taking fenfluramines. Dr Frank Rich, a cardiologist in Chicago, began seeing patients who had taken Redux with the same symptoms. And when one, a woman in Oklahoma City, died, Rich decided to go public, contacting the US news show Today.

"That was filmed in the morning and when I went to my office, within an hour later I got a phone call from a senior executive at Wyeth who saw the Today piece and was very upset. He warned me against ever speaking to the media again about his drug, and said if I did some very bad things would start happening, and hung up the phone."

The Wyeth executive concerned has denied Rich's version of events. But once legal liability cases began, evidence emerged from internal documents that Wyeth knew of far more cases of pulmonary hypertension than had been declared either to the FDA or to patients. Redux was taken off the market and Wyeth set aside $21.1bn for compensation. The company has always denied responsibility.

But with Wyeth out of the game, obesity was now an open door for other drugs companies.

British giant GlaxoSmithKline (GSK) found its antidepressant Wellbutrin had a handy side effect – it made people lose weight. Blair Hamrick was a sales rep for the company in the US tasked with getting doctors to prescribe the drug for weight loss as well as depression, a move that would considerably widen its market and profitability. In the trade, this is called "off-labelling".

"If a doctor writes a prescription, that's his prerogative, but for me to go in and sell it off label, for weight loss, is inappropriate," says Hamrick. "It's more than inappropriate – it's illegal; people's lives are at stake."

GSK spent millions bribing doctors to prescribe Wellbutrin as a diet drug, but when Hamrick and others blew the whistle on conduct relating to Wellbutrin and two other drugs, the company was prosecuted in the US and agreed to a fine of $3bn, the largest healthcare fraud settlement in US history.

Drug companies had attempted to capitalise on obesity, but their fingers got burnt.

Still, there was a winner: the food industry. By creating diet lines for the larger market of the slightly overweight, not just the clinically obese, it had hit on an apparently limitless pot of gold.

There now exist two clear and separate markets. One is the overweight, many of whom go on endless diets, losing and then regaining the weight, and providing a constant revenue stream for the both the food industry and the diet industry throughout their adult lives. (As former finance director of Weight Watchers, Richard Samber, put it to me – "It's successful because the 84% [who can't keep the weight off] keep coming back. That's where your business comes from.") The other market is the genuinely obese, who are being cut adrift from society, having been failed by health initiative after health initiative from government.

As Dr Kelly Brownell, director of the Rudd Centre for food policy and obesity at Yale University, explained, the analogy must now be with smoking and lung cancer: "There's a very clear tobacco industry playbook, and if you put it next to what the food companies are doing now, it looks pretty similar. Distort the science, say that your products aren't causing harm when you know they are."

But the solution to obesity could also follow the cigarette trajectory too, according to Brownell. It was only after a combination of heavy taxation (price), heavy legislation (banning smoking in public places), and heavy propaganda (warnings on packets; an effective, sustained anti-smoking advertising campaign; and most crucially, education in schools) was brought to bear on a resistant tobacco industry that smoking became a pariah activity for a new generation of potential consumers, and real, lasting change took place. Similar measures, Brownell says, could provide an answer to obesity.

And it's funny, that analogy with smoking. Because deep in the archive at San Francisco University is a confidential memo written by an executive at the tobacco giant Philip Morris in the late 1990s, just as the WHO was defining obesity as a coming epidemic, advising the food giant Kraft on strategies to employ when it started being criticised for creating obesity.

Titled "Lessons Learnt From the Tobacco Wars", it makes fascinating reading. The memo explains that just as consumers now blame cigarette companies for lung cancer, so they will end up blaming food companies for obesity, unless a panoply of defensive strategies are put into action. You might conclude that there was a good reason why the food industry bought into dieting – it was nothing personal, it was just business.

• Jacques Peretti presents The Men Who Made Us Thin, 9pm, BBC2, Thursday 8 August.