For the past two decades, the benefits of high-protein nutritional regimes have been relentlessly marketed to the general public, largely through the booming diet, fitness and protein supplement industries. However, while this has lined corporate pockets – the whey protein supplement industry alone was worth $9.2bn (£6.9bn) in 2015 – scientific research has suggested time and again that it may be harming our health.

Adding to the mound of evidence, a recently published study by researchers at the University of Eastern Finland, who tracked 2,400 middle-aged men over the course of 22 years, reported that a high-protein diet resulted in a 49% greater risk of heart failure. Many large, long-term population studies have also found that people who consume large amounts of protein, especially in the form of red and processed meat, are more likely to be obese or develop type 2 diabetes, cardiovascular disease and colon cancer.

So why have we been persuaded into eating more and more protein? Thomas Sanders, professor of nutrition and dietetics at King’s College London, says the idea of a protein gap in our diet was first broached by a professor at MIT, Nevin Scrimshaw, in the 1960s. He claimed that the protein that comes from plant sources such as vegetables was deficient in vital amino acids and that we therefore needed to eat more animal protein.

“A lot of this work was supported by the food animal industry in the US, which was all for getting people to eat more meat,” Sanders says. “But then it was shown that, by eating a variety of plant-based foods, you can get all the amino acids you need, and the theory was debunked by 1972. More recently, it has been revived by the health food industry, the diet industry and some of the farming lobbies.”

One of the main drivers for increased protein consumption has been the gym culture that took off in the late 1990s, and the accompanying trend for putting on muscle mass. But scientists believe that the idea of requiring additional protein in your diet to build up muscle, either through meat or supplements such as protein shakes, is a myth.

“There are some quite nice trials which now show that giving people extra protein doesn’t actually increase muscle mass,” Sanders says. “What builds up muscle is exercise and load bearing, and the body has ways of conserving its existing protein to do that. If you eat more protein, the body just breaks it down into ammonia and urea and you excrete it.”

In fact, compared to other mammals, humans are actually naturally adapted for a relatively low protein intake, requiring protein to make up just 10% of our daily calorie requirement. This equates to around 50-60 grams for the average person, but the National Diet and Nutritional Survey has found that we typically eat considerably more – in the region of 75-100 grams.

Over the past 50 years, research has consistently found that whenever we tinker with our natural protein needs, it can have adverse consequences, at all phases of our lives. Human breast milk is quite low in protein: when cow’s milk formula was first used to create an artificial replacement for breast milk, the excessive protein content was found to cause accelerated growth rates in early life. This became associated with an increased risk of developing chronic diseases such as cancer in later life, forcing the formula to be adapted to have a lower protein content.

In adults, high intakes of particular protein sources, for example red meats such as lamb, beef and pork, as well as processed or charred meat, have been linked to a variety of chronic illnesses. But while these trends have been known for a long time, scientists have only relatively recently accumulated evidence showing why this is the case.

The link between red or processed meat and heart disease is a particularly complex one, but one clue could be the content of these proteins. Red meat is very high in iron, while processed meats are typically high in salt, both known to be bad for the heart in large concentrations. In addition, excessive protein increases the amount of urea the body produces, putting greater strain on the kidneys. This increases through life, as renal function declines naturally with age. Unsurprisingly, studies have consistently found links between kidney disease and diets high in red meat.

“Chronic renal disease also contributes to cardiovascular problems, particularly heart failure, as the kidneys regulate things like blood pressure,” Sanders says. “I suspect one of the reasons why high protein intake may be linked to heart failure could be related to the kidneys not coping as well.”

Scientists have also developed theories as to why large amounts of red and processed meat can lead to colon cancer, particularly when the meat is overcooked. Chemical reactions between the heat and amino acids in the protein can release a variety of chemical compounds, such as heterocyclic amines and polycyclic aromatic hydrocarbons, that are thought to be carcinogenic. In addition, the high amount of nitrates found in processed foods can cause heavy inflammation in the gut, leading to the accelerated cell division that is characteristic of cancer.

Microbiome research has further suggested that high-protein diets can alter the pH and therefore the natural bacterial flora of the gut, with potential carcinogenic consequences. “If you look at people eating a diet without much meat, they have a totally different bacterial flora to people living on high-meat diets,” Sanders says. “And these bacteria degrade the bile in the gut into secondary bile acids that are thought to promote the growth of tumours.”

But not all protein has been associated with these negative consequences. Protein sourced from poultry, dairy and plants such as beans, peas and nuts, is thought to have a neutral or even beneficial impact on kidney and heart health, provided it is consumed in moderation.

Perhaps one of the biggest problems with high-protein diets is that the excess protein typically indicates an imbalanced diet, as it comes with a deficiency in another crucial food source. “A balanced diet is one that meets all your nutrient requirements and prevents chronic disease,” Sanders says. “High-protein diets are often low in fibre, and we think colorectal cancer and obesity are linked to low intakes of fibre. There has been so much negativity about fat over the years, but you see in the big population studies that weight gain typically occurs when a big proportion of the diet comes from animal protein.”