My lunch is some one-year-out-of-date Marmite peanut butter spread over cracked black pepper Nairn’s oatcakes (best before December 2021). It follows a breakfast of old pink Coco Pops and a mid-morning snack of crisps that the manufacturer suggests I should have eaten when Theresa May was still in Downing Street.

Many people are quite relaxed about food dates, sniffing their milk or inspecting their salad bag rather than relying on what the manufacturer has printed on the packaging. But there is a new breed of bargain hunter: one who actively seeks out food that is not just two days or two weeks out of date, but sometimes two prime ministers beyond its best before. Many do it purely to save money, others to help the planet, quite a few for the strange thrill of hunting down obscure jars of fish paste that would otherwise end up being thrown away. I have decided to join them, if only for a week, to discover why so much food destined for our supermarket shelves never makes it that far – and how retailers specialising in selling “recovered” or “rescued” food are booming. Crucially, I want to find out if any of it tastes OK.

The rules about food labelling are not simple. “Before we started seven or eight years ago, people were scared about best-before dates. To be honest, initially I didn’t really understand the difference: best before, use by, sell by. It’s crazy, really,” says Dan Parslow, 42, who runs Best Before it’s Gone, a website based in Daventry, Northamptonshire.

By law, nearly all food – with the exception of uncut fruit and veg, alcohol, salt, sugar and chewing gum – needs to have a date of some sort. Dairy, meat, fish and ready meals invariably require a “use by” date – if you eat it after this date, there is a risk to your health. But all other food needs only a “best before” date. This means it is completely safe to eat, but the quality may not be as good after that time. There’s nothing stopping a shop selling this food.

It is this category – best before, rather than use by – that Parslow and his fellow retailers specialise in selling. “Back then it was like Marmite,” he says of his business, which started as just a small shop on a housing estate. “I used to have people coming up to me in the pub and saying, ‘You’ve poisoned my kid’ because I’d sold them a pack of crisps that was past its best before.”

Warehouses selling unwanted stock at deep discounts were once the food industry’s dirty secret; supermarkets and manufacturers often used to insist their branding was stripped off packets or jars in case it damaged their reputation, but it is now a fast-growing market, worth anything between £130m and £1bn a year (depending on who you ask), as more people embrace the idea that products deemed not good enough for a supermarket shelf can still be good enough to eat.

• The Daventry warehouse of Best Before it’s Gone, run by Dan Parslow (top). ‘The cost of living crisis is definitely driving this,’ he says of the recent big rise in customer numbers. Photographs: Fabio De Paola/The Guardian

During Covid, Parslow’s business flourished and he now (with the exception of some click-and-collect customers) trades entirely online. As he shows me around his warehouse, I see boxes of Ferrero Rocher chocolates (with one week left to go before their expiration); squeezy bottles of Heinz mustard (June 21); WK Kellogg blueberry, apple and beetroot multigrain shapes (January 21); Tesco dried cranberries (January 22). It is all what the industry calls “ambient” food – room-temperature fare. “When you start getting into anything chilled or frozen, it becomes a minefield. The food hygiene regulations are a completely different kettle of fish,” Parslow says. It is also, for any online or warehouse retailer, far more expensive, requiring them to pay for refrigerated lorries and storage.

This explains why my week-long experiment lacks fruit or vegetables unless they come in a tin, jar or packet. For the first 48 hours, it doesn’t bother me; I have pasta with sauce I make from tinned tomatoes, olives and anchovies, which is delicious. But by the third day I am craving an apple or even a stick of celery. In a bid to get some vitamins into my diet, I resort to opening an enormous 2.62kg can of Country Fresh carrots in water (December 21) and eating the soft, over-blanched veg straight from the tin. They may cost only £1, but they are a poor substitute for the fresh version.

They are, at least, in good nick. “Tinned veg or fruit lasts for ever, really,” says Mark King. He is the co-founder and managing director of Rogers Wholesale Foods. The company runs five shops across England, sited on industrial estates and laid out like very basic cash-and-carry warehouses. There are no shelves; the pallets of fizzy drinks and biscuits are piled on the floor with customers weaving their way around as if in a maze. “In the last five or six weeks, it has really gone crazy,” he says, as the cost of living crisis drives an increase in customers. “So many more people are coming to us now. I think they are looking for alternatives.”

King, 49, is evangelical about how good food can be after it has passed its best-before date. His previous job was a cheese supplier. “It used to really annoy me that vintage cheddar needs to be more than 18 months old. But the moment you put it in a packet, it can only have a three-month date on it, which is ridiculous.”

Most food that is thrown away is done so by consumers. Of the estimated 6.4m tonnes of edible food wasted in the UK every year – the equivalent of 230kg for every household – 70% happens in people’s homes: salad wilting in bags, stale biscuits at the back of the cupboard, uneaten food scraped off plates. However, 16%, or 1m tonnes, is caused by manufacturers and retailers, according to anti-waste campaign group Wrap.

King believes there is little need for this. He insists crisps can last “at least two years” beyond their listed date, thanks to advances in packaging technology that see most of their packets made with a foil lining (first added to Quavers in the 90s) and something called modified atmosphere packaging. This is when nitrogen is pumped into the bag just before sealing, forcing out the oxygen. “When people moan about air in packets, that’s inert gas and it’s extending shelf life,” he says.

* * *

Experts say that advances in plastics technology have also been crucial to extending shelf life. Lynneric Potter, a packaging specialist at Campden BRI, a food science consultancy, says, “The more premium products often have packaging with a number of plastic layers laminated together, which help reduce oxidative rancidity and light-induced rancidity” – that distinctive smell and tang caused by oxygen or light entering the packet and which immediately tells you the food is off.

Even the humble plastic bag inside a cereal box may look unchanged since the 70s but has been improved. Sokhna Gueye is head of packaging at Nestlé UK and Ireland, a company that uses 3.8m tonnes of packaging each year, globally. “You see it as a very simple plastic bag. But in reality, it’s quite complex; it’s a combination of different grades of polyethylene.” A bag that contains Shreddies or Cheerios is made up of usually seven different layers. “Each layer is bringing a specific performance: one would be for the sealing, so you have a tight bag; one would be for the mechanical strength, so it doesn’t burst; one provides a moisture barrier. It’s all those different layers that combine to give us the final properties we need,” Gueye says.

• Customers weave around pallets of food and drink piled on the floor, and staff taste-test products, at Rogers Wholesale Foods in Wolverhampton, run by Mark King (above). Photographs: Fabio De Paola/The Guardian

Considering the high quality of most packaging, King argues, manufacturers are too cautious about best-before dates. Every day, 14m bags of Walkers crisps are made, and nearly all are given a 12-week shelf life (the date always, curiously, falls on a Saturday). PepsiCo, the parent company of Walkers, says it will not supply supermarkets with crisps that have less than eight weeks left. “To minimise wastage, any product below this threshold is offered to specialist discount retailers or to our charity partner, FareShare,” the company says.

Some food technologists are sympathetic about manufacturers being conservative with their dates. Valia Christidou is a fellow at the Institute of Food Science & Technology, and spent 30 years in the food industry, developing new products and testing shelf lives. “There is no such thing as the average human palate,” she says, pointing out that some people can start to taste rancidity weeks before other people will notice a crisp or biscuit has gone off. “Some food brands have been in existence for over 100 years and you can’t play games with them. Believe me, consumers will pick up the phone and complain if they don’t think their Jaffa Cake is perfect.”

* * *

Throughout my week, the quality of the food ranges between excellent and not great. Tic Tacs that are a couple of months out of date are perfect – and being 94.5% sugar, so they should be. Sugar, in theory, can last unchanged for thousands of years. Fusilli pasta from 2020 is good; experts point out any moisture that has evaporated in the box will be regained during cooking. But an Eat Natural protein bar (July 22) is a bit dusty tasting, and some Ferrero Rocher (August 22) have started to bloom – when the cocoa butter separates and leaves the surface white and cloudy.

The texture has deteriorated in a couple of products: a pouch of microwaveable Mediterranean tomato and herb super grains taste fine, but they have formed into a mushy lump. They were, however, well over a year out of date (June 21) and other pouches of grains, including Seeds of Change wholegrain rice with quinoa (April 22) are absolutely fine. The Marmite peanut butter (September 21) is a mouthful of umami wonder, but has separated. Only one product out of the 48 I try is genuinely off: some Kallo wasabi pea and corn snacks dated May 2019. There’s a rancid whiff of stale oil that means I abandon the pack after half a mouthful. But what did I expect? They are 28 months after their best before.

• ‘Only one product out of the 48 I try is genuinely off.’ Photograph: Fabio De Paola/The Guardian

Not all of the products that end up at these specialist retailers are out of date. Some end up there because, in one case, an airline has gone bust and the liquidators have sold all the assets, including the tiny sachets of on-board cheese crackers. There are also lots of varieties of Easter eggs, or cereals, or kombuchas that failed to win over consumers with increasingly fickle taste buds.

All the retailers I visited – in person or online – were selling large quantities of Kellogg’s “limited edition” strawberry and milk chocolate Coco Pops (February 22), a flavour that clearly failed to take off. Most food manufacturers, or distribution companies left with rogue pallets of food, sell off their unwanted produce via a small number of brokers, or “jobbers”. “So they’re taking stuff that’s in date, buying it at a price and spinning it,” Parslow explains. “They will traditionally sell into your Poundlands and 99p stores. But they will always be left with stuff that goes out of date. That’s where we come in.”

* * *

Offloading unwanted stock at a discount is as old as shopkeeping itself. Tesco’s origins stretch back to 1919 when Jack Cohen bought up some battered tins of Nestlé powdered milk, Maconochie’s meat paste and Lyle’s Golden Syrup and sold them on a market stall in the East End of London. Plenty of UK markets, to this day, have stalls specialising in end-of-life products, such as Wellesbourne Market in the West Midlands. Company Shop, based in Barnsley, has been selling supermarket surplus food since the 70s, including fresh and frozen food. It runs 14 outlets. But to buy the products you have to be a member – only frontline or charity workers or those on benefits qualify.

In recent years, a number of online players have emerged, some of whom operate out of little more than a lock-up with a TikTok account, some with serious backers. The newest is Motatos, a Swedish company, founded in 2013, which arrived in the UK over the summer with a flurry of activity on Instagram and plenty of funding from private equity – so far it has raised €130m (£113m) from investors and is valued at €300m (£262m), according to the company.

It started off as a normal, small supermarket just outside Stockholm. But one April a supplier called up the manager saying he had a stock of unsold Apotekarnes Julmust, a Christmas fizzy drink. “It’s a bit weird-tasting but it outsells Coke during the festive season in Sweden,” says Karl Andersson, 45, co-founder of Motatos. The manager agreed to take it for a third of the asking price.

Many customers, it turned out, loved the quirkiness of buying Christmas food in spring. “After two or three months we realised this was actually snowballing and something really exciting was happening. Customers started to go on to Instagram and Facebook to tell their friends that they were saving money on their shopping and saving the world,” Andersson says.

• Customers at Rogers Wholesale Foods, where footfall has increased 28% in recent months. Photographs: Fabio De Paola/The Guardian

It is now an entirely online operation, marketing itself heavily on the idea that by reducing food waste customers are also reducing carbon emissions. This is not strictly accurate, according to Parslow, his rival, who points out that hardly any unwanted food goes to landfill; most goes to animal feed or anaerobic digestion, which produces energy.

That may be true, but Motatos seems to have injected some gaiety into the otherwise worthy business of selling wasted food. “We bring the fun to food shopping. We have a really exciting assortment. You can find a weird marmalade or a muesli with a special kind of flavour or a seasonal product you love but can’t find elsewhere,” Andersson says, with infectious enthusiasm.

Motatos specialises in “flavours not really hitting the market”, as he calls unpopular products. Recent hits have included salted caramel Plant Bars and banana-flavoured Quaker Oat So Simple instant porridge.

Part of the appeal of these products may be their quirkiness, but it is also, of course, their low price. Strawberry Coco Pops may not have been popular with many consumers, but mine cost £1.49 at Best Before it’s Gone, half the £2.99 printed on the original box. Tins of Crosse & Blackwell soup at the Rogers warehouse in Wolverhampton are a third of the price compared with Asda. You can buy four packs of Tesco mince pies for £2; at Tesco they would cost you £5 – if they were in date.

Some customers I speak to in Wolverhampton say every penny matters, especially in a time of surging food inflation and other steeply rising bills. Mandy Kendall has driven 40 minutes from Kings Norton, Birmingham. She’s a full-time mother with five children (three living at home) and says her main source of income is universal credit. It’s her first time at Rogers: “I’m pleasantly surprised by the prices and the quality,” she says. Her weekly £70 shop from Aldi has gone up to £90. “Nearly everything has gone up. You really notice the difference.”

Rogers sells almost everything in bulk – you have to buy three boxes of cereal (£3, compared with £3 for each box at Tesco) and Sacla pesto comes in vast 940g jars originally destined for Israel, but at £1, it’s 14 times cheaper, on a per gram basis, than at Tesco. Having to buy large quantities doesn’t bother Kendall, who has also bought six boxes of Green’s egg custard mix, enough to make 36 custards. She says she enjoys baking and will make the pastry from scratch.

• ‘Strawberry Coco Pops clearly failed to take off.’ Photograph: Fabio De Paola/The Guardian. Set styling: Lee Flude. Grooming: Carol Sullivan at Arlington Artists

Other customers, however, are happy to admit they are far from the poverty line. Antony (who doesn’t want his surname used) is a business consultant shopping with his nine-year-old son. He has bought, among other stuff, 60kg of Golden Lotus Thai red rice in six boxes for £30 – a substantial discount on the £186 it would cost from an oriental supermarket. He is undaunted by the sheer volume of the purchase. “I bought about 100 cases of dried mango two years ago and we’re still eating it,” he says. “It tastes absolutely fantastic – the same as when we bought it.” Has he even tried the Thai rice? “No. I’ll risk it,” he laughs. And where will he store it? “I’ll leave it in the snooker room and eat it over the next couple of years. It is stuffed full of stuff from Rogers.”

Later he sends over a photo of his snooker room, which shows it is indeed stacked high with boxes of soy sauce, bottles of cooking oil, jars of Nutella and what appears to be a whole wheel of parmesan.

Many of the more affluent customers know full well they have an advantage when it comes to bargain hunting. “In order to bulk buy, you’ve got to have enough room,” says Karen Dowd, 58, an art lecturer who is shopping with her husband, Kevin, 70, and who says she has a big enough pantry for her bargains. “Bulk buying doesn’t benefit those at the bottom.”

* * *

Parslow is adamant that the rescued food market is only going to get bigger; in recent weeks they have noticed a big uptick in the number of customers visiting the site – “The cost of living crisis is definitely driving this”

– and they are not the only ones. Company Shop added 250,000 new members in 2021. Rogers Warehouse has seen footfall increase 28% in recent months.

How big could it become? Parslow’s older brother and co-founder, Simon, says: “We’re pretty tiny and we turn over a million quid; we haven’t even scratched the surface yet. It’s a billion-pound market, without a doubt.”

That sounds like a large number: proof that there is a vast amount of wastage within the food chain, much of it caused by us, the consumer, expecting a ceaseless selection of new food products and flavours.

• Crates of beer at Best Before it’s Gone and co-founders Simon (on left) and Dan Parslow. Photographs: Fabio De Paola/The Guardian

I am certainly guilty of this, always excited to try the latest Snickers bar innovation or fancy crisp variety. Which is possibly why I enjoyed lots – but not all – of my week of eating nothing but rescued food. The silly snacks were fun; so, too, the thrill of realising that some food is genuinely perfectly good a year, even 18 months, after its official end of life. Two years? Not so much.

I am dubious about a couple of aspects, however. In total, I spent £70 across three different retailers, a 41% saving on the £120 if all the food was in date. Though that is a hefty discount, some of the products, from the excellent Napolina tomato passata to the Tilda basmati rice (£3 for 1kg), are undoubtedly premium products. If I really wanted cheap rice, I could go to Aldi to buy its basmati for £1.59 or Aldi Everyday Essential long-grain for 45p. These retailers tend to stock far more big-name brands than cheaper supermarket own-brands.

And, after seven days, my enthusiasm for jazzing up yet another bowl of rice and fish paste with some Yo! sriracha mayo starts to pall. The complete lack of fresh fruit, veg and protein means these retailers are just not a viable option for a weekly shop.

But for the occasional store-cupboard stock-up? For half-price porridge, pasta, eccentric chocolate bars and spectacularly crunchy Pringles designed to be eaten while Trump was still in the White House – try and stop me.