It was the second week of March 2015 and, as happens this time every year, downtown Austin, Texas, was overrun with conference goers glued to their smartphones.

Adam Gazzaley, Daphne Bavelier and Eddie Martucci were seated in a fourth floor lounge at the W Austin Hotel. They had just finished giving a presentation at SXSW Interactive, the first segment of Austin’s annual festival dedicated to technology, film and music, on how video and mobile games can be integrated into healthcare. Like most conference goers, Gazzaley and Bavelier, both cognitive neuroscientists, and Martucci, the CEO of cognitive-therapy tech company Akili Interactive, were excited about mobile technology’s disruptive potential. (Akili is currently working to produce what they hope will be the first FDA-approved video game to be played by prescription.)

But the medical industry moves far slower than the tech industry. For Gazzaley, Bavelier and Martucci, the fizzy immediacy of SXSW made them impatient for change – starting with clinical trials. The policies surrounding biomedical and behavioral clinical trials involving human participants are heavily regulated and slow to change. Launching a test for a new drug is lengthy, costly and often lacking diversity. “The people who surround universities are often pretty similar populations,” said Gazzaley.

What’s more, clinical trials still rely on sometimes outdated metrics. Particularly for pain management and mood disorders such as depression, measuring real-world improvements is difficult because it depends almost entirely on a patient self-reporting, a valuation subject to bias.

“We don’t know how to marry together the rigor that we need to have health and educational tools with the insanely rapid innovation that we see in the tech world,” Gazzaley said. “We have to figure that out.”

A test case

In 2011, a company did attempt to figure that out. Pfizer, one of the largest pharmaceutical companies in the world, threw its resources behind an all-electronic remote trial. While the study tracked the effectiveness of over-active bladder drug Detrol, Pfizer’s actual purpose was to compare the mobile trial data to that from a traditional, 600-patient four-month trial conducted in 2007, in which the drug was compared to a placebo.

Instead of coming into the lab, patients in the electronic trial participated from their homes. Pills were mailed to them. Blood was drawn at nearby clinics or via home visits, and they periodically filled out assessment forms online. It was a leap – one that ultimately failed to land. The trial was discontinued due to a lack of participation. “We shut down the study early, but our takeaway wasn’t that it was a complete failure,” said Craig Lipset, Pfizer’s head of global innovations. According to Lipset, the company has since incorporated facets of the study into its patient management infrastructure, including mobile apps designed to monitor participants’ progress and inform them what to expect on their next visit with a clinician.

Pfizer hasn’t attempted to conduct another remote trial, however. To be fair, neither have many other pharmaceutical companies. “We were really public about what we were doing, about what worked and what didn’t,” Lipset said. But he would not disclose how much the remote trial cost to conduct.

According to Janet Woodcock, director of the Center for Drug Evaluation and Research at the Food and Drug Administration (FDA), regulators want change just as much as pharmaceutical companies. “We’ve long been interested in innovation in the clinical trial setting, because the more expensive, long, arduous and under-enrolled a trial is the harder it is to get the evidence to use the drug properly,” she said.

In 2007, the FDA established the Clinical Trials Transformation Initiative (CTTI) in partnership with Duke University, in part to kickstart mobile clinical trials. The potential benefits are plentiful, starting with cost: the estimated average price to bring a new drug to market is over $1bn.

Dr Martin Landray, professor of medicine and epidemiology at Oxford University and one of the leaders of CTTI’s mobile clinical trial project, estimates that in some cases, shifting to a mobile-based trial may reduce costs by 10-fold. This cost cutting, coupled with the fact that mobile trials’ recruitment efforts are not limited by geography, has the potential to advance our scientific understanding of rare diseases.

The ability to continuously monitor and assess patients over prolonged periods of time in real-world conditions could also lead to better treatments for common diseases. Even incremental improvements in asthma management, which affects more than 22 million people in the US, and heart disease medication, which kills more than 610,000 people every year, could save millions of lives.

The current system for measuring a drug’s effectiveness is flawed, said Landray. Take the six-minute walk test, which records the distance an individual is able to walk over a total of six minutes on a hard, flat surface, a metric frequently used in clinical trials to evaluate patients’ exercise capacity. As a measurement tool, the medical community puts undue faith in it, according to Landray: “It’s not a gold standard, but it happens to be what we’ve used for a long time – what [regulators] are comfortable with. But it’s not necessarily giving us the best information.”

New standards based on wearable data may provide a more accurate reading of patients’ exercise capacity, he said. Activity monitoring via an accelerometer, for example, allows researchers to assess progress continuously and in fluctuating conditions. “Your ability to exercise may vary by day,” Landray said. “If a drug only improves patients’ capacity in the morning, that’s an important thing to find out.”

More data points also allow researchers to do detective work. If a patient wearing an EKG monitor falls, the device will do more than register the event – it can also collect information on how the fall affected blood pressure, heart rate and other variables.

The problem then becomes validating this new data. Until regulators accept new forms of validation gleaned from wearables and remote tools, pharmaceutical companies won’t use them when trying to bring a drug to market. And, according to Woodcock, until pharmaceutical companies begin to experiment with new techniques, regulators can’t approve new surrogate methods.

Building a new framework

Validating new methods “can be expensive and burdensome” according to Pfizer’s Lipset. For drug companies, the safe, economical option is to use existing standards. According to Landray, pharmaceutical companies know that if they demonstrate improvements using the existing in-lab framework, the FDA will accept that as a reasonable outcome.

Lipset, Landray, Woodcock and Akili’s Martucci all agree the way forward involves validating new measurements against existing standards. According to Martucci, if status quo metrics match mobile results, doctors and regulators will start to appreciate that data as being just as valid.

But while remote clinical trials offer important opportunities, they also contain inherent challenges. Remote studies may enable participation from a larger and more diverse group, but the lack of in-person contact with a researcher makes it harder to verify participants’ identities, according to Landray. Essentially, researchers are taking a leap of faith that a depressed 35-year-old who enrolls in an online trial is actually a depressed 35-year-old.

The real deal

Patricia Arean is the lead researcher of the Brighten study, a federally funded mobile clinical trial designed to track how participants who suffer from depression interact with and respond to mental health apps. The 12-week study, currently under review, highlights both the opportunities and challenges in remote research.

Unlike Pfizer, Arean had no issues with initial recruitment. In the first week, 200 participants were screened, accepted and randomized. By comparison, in a standard clinical trial, Arean said the rate is more like two to three people a week.

Altogether, 1,100 participants were accepted. The sample was nationally representative of the US in both ethnicity and income, and included participants from rural states, demographics traditional clinical trials often fail to reach. Sustained engagement proved trickier, however. A large percentage of participants dropped out after the first week. While most of those who made it to the two-week mark completed the entire trial, there was a 40% drop-off rate. (According to Arean, for traditional trials that study depression, the drop-off rate is closer to 30%.)

Surveys from participants after the full 12 weeks revealed they wanted more feedback from the research team and detailed guidance as to how to interpret their own data. In traditional clinical trials on depression, patients typically meet face-to-face with a researcher in the initial stage of the trial, before checking in on a monthly basis. Arean understands the desire for human contact, particularly for problems like depression. “Having another person to support your attempts at recovery is important,” she said.

Still, she sees a future model in which features from traditional clinical trials are spliced with mobile-based, mood-tracking technology. Beside the cost advantage and potential access to larger, more diverse patient populations, remote clinical trials are able to move much more quickly. “We could analyze our data in seven months rather than six years,” said Anean.

Building a scientifically rigorous, regulator-approved consensus on how to interpret mobile and wearable data points will take time, however. At this point, even hybrid approaches aren’t very common, according to the FDA’s Woodcock. “Companies are dipping their toes in the water,” she said. Despite the potential, adoption will likely be slow. “Will this [transition] happen in the next couple years?” said Martucci. “I doubt it.”

But, as private companies like Google and Apple begin to throw their weight behind healthcare initiatives, the infrastructure will eventually change – even if it isn’t as fast as everyone would like.

• This article was amended on 28 January 2016. An earlier version misspelled Dr Martin Landray’s name as Landry.