Investors in early stage companies must often take a leap of faith that the academic research behind a company's strategy is well validated and can be reproduced by independent hands. But according to Atlas Venture partner Bruce Booth, most academic studies are not conducted with sufficient rigor to warrant such a leap.

"Validation is a huge challenge to any translational effort," Booth told SciBX. "The problem is not being able to reproduce academic results" outside the original laboratory.

Booth said that the "unspoken rule" among early stage VCs is that at least 50% of published studies, even those in top-tier academic journals, "can't be repeated with the same conclusions by an industrial lab."

He said a nightmare scenario for investors is for strong early results and excessive optimism by founders and technology transfer officers to lead to a multimillion-dollar series A funding for a company whose subsequent work is predicated on erroneous data.

As a result, Atlas now insists on external validation studies of a new company's basic science as a precondition to further investment.

Other investors polled by SciBX said they use more subtle tactics for validating early stage work.

Wet diligence

According to Booth, one problem facing early investors is the different standards of data quality between bleeding-edge academic research and the sort of robust assays needed for drug development in industry.

He also noted that complicated experimental systems familiar to only a handful of highly specialized academic researchers often prove difficult to reconstruct outside of their home laboratory.

Thus, Booth thinks NIH's new translational research center and university technology transfer offices should do more to vet the quality of the translational data being pitched to investors.

"NIH has set up a drug discovery center, but they could also facilitate a broader industry ecosystem for validation," he said.

Booth noted that because validation studies fall outside of NIH's emphasis on original research, "a separate granting process" may be needed to pay academic researchers to confirm the work of others.

Likewise, "technology transfer offices that have 'birdseed' funds could really create a lot of value for themselves and investors" by spending some of this discretionary money on confirmatory studies, Booth said.

For would-be investors, Booth thinks there are two main ways to decrease the risk of irreproducible results.

One approach is to wait for other academic laboratories to confirm or extend the findings that would underlie a prospective company's technology.

This clearly takes time, but fear of missing an opportunity should not trump the need for caution, said Booth. In today's VC funding market, he said, "there's such a scarcity of venture firms willing to take on early work, you rarely find yourself having to jump in quickly."

Booth said Atlas takes a second, more proactive approach. The firm makes relatively small seed investments-typically under half a million dollars-specifically earmarked for validation studies performed by external CROs.

Such a 'wet diligence' validation study costs from $50,000-$100,000 on the low end to $500,000 on the high end, he said. If the results look good, this triggers "a proper multi-tranche series A round," Booth said.

As an example, he cited his investment in Zafgen Inc.

"The first tranche for Zafgen was half a million dollars to validate its obesity results," said Booth. "The subsequent A round was $2 million in tranches of half a million, each one to address the next set of derisking experiments."

Zafgen's ZGN-433, a methionine aminopeptidase 2 (MetAP2)
inhibitor, is expected to enter Phase II testing for obesity this year.

Stepping forward

Camille Samuels, a managing director at Versant Ventures, cautioned that putting researchers on the spot to prove their data could set a bad tone for subsequent relations with a company's founders.

"I think the best way to prevent yourself from funding biotechs that have a faulty scientific basis is to develop a trusting relationship with the scientific founders," she told SciBX. "I think that starting a productive, long-term business relationship is hard to do if you use a 'guilty before proven innocent' approach."

Samuels favors vetting the science with a top-notch scientific advisory team before launching a company.

"If you hire great scientists to the company you will uncover the 'overreaching' before you've spent any real money," she noted.

Daphne Zohar, cofounder and managing partner of PureTech Ventures, chooses a middle route. In PureTech's portfolio companies, validation studies are built into follow-up experiments that move the work forward.

She agreed with Booth that investors "shouldn't just take something and advance it" into clinical development. "One of the first investments should be to validate the work," she said.

PureTech's strategy is to set up under-the-radar seed-stage companies to advance the science to preclinical proof of concept and only then open the companies up to further investment.

"These companies are physically co-located with us and managed by us, with several dedicated team members per project," she said. "We are running various experiments in a combination of third-party labs, including contract research organizations."

Zohar noted that rather than repeating published experiments per se, PureTech's incubated companies try to do the so-called killer experiment, which is usually a demonstration of therapeutic relevance whose success is predicated on the validity of previous data.

"The killer experiment can increase the value of the company" in subsequent funding rounds or partnering agreements with established companies, said Zohar.

Osherovich, L. SciBX 4(15); doi:10.1038/scibx.2011.416
Published online April 14, 2011


      Atlas Venture, Cambridge, Mass.

      National Institutes of Health, Bethesda, Md.

      PureTech Ventures, Boston, Mass.

      Versant Ventures, Menlo Park, Calif.

      Zafgen Inc., Cambridge, Mass.