Posts Tagged ‘Scientific Communication’

Testing DTC Genomic Testing I

Monday, November 14th, 2011

Now that we’ve all gotten over the shock of being able to order a genetic test through Amazon.com, we can begin to actually ask some useful questions about DTC genomic testing and its utility.  Not much is really known about how useful it is.  However, in the tradition of Hippocrates, the first question we must ask is “is it harmful?”

The medical research community was all over the task once the DTC genomic testing services emerged from the intellectual garages of Silicon Valley and elsewhere and hit the streets.  Two studies of note asked the question above, using Navigenics’ service as a model.  The first paper was published by Bloss and colleagues in the New England Journal of Medicine in February of this year.  Just last month, James and colleagues published another study of the effects of DTC genomic testing in the Proceedings of the Mayo Clinic.  Neither paper reported any untoward effects of genomic testing under these circumstances on the test population.

More studies of DTC genomic testing are in the offing, I’m sure, in addition to the commentary and other reports on the technology so far published.  Now might be a good time to take a longer look and ask some of the interesting questions about DTC genomic testing, what it means, and why it created such a commotion.  I am in the process of shifting through many of these reports and opinions and will write another post soon on the subject.

Are most published research findings false?

Friday, October 7th, 2011

Many people are aware of the work of John Ioannidis regarding the analysis of research findings and the conclusions drawn from those analyses.  In particular, these concepts were described by him in a paper published in PLOS Medicine in 2005 is apparently the most downloaded article from that journal.

I’ve had this article on my mental favorites list for some time now.  I am finally putting a few words in print about it mostly to put a stake in the ground on this issue because I believe it is an important one in this era of high volume research reporting.  In short, I agree with the article’s main conclusions, although I might phrase it as “most published biomedical research conclusions are not true”.  This is not to say I think there is some conspiracy or that statistics are useless.  To the contrary:  statistics is an enormously useful field of applied mathematics.  I also think a great deal of very good research is being done in labs and clinics around the world by very dedicated and smart researchers.

My concern over the veracity of biomedical research and how these results are reported stems from the nature of statistical models and test versus how they are interpreted and reported.  Within that discussion is another around the unspoken assumptions underlying both our biological and statistical models.

Perhaps the stickiest issue for me is the use, or misuse, of p values in many published studies.  Without getting too long-winded about it, far too often the p value is used all by itself and given the status of a “stamp of approval”.  Using a p value in isolation (i.e. p=0.001 therefore I won!) is ignoring a lot of important information.  What type of test did you “win”?  What distribution of p values for this test did you assume?  Are your assumptions correct?  Did you keep testing data until you found the p value you were hoping for?

Fortunately, I think the wider scientific community is waking up to the deficiencies in the most commonly used statistical analysis scenarios.  This recent article from Genomeweb does a nice job describing the basic appropriate role for statistical analyses in biomedical research.  An important distinction pointed out in their article is that statistical significance and biological (or clinical) significance are two different things.  When we rely on statistics to identify important relationships within a vast ocean of information, it is all the more important to understand what these mathematical tools are telling us.

As the wise scientist once said, “Never assume anything other than a 4% mortgage.”  I mentioned assumptions above in the sense of statistical models; assumptions also come into play in experimental design.  My sense of it is that these assumptions are usually underappreciated or perhaps even ignored.  The danger, of course, is that incorrect assumptions, statistical or experimental, can invalidate the results and conclusions of any research.  Often these assumptions difficult to verify, which we might be able to cope with, if we knew what these assumptions were.  Unfortunately, they are not part of the standard scientific reporting paradigm.  This recent article in PLoS Computational Biology sheds some light on the issue of reporting experimental assumptions.  Again, by bringing the issue to light there is hope that we can begin to change our science reporting procedures to incorporate some discussion of assumptions.

I find it reassuring that these discussions about accurate analysis and reporting of scientific research are surfacing.  Opening up communication about these critical issues will greatly enhance our ability to navigate through the ocean of biomedical studies available to us.

Genomic Test for your Kid’s Sports Ability? Oh, Please!

Friday, May 27th, 2011

The US FDA just sent out more letters to genomic testing firms asking them to explain why their testing kits should not be regulated.  The companies in question (and their target market) were Lumigenix (disease predisposition), American International Biotechnology Services (AIBiotech, workout optimization for athletes and also disease predisposition), and Precision Quality DNA (PQDNA, disease predisposition and drug response).

Based on the blogosphere reaction, the testing of genomic influences on athletic performance drew the most attention.  I don’t know much about genetic influences on athletic performance, but I don’t think anyone else does either.  Hence the reaction to such a product—is there really any value there?  I already have to submit a copy of my son’s birth certificate to enter him in certain sports tournaments.  Am I also going to have to submit his genetic profile so he can join AYSO?

For all three it appears to me that the FDA was pretty reasonable in exercising its mandate to protect the public health by blocking unreasonable medical claims for products.  It’s unfortunate for those companies that are trying to do the right thing by backing up their genetic testing services with real data; they may well have to carry the burden of federal regulation soon.

 

Failure of the Genome?

Thursday, May 19th, 2011

I recently read an article written by Jonathan Latham in the Guardian (UK) online with the title, “Failure of the Genome” (credit to Genome Web for pointing the post out to their readers).  Following the eye-grabbing headline, the article goes on to posit that the Human Genome Project has not turned up much of use.  Indeed, the author asserts that there is scant evidence supporting the genetic underpinnings of disease paradigm.  To a geneticist or one of the many others who have backed the various genome projects, this is provocative indeed.

So, why did this provocative headline catch my eye?  I have to admit, I am one of those who have become weary of seeing the “Gene For…” headlines ad nauseum over the last couple of decades, only to see those claims vanish into the twilight of yesterday’s news time and again.

The magically vanishing claims of genetic causation that show up daily certainly have jaded even a dedicated molecular biologist like me* to an extent.  As such, I can understand how this pattern of hype of research results followed by disillusion when the claims quietly die would dishearten others who see these stories.

So, maybe what grabbed me was the sense that we, as a society, have swallowed the genetic-cause-of-disease hype hook, line, and sinker—and this article is evidence of rising discontent the emptiness of those hyped promises that come in company and university press releases.  The real story about genes and disease is more complicated than will fit in the easy to digest news bits that are our common food in these information-intense times.

I hope that we scientists will recognize this backsplash as a signal to examine how we communicate our science.  What I most value in the scientific enterprise are the truthfulness and credibility that are a part of this culture.  I hope we will work to preserve these qualities.

 

*I come to this as a molecular biologist with a couple of decades of experience doing research in oncology, in particular, the role of oncogenes in the development and progression of cancer.  Based on my own hands-on experience in trying to understand the role of genes in a disease, I’d be hard pressed to simply dismiss genetic variation as a factor is disease causation, as Mr. Latham does.  It is pretty clear to me that variation in genetic makeup leads to variation in phenotype, some of those phenotypes being what we call “disease”.

 

DTC Genomic Testing: a Window on Society?

Tuesday, April 19th, 2011

I continue to believe that the discussion surrounding the Direct to Consumer (i.e. DTC) genomic testing is basically a healthy thing (pardon the pun).  It seems to me to be a window into the larger conversation about the role of health care in society and evidence that we do pretty well in letting information flow freely.  I think that’s a good sign of a free society.

The DTC genomics testing debate seems to have looped in a group of people that are: a) health conscious, b) sophisticated, and c) medical non-professionals.  These folks are interested in taking advantage of the health empowerment that information technology and genomic technology have provided.  In this endeavor, they are bumping up against the health care system, which is fairly conservative and has its own status quo.  This latter element is certainly something those sophisticates who have been empowered by genomic technology (via DTC genomics companies) are not necessarily interested in minding.  The good news is that the debate around genomics and health care empowerment has been civil so far.  The court in which the debate is being conducted is the halls of the US Food and Drug Administration.

The FDA had a meeting…

In between the summer of 2010 and the US FDA’s meeting in March 2011 to hear feedback regarding DTC testing, Amy McGuire and colleagues published an article in Science proposing a mechanism for regulating these new genetic tests.  Essentially, McGuire et al proposed a “risk-based stratification” of regulation of these tests.  This means that tests thought to carry higher risks of harm will faced tighter regulation.  For example, testing for breast cancer predisposition might be subject to significant oversight because women face the risk of surgery or other significant and potentially harmful healthcare consequences based on the outcome of the test.

On March 8-9, the FDA’s Molecular and Clinical Genetics Panel heard testimony from a variety of stakeholders of DTC testing.  On one side, as one would expect, were the DTC genomics testing providers arguing that these tests should not be regulated out of hand.  In fact, the representatives of these companies seem to be willing to accept the risk-based stratification approach that the FDA appears to be leaning toward.  Their backers further argue that individuals should have access to their own genetic information and that the FDA should not come down hard on a promising new industry.

On the other side were some significant groups, such as the American College of Pathologists, who argued that lax regulation could be harmful to consumers.  The significant risks of DTC genomic testing were pointed out, including privacy concerns and inadequate support for the inevitable health-related questions from the recipients.  These folks also raised other interesting new concerns that need to be vetted.  For example, inappropriate dissemination of genetic information could be harmful, both to the tested individual (through inappropriate use by employers and insurance companies) and to their relatives who share some of their genotype.

Others brought recent information to the FDA panel.  Subsequent to the initial actions by the FDA last summer, a number of studies have been performed to examine the behavioral consequences of DTC testing among consumers.  So far the results have not suggested any systematic negative consequences, such as anxiety, for those who undergo testing.

All told, my sense of the reports from the meetings was that it was a pretty fair exchange of information, which raised legitimate concerns both for and against DTC genomic testing.  Yet, there are a lot of questions still to answer about how these technologies will serve society.

What about prenatal sequencing and other ethical questions

Although I don’t have the answer to this question, the recent paper from Lo et al in which the genome of a fetus was sequenced, raises questions about the appropriateness of massive prenatal testing.  Prenatal testing for even a single gene faces tough scrutiny, so what do we do when we can test for thousands of gene variants?  In this example the testing was for a single gene mutation (beta thalessemia), but the result showed that an entire fetal genome can be sequenced using a blood sample from the mother.  One positive here: this testing approach is non-invasive.  However, one can imagine the potential demand for prenatal sequencing to determine not only disease susceptibility, but also “soft” traits, like presumed intelligence.

Another question that I think may be under appreciated by people in favor of DTC testing (as pointed out by CAP) is the impact of the results on those genetically related to the tested individual.  What if those people, brother and sisters, are not interested in their genotype or would rather not know?  Does one have the right to post their genotype on the internet?

An extension of that idea came to life with the West family, in which each of the four members of the family underwent whole genome sequencing.  Although the genomic sequences of the kids are not public, one of the parents did submit their genome sequence to the NCBI database of genome sequences, thus releasing half of the genetic info of the kids.  Is that a privacy breach?

Another twist to that theme revolves around research on parental choice with respect to DTC testing of their kids.  A report from Tercyak et al in the journal, Pediatrics, suggests that parents who themselves undergo DTC testing are more likely to have their kids tested.  Parents were also more likely to favor testing if they thought their child was at risk or if they had a positive risk-benefit view of DTC testing (duh!).  Given the variability in quality of follow-up with these tests, it seems fair to question the use of DTC genomic tests in children.

Social liberty?

Not to overdo it, but what I find interesting in the DTC genomics debate is the renegotiation of power between health care consumers and health care providers.  It’s a bigger trend than just DTC testing, as evidenced by broader trends in consumerism in medicine.

One of the key arguments heard from those in favor of DTC testing is that they have the right to know information about themselves.  It’s hard to deny the truth in that.  But, there is definitely a balance that must be struck between individual liberties and the welfare of society.  The discussion of what to do with DTC testing is an interesting place to listen to that discussion in a very personal way.

Recent Developments in DTC Genomic Testing

Wednesday, January 5th, 2011

It’s been relatively quiet on the DTC Genomic Testing front since the turmoil over the FDA’s decision to consider regulating DTC genetic/genomic testing last summer died down.  Nevertheless, there have been a few news items that yielded glimpses of what is going on in the industry.

In November 2010, 23andMe received additional investments from Google, New Enterprise Associates, and Johnson and Johnson Development Corporation.  In the last few days, the company announced that it would “experiment” with new pricing models.

In mid-December 2010, the Institute of Medicine (US) announced that it would be researching, then issuing a report on possible ways forward for and possible legitimate uses of predictive ‘omics-based tests.

Also in mid-December, DeCode Genetics published findings in Science Translational Medicine of a study of SNPs related to PSA expression in prostate cancer.  The DeCode group interprets their results as refining the utility of the PSA protein marker for prostate cancer.  In essence, the SNP markers described can be used to individualize PSA test results by predicting whether the individual’s range of PSA expression can be expected to be higher or lower than average.

Cytokinetics ALS Drug “Helps” Patients, but Stock is Down 22%?

Monday, December 13th, 2010

OK, I admit that I am probably too much of an idealist, but today’s news about Cytokinetics and its amyotropic lateral sclerosis (ALS) drug bugs me.

The first story I saw has the headline, “Cytokinetics, Inc. (CYTK) Stock Plunges 21% as of 10:24 AM EST After Phase II Results Released” from MarketWatch.

The next story I saw, in the same news digest, trumpeted “Cytokinetics drug helps ALS patients in small trial” (Reuters).

Is it just me or do these two news items seem incongruent?  It just bugs me.  I know that the perspective of these two news items is different.  In story one, what is likely the case is that investors were looking for a “bigger win” than the company reported (Importantly, no data were reported in either story).

In story two, the company was most likely putting their spin on the results.  I’m sure it is truthful, however, whatever the data were, investors must not have been impressed.

“Companies do what they gotta do to survive” might be the byline for story two.  I understand this, but it also highlights my chief concern with “science by press release”, a practice which is increasingly common these days.  It’s hard to tell what the real story is when the news items you read may have a pretty heavy spin on them.