Waste Not, Want Not: Why Rarefying Microbiome Data is not an optimal normalization procedure

57 mins 14 secs, 218.87 MB, iPod Video 480x270, 29.97 fps, 44100 Hz, 522.12 kbits/sec

Share this media item:

Embed this media item:

<iframe width="_width_" height="_height_" src="https://sms.cam.ac.uk/media/1684613/embed" frameborder="0" scrolling="no" allowfullscreen></iframe>

Choose size:

About this item

Available Formats

About this item

Description:	Holmes, S (Stanford University) Friday 28 March 2014, 11:45-12:30


Created:	2014-03-31 15:15
Collection:	Mathematical, Statistical and Computational Aspects of the New Science of Metagenomics
Publisher:	Isaac Newton Institute
Copyright:	Holmes, S
Language:	eng (English)
Distribution:	World (downloadable)
Explicit content:	No
Aspect Ratio:	16:9
Screencast:	No
Bumper:	UCS Default
Trailer:	UCS Default


Abstract:	Co-author: Paul Joey McMurdie (Stanford University) The interpretation of metagenomic count data originating from the current generation of DNA sequencing platforms requires special attention. In particular, the per-sample library sizes often vary by orders of magnitude from the same sequencing run, and the counts are overdispersed relative to a simple Poisson model. These challenges can be addressed using an appropriate mixture model that simultaneously accounts for library size differences and biological variability. This approach is already well-characterized and implemented for RNA-Seq data in R packages such as edgeR and DESeq. We use statistical theory, extensive simulations, and empirical data to show that variance stabilizing normalization using a mixture model like the negative binomial is appropriate for microbiome count data. In simulations detecting differential abundance, normalization procedures based on a Gamma-Poisson mixture model provided systematic improvement in performance over crude proportions or rarefied counts -- both of which led to a high rate of false positives. In simulations evaluating clustering accuracy, we found that the rarefying procedure discarded samples that were nevertheless accurately clustered by alternative methods, and that the choice of minimum library size threshold was critical in some settings, but with an optimum that is unknown in practice. Techniques that use variance stabilizing transformations by modeling microbiome count data with a mixture distribution, such as those implemented in edgeR and DESeq, substantially improved upon techniques that attempt t o normalize by rarefying or crude proportions. Based on these results and well-established statistical theory, we advocate that investigators avoid rarefying altogether. We have provided microbiome-specific extensions to these tools in the R package, phyloseq. Related Links: http://arxiv.org/abs/1310.0424 - Arxiv Version of Paper. http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0061217 - Phyloseq Package Description and Philosophy

Abstract:

Co-author: Paul Joey McMurdie (Stanford University)

The interpretation of metagenomic count data originating from the current generation of DNA sequencing platforms requires special attention. In particular, the per-sample library sizes often vary by orders of magnitude from the same sequencing run, and the counts are overdispersed relative to a simple Poisson model. These challenges can be addressed using an appropriate mixture model that simultaneously accounts for library size differences and biological variability. This approach is already well-characterized and implemented for RNA-Seq data in R packages such as edgeR and DESeq. We use statistical theory, extensive simulations, and empirical data to show that variance stabilizing normalization using a mixture model like the negative binomial is appropriate for microbiome count data. In simulations detecting differential abundance, normalization procedures based on a Gamma-Poisson mixture model provided systematic improvement in performance over crude proportions or rarefied counts -- both of which led to a high rate of false positives. In simulations evaluating clustering accuracy, we found that the rarefying procedure discarded samples that were nevertheless accurately clustered by alternative methods, and that the choice of minimum library size threshold was critical in some settings, but with an optimum that is unknown in practice. Techniques that use variance stabilizing transformations by modeling microbiome count data with a mixture distribution, such as those implemented in edgeR and DESeq, substantially improved upon techniques that attempt t o normalize by rarefying or crude proportions. Based on these results and well-established statistical theory, we advocate that investigators avoid rarefying altogether. We have provided microbiome-specific extensions to these tools in the R package, phyloseq.

Related Links:

http://arxiv.org/abs/1310.0424 - Arxiv Version of Paper.

http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0061217 - Phyloseq Package Description and Philosophy

Available Formats

Format	Quality	Bitrate	Size
MPEG-4 Video	640x360	1.94 Mbits/sec	833.33 MB	View	Download
WebM	640x360	601.1 kbits/sec	251.98 MB	View	Download
iPod Video *	480x270	522.12 kbits/sec	218.87 MB	View	Download
MP3	44100 Hz	249.77 kbits/sec	104.79 MB	Listen	Download
Auto	(Allows browser to choose a format it supports)

Streaming Media Service Upload

Waste Not, Want Not: Why Rarefying Microbiome Data is not an optimal normalization procedure