I don't think that there is any priority. It does not really matter how you define the selection criteria as long as the criteria are cearly communicated. The selection based on the FDR could have some advantage if *all* the selected candidates are to be further analyzed and if there is some cost or utiliy function of the further analysis so that some sensible FDR value could be selected. I don't think that this is anywhere the case in academic research. Using a (high) LFC as selection criterion has the advantage of giving a list of genes with a rather large degree of differential expression, what may be preferrable genes which DE is simpler to analyze with other methods.

I think that such high-throughput technologies give you a global picture and it makes little sense to concentrate on some (neccesarily arbitrary) part like lists of some (neccesarily arbitrarily selected) "candidates". The data as such is (should be) available through some (preferrably public) database anyway, so showing long list in papers is not really required. However, it can be informative to show/list some "interesting candidates", which may be genes with very strong DE or genes that have a very small variance (large signal-to-noise ratio, large t-values, small p-values). Information about the relevance of the gene function in the experimental context may also play a role in the decision which genes to show.

I am curious how other people think about this. So far I have only encountered people how had no opinion and were only looking around what others do, without understanding the pros and cons of their approaches.

I personally prefer ranking genes by the (corrected) p-value for differential expression and then applying a fold change cut-off if appropriate. The p-value already takes into account the fold change - in order for a gene to be differentially expressed at a statistically significant level, it needs to be quantitatively different between the 2 biological groups and relatively consistent within each group of replicates.Therefore a higher fold change at the same standard deviation between replicates will lead to a lower p-value.

You can then apply the fold change cut-off to remove genes which achieve high significance through being very consistent between replicates but not too different in an absolute sense. The alternative way, i.e. ranking by fold change and then filtering by p-value, has the disadvantage of only ranking by a single parameter, and also fails to properly rank genes which have zero expression in group 1 and a non-zero expression in group 2 - the fold change in this case would be theoretically infinite. In contrast, a p-value will give you a more tangible quantification of how significant such differential expression event is.

I agree with Jochen Wilhelm. It depends on what you what to show and, essentially, if you are clearly communicating your methods, every method is fine. Personally, if I have to select a limited number of genes, I filter them by log2 fold change AND p-value (0.05).  

I have done DEG expression analysis from RNA seq data and I find it better to rank them by corrected p value. Sometimes I was unable to validate DEGs with qPCR even if fold change was as high as 10 and uncorrected p value below 0.05. 

Just a note: the correction of p values is a monotone transformation, to this won't change the order/ranks of the p-values. It thus does not matter if the list is sorted by the "raw" p-values (p) or by some corrected p-values (p*). It will only make a difference (for sure!) if the list is cut depending on the size of p or p*. The number of genes with p < α is always (much) larger* than the number of genes with p* < α (for any 0

I agree with the others above that corrected p-value is the only filter you need. An arbitrary cutoff of FC will help you when designing confirmation experiments based on other methods less sensitive/more noisy in measuring the difference. RT-qPCR, northern and nuclease protection fall into this category. Be aware though that RNA-seq can report generally abundant transcripts as highly significant DEG (low p-value) with quite small FC (

I agree with Philip saying corrected p-value parameter in this kind of analysis represents the right needed filter. The same parameters can be use when you compare differentially expressed genes from different gene expression platforms. Due to their different sensitivity fold change is not a suitable parameter comparing gene expression among different gene expression platforms

I showed that microarray gene expression Platforms contrast with RT-PCR and RNA-Seq approaches for gene with low fold change value. Then the best parameter evaluating your differential expressed genes is the p-value or corrected p-value (FDR).

Article EVALUATION OF MICROARRAY SENSITIVITY AND SPECIFICITY IN GENE...

You should be careful, when using a combination of FDR and an arbitrary logFC cut-off. Remember, at any given FDR you expect some tolerated number of false positives. Say, at FDR 0.05 you expect 20% (correction, 5%) of your genes to be in fact NOT differentially expressed. So filtering on logFC you can arrive to a situation where all your filtered genes are in fact false positives. The best way to deal with this problem to my knowledge is to use the TREAT function from the limma package in Bioconductor. What it does is performing a test against a specified logFC. So rather than your null hypothesis being "the FC between the conditions is 1, i.e. no change", the null hypothesis is "the FC between the conditions is not more than [insert number here] times. Since it looks like you're dealing with RNA-seq experiment, you will need to use a limma-voom workflow (you can start with a table of counts, as long as they are raw counts, not rpkm/fpkm values).

You can find out more about TREAT here:

http://www.ncbi.nlm.nih.gov/pubmed/19176553

Aliaksei, controlling the FDR means to control the expected proportion of falsely rejected null-hypotheses among all rejected null-hypotheses. This means: is a list is constructed controlling the FDR at 0.05 you would expect up to 5% (not 20%) false positives in this list.

But, as you seem to have indicated by your hint to TREAT, testing point-null hypothesis is always a rather pointless procedure for biological data, because the null-hypothesis is never correct anyway. Therefore it is somewhat strange to expect falsely rejected hypotheses at all. The question remains which genes one may consider "significantly " regulated, and this requires thinking about the effect size and the role of the gene (so I don't think that any automatable or general selection criterion is sensible here).

Nevertheless, some automated selection can narrow down the list of "potentially interesting candidates" to some few genes that can be further studied. But I don't think that there is some golden way to do this selection. It depends on the purpose and the experimental context. Selecting by a fold-change can be as good and as bad as selecting by p-values (adjusted or not). (controlling some kind of error-rate is only a variation on the selection by p-values).

Johen, my bad, I had 1 in 20 in my head and that somehow translated to 20%. Of course FDR 0.05 means 5% of false positives :) Thanks for spotting it.

What I meant by alluding to the TREAT is that if one is to use logFC to select the genes with the higher effect size, one may as well do it in a statistically sound way. I.e. testing a hypothesis that fold change is significantly higher than a particular threshold. As opposed to testing for ANY fold change and applying the threshold independently.

The answer to you question depends on where you go with those DEGs next:

However, low fdr (somewhere between 1 and 10%) is requirement in any case.

A large fold change is helpful if you plan to measure  some of these genes using different technology (rt-pcr, nanostring , etc). 

large fold change a is not important if your next step is data-mining of DEGs using some sort of gene set enrichment analysis. In fact, you would under-power your  analysis by cutting out the genes that are highly significant (low fdr), but have small fold change.

On separate note, large fold change for fairly high but significant FDR (10%) indicates that expression of your gene is not uniformly distributed   suggesting biological heterogeneity  within  a class/group defined as the same thing otherwise.

@Andrey Morgan

@Jaroslav Zak

 Sir, I have a question, the software GFOLD gives FDR=1 for all genes, when run for samples without any biological replicates and gives GFOLD value which is equivalent to log2 fold change. What will be the criteria to select significant DEGs, only fold chang? will this FDR=1 for all genes justifiable?

I usually combine FDR with fold-change. it depend on your specific question. if you want the top genes or extract patterns of gene expression (clustering, PCA), combine FDR with fold-change. if you are interested in enrichment and pathway analysis, then FDR alone is better because small changes in gene expression might affect the whole pathway.

Hi I suggest this paper from proteomics data

Proteomics. 2011 Jun;11(12):2565-7

the fold change tresholds should consider the power of statistics an thus the number of replicates and the overall variation

Hi Parviz Heidari

I think filtering using FDR-correct p-values first then rank by logFC cutoffs is appropriate to select DEGs also commonly used in many RNA-Seq studies. Of course there is currently no gold-standard, also with different software you may get slightly different results but you just need to make this clear in the method section of your manuscript/study how you would define these DEGs.

If you get many genes with low FDR, then use FDR. Otherwise, you can use P value for filtering DEGs. You must set a threshold value of log2FC or FC for filtering the important DEGs with significant FDR value. You should text in methods about the threshold value. For example, FDR1.

Jaroslav Zak Thank so so much for your thorough answer. I just wanted to bring up a way of circumventing the issue of infinite fold changes when a gene has zero expression in one of the groups. As you mentioned in your answer, when a gene has zero expression in group 1 and positive (greater than zero) expression in group 2, the fold change (FC) would tend to infinity:

FC = group2/group1 = group2/0 --> tends to infinity

However, if we replace all the 0s with a really small number, such as 1E-10, we can avoid that problem. In fact, we can get a fairly accurate picture of how the gene expression compares between two groups.

For example, if group 1 doesn't express the gene (i.e. we have 1E-10 reads for that gene in group 1) while group 2 does, the fold change will be a really large number. On the other hand, if the gene is not expressed in either of the two groups, the fold change will be 1, and the log(FC) will be 0.

This can be especially useful if we need to explicitly know the fold change and p-value for data visualization purposes (e.g. volcano plots).