I am having a difficulty in implementing the Permutation Renormalization and Bootstrap method described by Faust et al in Microbial Co-occurrence Relationships in the Human Microbiome. My key difficulty appears to be in resolving visibly distinguishable communities with igraph. Community detection finds distinct communities (Louvain method), but I am still having trouble generating network plots with clearly distinct clusterings/communities of nodes. The networks "look" random and have densely packed edges.
While I thought this was to be expected given the number of nodes in my network is 2700, using the method created by Faust et al just doesn't seem to mark enough correlations as insignificant. I've even tried taboos like p-value hacking and setting percentile thresholds on the resulting nodes that are marked as "significant" in an effort to cut down on nodes.
To give an idea of the network's scale, I used a subset of my abundance data consisting only of 1,500 OTUs out of the original 2700 (by setting the rowsum abundance threshold to 12) when following only the protocol in the literature I get a network with 500,000 edges (In networks constructed with all 2700 OTUs or close to, that number approaches 3,000,000). The testing only appears to be removing roughly half of all edges. I am also removing all negative edges as I am interested in co-occurence only at this point in time.
I further pruned this network by setting a 99th percentile threshold on the correlation strengths of the significant correlations. I was then able to get a network of ~5,000-10,000 edges depending on the specific threshold. I'd like to say that the network is sufficient but this is obviously not the case. Is there any insight anyone can give me with this problem? Should I use the method in Faust to only remove correlations due to compositional effects and then perform a percentile threshold on the remaining correlations?
Hello David,
Based on my experience, it should not happen.
Are you using CoNet to apply ReBoot, or have you coded yourself?
Secondly, it seems, you're using raw abundance. Try running the algorithm after normalizing it for each sample, and remove OTUs based on presence (select OTUs present in more than 10% of samples and mean abundance of 0.1%; this is because, sparsity can also affect the number of edges, and network inference is mostly driven by the core microbiome).
Try changing the p value merging method, and apply p value correction.
Also, the number of edges depends on the correlation method used. Spearman or Kendall work very well, while pearson sometimes increases the coefficients due to few outliers.
I hope it helps.
- Badges
- Science topic
- Similar topics
- Linguistics
- Composition Studies
- Writing