Dear Maria,

your question was very relevant and it seems that many people have the same problem, according to the many views. The function I posted is now in use by many people, but once in a while I get a message from somebody claiming that the function is returning an error. I made some debugging to the original function and also added to possibility to use dissimilarities generated by 'daisy' from package 'cluster' rather that 'vegdist' from 'vegan'. You can find this function if you browse to all the posts, but my first answer was voted by many people so it appears first.

So It is very important that everybody of you vote to this answer here, were I am posting again the final function (below). Please use only this function.

To use the function. Read the header of the file below.

You can just copy&paste the content of the file to your running R session.

The use like this:

data(iris)

pairwise.adonis(iris[,1:4],iris$Species)

more examples in the file below.

enjoy, Pedro

Hi María,

One possible way would be to preselect only samples for each pair-wise comparison from your community matrix a then run the anonis() on this subset. You can repeat this for all combination of the treatments, or only for combinations which make sense for your dataset.

However, you should be aware that this is multiple-testing and you should use some form of correction for multiple-testing (e.g. Bonferroni) or at least to interpret the p-values close to the predefined level of significance (e.g. 0.05) carefully.

Best regards,

martin

Hi María,

As Martin pointed out, there doesn't seem to be another solution apart from building some customized code. The function betadisper can give you pairwise differences for differences in variance among treatments though, which could be potentially interesting for your study.

Otherwise, you may consider the following free software: http://folk.uio.no/ohammer/past/. It gives pairwise differences with PERMANOVA and is relatively straightforward to learn (haven't tried the latest versions though).

Best of luck!

Raphael 

Hi María and all, 

I have the same problem! Did you find the way to solve it?

I have the software Past and it actually gives pairwise differences with PERMANOVA. But I have problems to export the distance matrices from R. Do you know how can I export a distance matrix? (They are to big to copy from console)

Thank you,

Regards, 

María Eugenia

Hola María,

your post is quite old, so you will have solved the problem by now.

It happens that I had the same problem, and google didn't gave me any solution.

I took up the comment of Martin to program a function for pairwise adonis using subsets of the dataset. You will find the function below. After copy-pasting the code below into your R session, you will be able to use the function pairwise.adonis() with following arguments

x = the community table

factors = a column or vector with all factors to be tested pairwise

sim.method = similarity function, one of the functions available in vegdist(); default is 'bray' for bray-curtis

p.adjust.m = the p.value correction method, one of the methods supported by p.adjust(); default is 'bonferroni'

The function will return a table with the pairwise factors, F-values, R^2, p.value and adjusted p.value

Regards Pedro

##start copy here for function pairwise.adonis()

pairwise.adonis

Hi everybody,

Pedro Martinez Arbizu, thanks for sharing your code - the way you did it is brilliant!

At first the code wasn't working for me so my friend Robert Danczak slightly modified it:

#PERMANOVA with Adonis

x = data

adonis(x~factors)

##Now, pairwise.adonis() for my factor location

factors=factors$Location

pairwise.adonis

Hi Pedro Martinez Arbizu,

      First, congratulations for this amazing function!!!! I try to use with two factors, this is possible? Look my code exemple, please, this is correct?

#PERMANOVA with Adonis

#1st factor

treat

Hi everybody,

Quiterie Duron wanted to use 'gower' as similarity metric as calculated by daisy (library cluster) rather than vegdist from vegan, because the daisy implementation allows for categorical variables.

I have change the code of pairwise.adonis to allow choosing for daisy or vegdist as function to calculate the similarity.

So you can call the function as:

pairwise.adonis(your_data, your_factors, sim.function='daisy', sim.method='gower')

or for example:

pairwise.adonis(your_data, your_factors, sim.function='vegdist', sim.method='bray')

The code is quite straight forward.

Just take a look at it.

Pedro

Please can anyone let me know what other methods are available for the

,p.adjust.m = "bonferroni"

field?

cheers Julian

...any of the methods supported by p.adjust()

"holm", "hochberg", "hommel", "bonferroni", "BH", "BY",  "fdr", "none"

https://stat.ethz.ch/R-manual/R-devel/library/stats/html/p.adjust.html

Pedro

For me "BH" give best results!!

Alexandre, how did your results came out?

I am also interested in pairwise results for a two factor PERMANOVA, but still could not run your script correctly.

Thanks!

Hi Marcelo, what kind of errors shows?

Sorry, my mistake! I was running with some typographical errors and results were messed up (no error messages). I think I have corrected all mistakes and now the script is running perfectly. Thank you very much!

Hi. You probably should check the mvabund package for R, which run multivariate analysis of species abundances or presence-absence data using a model-based approach (glm´s). Therefore it allows for testing the effects of predictors directly on the abundances, rather than on distance/similarity measures.

http://onlinelibrary.wiley.com/doi/10.1111/j.2041-210X.2012.00190.x/full

Hi, try but dont undestand, how give the permutation number

'nperm' >= set of all permutations: complete enumeration.

Set of permutations < 'minperm'. Generating entire set.

Hi,

New problem with pairwise.adonis(), when I used sim.function = 'vegdist', sim.method = 'mahalanobis', p.adjust.m ='BH' parameters the code runs OK, but if I use another approach by gower and daisy (sim.function = 'daisy', sim.method = 'gower', p.adjust.m ='BH' ) don't run. The output error was - Error in G * t(hat) :  arrays dimension are not compatible- and I'don understand how is possible?

Below my pairwise.adonis() changes:

#

pairwise.adonis

Alexandre,

In order to understand errors, you need to atomize the problem.

What I will do is to check first if I can run a normal adonis on the data with a gower distance matrix that I generate with daisy?

So step by step

1.) load library cluster,

2.) generate the dist-matrix with daisy and gower metric

3.) run adonis with 2 factors

Or you can run the function with daisy, but metric 'euclidian' to test if the daisy package is working as desidered.

If this also doesnt work, then the error will be somewere else. I would the recommend that you run the script not as function, but that you run command by command and see were you get the error message.

The fact that " arrays dimension are not compatible" seems to me that there was a problem with creating the similarity matrix. Do you have install "cluster"?

I have not look at this function since very long time, and would need some time to come into it again

In the mean time the general validity of Permanova has been questioned.

As Francisco Mora pointed out people are moving towards GLM and MIxed models.

MVabund is worth to give a try.

Pedro

Thank you very much for your attention Pedro. I will make this.

Best wishes

Hi, Richard Cadenillas, I have the same problem,did you solve it ?

Hi, Alexandre , I'am doing the pairwise PERMANOVA with two factors using your code, but I don't know how to revise the code according to my factors, my factors are treatment(T1,T2,T2) and Day(3,5,8), could you please help me revise the code,thank you~~~

Hi Xiaoping Fan I can see your code!!

Hi,Alexandre, in order to make it clear, I've uploaded my part of my data and the R code, but I think the code I revised was a bit wrong, so would you please revise it correctly according to my data, thank you very much~~

I also have a question about this pairwise permanova, since I read some reference which says that pairwise permanova can add the frequency of typeⅠerror. According to my pairwise permanova with one factor, the results of pairwise permanova and the method Martin Kopecký mentioned  above are different. I wonder if you have tried these two methods respectively,and how about the results. 

Dear Maria,

your question was very relevant and it seems that many people have the same problem, according to the many views. The function I posted is now in use by many people, but once in a while I get a message from somebody claiming that the function is returning an error. I made some debugging to the original function and also added to possibility to use dissimilarities generated by 'daisy' from package 'cluster' rather that 'vegdist' from 'vegan'. You can find this function if you browse to all the posts, but my first answer was voted by many people so it appears first.

So It is very important that everybody of you vote to this answer here, were I am posting again the final function (below). Please use only this function.

To use the function. Read the header of the file below.

You can just copy&paste the content of the file to your running R session.

The use like this:

data(iris)

pairwise.adonis(iris[,1:4],iris$Species)

more examples in the file below.

enjoy, Pedro

Hello! Has this code been published somewhere? How can I cite it? Cheers!

NO, is not published. Just cite this research gate page.

Pedro

 https://www.researchgate.net/post/What_are_the_criteria_for_selecting_variables_for_PCA_analysis_in_Primer_6_how_to_screen_variables

Hi Pedro,

I have tried to get this code to work but when I run it with my data I get the error message ' Error in unique(as.character(factors)) :

  argument "factors" is missing, with no default'

What does this mean and how do I resolve this?

Best wishes, 

Kate

Hi Kate,

view my post from Jan 4th. Copy the file parwise.adonis.r

Open it in an editor (I recommend Notepad++). Copy & past the content in your running R session. Then the function is available to your session and you should be able to try this example:

data(iris)

pairwise.adonis(iris[,1:4],iris$Species)

If this works, then use with your data:

pairwise.adonis(your_data, your_factors)

Change "your_data" and "your_factors" with your real data.frame and column

If this doenst work. I will need to see your data. [email protected]

Pedro

Hi Pedro,

Thanks very much for your reply. I have got the code to run and perform the pairwise test.

Is there any updates on how to cite this code in papers?

Best wishes,

Kate

Well, I used myself in an analysis that we are ready to submit. Perhaps I should include the code there so that people can cite it.

I will let you know. For the moment just cite this research gate page.

Thanks Pedro

Hello Pedro,

If you publish your code please let me know. We modified the code so it will work using a distance matrix as an input. We are going to publish the modified code as a supplemental material, citing you, your original code and the link to this RG page there. However, if you publish it, we can cite you directly in the main text of the manuscript, acknowledging you in a more proper way.

 Hi there Pedro,

I got interested in your script to run pairwise comparisons. Did you managed to have this code published as you mentioned? Please, let me know how could I cite your code in a publication. Thanks

Cheers

Alexandre

Hi Pedro, I am interested in your solution to run pairwise comparisons after adonis test. I tried your script but I am getting an error.

Error in `contrasts

Hi Joao,

this error message tells you that the column you choose for the factors only has level.

There are many reasons for this. One can be that R is not finding the column you specify. Try using yourdata$column1 instead of just column1.

or  (assuming your factors are in the first column) yourdata[ , 1] not just  yourdata[1].

Best is if you try the example with iris dataset. This should work. When you get this running try your own data:

Make sure you use the last version I uploaded. See message from January 4th 2017

You can just copy&paste the content of the file to your running R session. Then you can use the function.

The use like this:

data(iris)

pairwise.adonis(iris[,1:4],iris$Species)

Good luck, Pedro

Thank you Pedro! 

The original code gives the error referenced by Joao Gouveia,

but the new .txt download works well. Good work!

Hi Pedro- great little function! One question- how exactly are the p adjusted values determined? These are much higher than the p.values (i.e all 1.00, wheras many p.values are

Joseph,

the function uses p.adjust() to adjust p-values for multiple comparisons. Default method is bonferroni, which is very conservative, but you can choose others. Some people prefer holm instead.

Read this:

https://stat.ethz.ch/R-manual/R-devel/library/stats/html/p.adjust.html

Pedro

Hi Pedro

Please, let me know how could I cite your code in a publication. Thanks

Regards

Luján

Hi All and Alexandre Santos,

I'm trying to use your two factor function and get the error

'nperm' >= set of all permutations: complete enumeration.

Set of permutations < 'minperm'. Generating entire set.

My factors are Location and Temperature

I think it has nothing to do with problems in the function itself but more with the statistics since I also tried a combined factor that is "location a temp x" when I tried to solve the two factor permanova by myself. 

Any ideas?

Thank you

Hila

Hi Hila,

is this an error? It seems to me to be a warning or just an information.

This is raised by adonis from vegan, not by my function (which is just a wrapper). It means that you are trying to do more permutations than physically possible with your data. And is informing you that ALL possible permutations are used.

See also here:

https://www.researchgate.net/post/ANOSIM_using_R

Check discussions elsewhere, on using MonteCarlo simulation to overcome the low number pf possible permutations- problem

Pedro

Hi Pedro,

Thank you for posting your pariwise permanova function. It is very useful. I'm wondering if you know how a monte carlo correction could be included in your function? I've looked around for an answer but can't seem to find an easy way to implement monte carlo in R.

Any tips would be appreciated. Cheers

Hi Jessica,

no I have not think about implementing Monte Carlo with adonis.

To be honest I have not look into details of this. The main reason is that I don't think that MonteCarlo will solve the problem of having low amount of samples. My recommendation if you need to improve the power of your analysis would be: take more replicates!

I know this is not the answer that we want to hear, specially if you are working on samples from remote places that were taken years ago....

Pedro

Thank you very much.

I decided to put the function into an R package, which makes it easier to install, reuse and share.

To install from github:

library(devtools)

install_github("pmartinezarbizu/pairwiseAdonis/pairwiseAdonis")

Note that in Windows you need to install Rtools in order to be able to install packages from source.

Follow instructions here:

https://github.com/pmartinezarbizu/pairwiseAdonis

For the many people asking how to cite this function:

Martinez Arbizu, P. (2017). pairwiseAdonis: Pairwise multilevel comparison using adonis. R package version 0.0.1.

Pedro

Hi Pedro! Thanks so much for this great function! I am wondering if there is an easy way to do custom contrasts instead of pairwise for all levels? (comparing one level of a factor against all other levels - i.e., similar to the "ctrl.vs.trt." function in lsmeans?) Thanks so much!

Well, an easy way, without re-coding the function, will be to subselect the result using regular expressions.

data(iris) res

Sorry, it got a little bit scrambled, here again

data(iris)

res

Thanks, Pedro! Will this also adjust the p-values for only doing those comparisons? Or are those adjusted p-values based on the large number of original comparisons?

Oh, I see. You are rigth, the corrected p.values consider the whole set, not only the reduced set. I will think about how to change the code. pedro

Thanks, Pedro!

Gareth,

I have added the argument 'reduce' to the function.

say you want to see only pairs including 'setosa':

pairwise.adonis(iris[,1:4],iris$Species,reduce = 'setosa')

say you want to see setosa and virginica:

pairwise.adonis(iris[,1:4],iris$Species,reduce = 'setosa|virginica')

Install function from github

library(devtools)

install_github("pmartinezarbizu/pairwiseAdonis/pairwiseAdonis")

Please try

Pedro

...again formatting problems

Gareth,

I have added the argument 'reduce' to the function.

say you want to see only pairs including 'setosa':

pairwise.adonis(iris[,1:4],iris$Species,reduce = 'setosa')

say you want to see setosa and virginica (use pipe):

pairwise.adonis(iris[,1:4],iris$Species,reduce = 'setosa|virginica')

Install function from github

library(devtools)

install_github("pmartinezarbizu/pairwiseAdonis/pairwiseAdonis")

Please try Pedro

Hi,

I have an OTU table from 16SrRNA microbiom and bed/bim/fam files from genotyped individuals. how I can say which SNPs are related to which bacterial taxa in my ???

please help me, I was really exhausted by too much googling

Thank you all

Pedro,

Great job creating the pairwise comparison tool!

But I think you probably should caution users that your library only works for cases where free permutation (or by the default values in adonis() function) are appropriate since there are many experiment designs and hypothesis tests, in which the users need to specify the permute() function based on the design and tests.

Certainly, users can modify your functions to get there, but I am afraid some users may not be aware of the limit.

Xingtao

Hi Xiangtao,

thank you for this important remark. Could you please elaborate a bit more your comment. This will be helpful for everybody. Do you have an example?

Pedro

Hi Pedro,

There is a parameter called permutations in the adonis or adnois2 function, which allow the user to specify how samples should be shuffled for the intended hypothesis test, and the default value is 999 free permutations. But in many designs and hypothesis testing, we should not use free permutations.

One example was given in R help. If you bring up the adonis help in R by typing ?adonis, the last example mentioned the incorrect way (no strata) and correct with strata and how to set up the strata.

Specifying the permutation can be very complicated depending on the design, and there is a R package called "permute", which enable users to set up the permutation for some quite complex designs, but it requires some statistical background to comprehend the package.

Thanks,

Xingtao

Hello all,

I am using PAST for PERMANOVA. PAST has pairwise comparions for one way but I could not figure out for two way PERMANOVA. Is there way to perform pairwise comparisons in two way PERMANOVA in PAST? Would you please suggest me how to?

Regards,

Dipak

Hi Pedro,

Is there a way to use your package using multiple factors? Such as pairwise.adonis(df , A * B, data = df.env)

Thanks in advance! Your script is great!

Hi Pedro,

Can the function be applied to user defined distance matrices? For instance, I would like to do a pairwise adonis using Weighted Unifrac distances. Unifrac methods are not included in the vegdist() methods. Instead they can be found in phyloseq:distance() or GUnifrac.

We have a modified code published here: Article Livestock disturbance in Brazilian grasslands influences avi...

Hi Sylvain, thanks for the update. Feel free to add your code to the developing version in github. i would love to incorporate them.

Regards Pedro

I need a help, I've like to know what is the basic inference if I use vegdist or daisy in adonis () function? I found completely results and lots of warnings.

Hello,

I am interested in using the pairwise.adonis to run multiple comparisons, but as mentioned earlier I am only interested in specific contrasts. I understand that the 'reduce' option but unfortunately it doesn't work for my case. I need to test the effect of treatment (2 levels) within each month(4 levels), so the reduce argument doesn't work for me.

Any ideas?

HI Gabriella,

seems that many people need the contrast argument. So I will try to implement it. The goal is that you can enter the arguments just as in adonis. I am just back from a 2 months cruise and need to cath-up with normal institute duties, so it will take some weeks before I can look at it. If there is anybody out there who wants to help with this, I would be more than happy. The code is in Github.

https://github.com/pmartinezarbizu/pairwiseAdonis

Regards Pedro

Hey Gabriella,

When I have faced this issue in the past I create a new variable by concatenating the two variables of interest (in your case "treatment" and "month" would become something like "treatmentXmonth"). Then calculate all possible pairwise contrasts with this new variable (without Pvalue correction), filter the results dataframe to contain only the contrasts that are reasonable, and then apply your desired pvalue correction to the calculated p-values.

Good point. So

newcolum

Thank you for your quick responses!

I ended up doing what Julian suggested, running all the pairwise comparisons with no p value correction and then applying the correction separately to the contrasts of interest.

Hi,

I am having a hard time editing the pairwise.adonis function Pedro created and was hoping someone could help me.

I've editted the code as so:

factors=PGuilds2018 [,1]

pairwise.adonis

I just updated the function to include Sylvain changes,

please reinstall from github

https://github.com/pmartinezarbizu/pairwiseAdonis

v.02

• input can now be data frame or distance matrix
• examples to use strata are given

Hola Pedro. Tu formula ha sido de gran ayuda. Gracias. Me preguntaba si puede ser utilizada para comparar interacciones entre factores. Por ejemplo, temperatura*profundidad o sitios*temperatura. Gracias de nuevo!

Hello everyone, I have a different question. Is it possible to do PerMANOVA with one time sampling data though I am having replicate data?

Hey Pedro Martinez Arbizu ,

Firstly, thank you for a great package!

It is working perfectly with my 1-way permanova, however, I am trying to do the pairwise comparison with a 3-way permanova (without interactions), is it possible to do it with more than 1 factor? I see several have asked the same question here, but I can't see an answer.

Cheers, Barbro

Gracias Sylvain!!

Hi Barbro and Sylvain,

we need to figure out the correct procedure.

In order to use strata there are at the moment 2 posibilities:

Check with me in your R session

library(pairwiseAdonis) data(iris) pairwise.adonis(iris[,1:4],iris$Species)

#create a new vectors of factors

strata

Looking myself at the results, I guess that the 2nd method, should be correct.

pairwise.adonis(iris[,1:4],paste(iris$Species,strata))

This is returning significant diference between all combination of species, irrespective of summer or winter, but non-significant within species for winter-summer.

So, setosa winter is not different to setosa summer, etc

This makes sense as the factor winter-summer was created by us and is not comming from the original iris dataset. We are just splitting the dataset in two equal parts.

By the way you can reduce to comparisons between factors with 'reduce' argument:

pairwise.adonis(iris[,1:4],paste(iris$Species,strata),reduce='setosa')

Pedro

Dear Pedro,

I'm using your fantastic pairwise.adonis.r function. I have a question though, since it's a multivariate analysis, what is the default dissimilarity matrix? I added: method="euclidean" to make sure I'm not using "bray" or something else, but I got an error. I hope you can answer me.

Cheers,

Michal

Hello Michal,

is important that you install and use the github version.

https://github.com/pmartinezarbizu/pairwiseAdonis

This is the only one that we are maintaining once in a while.

Read the help

?pairwise.adonis()

#similarity euclidean from vegdist and holm correction

pairwise.adonis(x=iris[,1:4], factors=iris$Species,sim.function='vegdist', sim.method='euclidian',p.adjust.m='holm')

Regards Pedro

Dear Pedro and/or Sylvain,

First I want to say thank you for having put this useful function together, but I have a layered question related to the stratum option that was implemented. I tend to use it for blocking individual subject/site variation when there has been repeated sampling so that the code may look like:

pairwise.adonis(unif_w, factors = met$Treatment, p.adjust.m = "BH", stratum = met$Subject)

With some of the datasets I have used this on, it works and tends to increase the power when compared to using without the stratum option and with others adonis doesn't seem to work correctly and I get p.values equal to 1 for all comparisons. Usually these results would be accompanied by a normal adonis where I have seen that the factor I am testing is significant, and so I expect atleast one comparison to be significant if I set it up correctly.

The difference in whether that code worked seemed to be related to whether or not I had a design where every factor level occurred with every subject level, and that is not the case in every dataset I have attempted to use this function for. So, it only worked if every subject occurred with every factor level. Does that conclusion make sense with your understanding of how stratum constrains the permutations?

I'm sorry if this is only a different version of something that has already been asked in this thread, but I'm curious if you have looked at a scenario like this and thought about how to account for individual variance in the function? I'm also curious if you think that is an appropriate use of the stratum option (if so then maybe I have a good example dataset or two for Sylvain in his most recent post).

Thanks, Craig

Craig,

the original pairwise.adonis function was intended to be a simple wrapper for doing adonis on any possible combination of factors and putting the results together in a simple table.

I would say, that this is still the main aim of this function. The strata option in pairwise.adonis should be treated as beta option and be used 'at own risk'.

I have been surprissed that many people request to include strata in the pairwise.adonis function. To find the correct syntax in the R formula to reflect the correct strata and nestedness of the design is quite difficult. I dont feel myself confident with the required syntax. Small errors there may leed to wrong results.

See this post for an example:

https://stats.stackexchange.com/questions/188519/adonis-in-vegan-order-of-variables-or-use-of-strata

To be able to accept any of the inputs that adonis can accept, we will need to re-program pairwise.adonis to use the formula format and pass this unmodified to adonis (of course with a new combination of factors in every pariwise comparison). So, x shoudl be of the format a ~ b ...

The way we have it now with a, b as separated arguments (datamatrix, factors) is more conveniant for processing the arguments within the wrapper function.

I still didnt have time to write the new code, but I belive is the way to go.

Pedro

Hello folks,

finally, I had some time to look at the strata issue.

As a result I have recoded pairwise.adonis and programed a second function pairwise.adonis2. Please update your package:

install_github("pmartinezarbizu/pairwiseAdonis/pairwiseAdonis")

pairwise.adonis is no longer supporting strata, as it was never intended for that purpose.

If you need strata, use pairwise.adonis2. You will be able to use formula syntax for the model exactly as in adonis. This allows you to use interactions between factors.

As a caveat there is a specific syntax you need to use.

See ?pairwise.adonis2 for details.

Please test and report any bug to me or raise an Issue at the GitHub page.

Pedro

Hi Pedro,

I found your pairwise.adonis2 very useful, thank you.

It does not provide adjusted p values, does it? Also, is the default sim.method also "Bray"?

Differently from pairwise.adonis, I could not set these parameters and didn't find any information on ?pairwise.adonis2 help.

Thank you very much,

Lucas

I have a doubt about your "pairwise.adonis" function. When I run adonis function to perform the PERMANOVA, p > 0.05; but when I run pairwise.adonis to look for pairwise differences, at least one pair has significant differences (p

The pairwise function takes each pair and runs the adonis test independently. Then it applies a bonferonni correction.

So it is very possible that the overall adonis would not be significant while individual pairs may be significantly different.

Thank you very much, Tania L F Bird

Another question. What if I want to use the turnover component of Jaccard as dissimilarity measure? What part of the script do I have to modify?

Thank you all!

Hi Pedro Martinez Arbizu

Is there a citation that can be used for your PERMAANOVA pairwise comparisions for publications?

Thanks,

Kate

Kate Mathers, I'm assuming you can use the citation from Pedro Martinez Arbizu's GitHub? https://github.com/pmartinezarbizu/pairwiseAdonis

Martinez Arbizu, P. (2019). pairwiseAdonis: Pairwise multilevel comparison using adonis. R package version 0.3