Are you trying to identify gene families? Or do you already have families and are trying to understand the relationships therein?

If you are trying to find the families to explore you can use a clustering algorithm, such as OrthoMCL, to create the families.

To parse the relationships within the gene families, I would think the best approach would include blasting all of the organisms/genomes of interest with known members of the family, aligning the hits with the queries, and treeing the resulting alignment. Depending on how you define the term "family" you would also want to include known negatives in the analysis as well. I.e. a closely related families that are not part of the one you are examining. This will allow you to see the boundaries of yours and not misidentify divergent genes as new versions in the family. 

In contrast to Ariel Chipman, I favour ML and Bayesian methods. NJ and other less computationally intensive tree-building tend to fall victim to artefacts in complex data more easily than ML and Bayes, such as long branch attraction (although they have their failings, too). I would interpret polytomies as he suggests, but would conclude that NJ is falling prey to the data rather than answering a different question. But I do not aim to start a fight. Really, we would hope that each method would yield similar or identical results. That is how you can be most confident. ML will work best if you use an appropriate model. Programs like jModelTest (DNA) and ProtTest (protein) can give you an estimate of the best model. I prefer the AICc criterion for my DNA sequences, but ideally AICc, BIC and DT will all agree on the same model. Bayesian programs, such as MrBayes can either be instructed to use this model or you can skip the model testing and instruct the program to select the best model as part of its run using the "mixed" parameter.

Hi Nico

There indeed does not seem to be a standard answer as such, and a robust tree is just that: it holds up under various different types of analytical approaches. As such, I have used various different methods over the years. It depends also how divergent the genes are, whether you have complete sequences or partials etc, and have some inkling s to whether different parts of the genes are under different types of selection (-, neutral, +).

Given that you mention homeobox containing genes, this worked for us for those kind of genes (see material and methods section) (note, these guys are highly divergent...):

http://journals.plos.org/plosgenetics/article?id=10.1371/journal.pgen.1004698

Cheers

Casper

There are a lot of existing answers on similar questions. The answer will be different depend on what level of phylogeny you interested in; how large your samplin and how closely related different taxa in it; what computational facilities you can use etc.

Except identification of ortologs/parags you probably will have the problem connected with that in genome a lot of genes have multiple very similar, but not the same copies. So you will firstly need to be sure that markers you fing are applicable in phylogenetic reason and only after this you'll be able to reconstruct something by any method. 

Hi Nico,

Not to repeat what the previous replies have said - my intuitive feeling has always been that for gene phylogenies, a simple UPGMA or NJ analysis is not only good enough, but sometimes even better than more computationally intensive approaches like ML or Bayesian analysis. I don't know if this feeling stands up to rigorous analytical standards, and I suspect no-one has ever looked at this specifically. However, my experience is that NJ/UPGMA usually results in a well-resolved tree that identifies orthologs  in a way that is consistent with my biological expectations. On the other hand, running the same analyses with Bayesian or ML approaches results in polytomies or trees with no statistical support. Hard-core phylogeneticists will probably tell you that this means that the quality of the data is not good enough to resolve evolutionary relationships. I would say that this means that ML/Bayesian algorithms are optimised for a different kind of question.

Ariel

Are you trying to identify gene families? Or do you already have families and are trying to understand the relationships therein?

If you are trying to find the families to explore you can use a clustering algorithm, such as OrthoMCL, to create the families.

To parse the relationships within the gene families, I would think the best approach would include blasting all of the organisms/genomes of interest with known members of the family, aligning the hits with the queries, and treeing the resulting alignment. Depending on how you define the term "family" you would also want to include known negatives in the analysis as well. I.e. a closely related families that are not part of the one you are examining. This will allow you to see the boundaries of yours and not misidentify divergent genes as new versions in the family. 

In contrast to Ariel Chipman, I favour ML and Bayesian methods. NJ and other less computationally intensive tree-building tend to fall victim to artefacts in complex data more easily than ML and Bayes, such as long branch attraction (although they have their failings, too). I would interpret polytomies as he suggests, but would conclude that NJ is falling prey to the data rather than answering a different question. But I do not aim to start a fight. Really, we would hope that each method would yield similar or identical results. That is how you can be most confident. ML will work best if you use an appropriate model. Programs like jModelTest (DNA) and ProtTest (protein) can give you an estimate of the best model. I prefer the AICc criterion for my DNA sequences, but ideally AICc, BIC and DT will all agree on the same model. Bayesian programs, such as MrBayes can either be instructed to use this model or you can skip the model testing and instruct the program to select the best model as part of its run using the "mixed" parameter.

In a phylogenetic approach,  you should consider that no all sequences share the same characteristic, then in order to reduce errors in the phylogenetic inference, you should be aware of  substitution patterns present in your sequences set, that is proportion on nucleotides composition (ATGC), substitution frequency between transitions and tranvertions (purines/purines, purines/pyrimidines), and heterogeneity among sites (not all sites change at the same rate, in protein coding sequences the third position mutate faster than first), then it is important run a model test (such as jmodeltest) to know which is the best model for a specific sets of sequences (JC, K2 TN93, GTR…), then you can use  this model to phylogeny reconstruction based on distances NJ,  ML, or Bayes.

Perform more than one phylogenetic approach is recommendable to know if the result are consistence with different approximations, since this kind of analysis have not real repetitions, phylogenetic analysis with more than one approaches is one alternative  to access accuracy in the reconstruction.

In a phylogenetic approach,  you should consider that no all sequences share the same characteristic, then in order to reduce errors in the phylogenetic inference, you should be aware of  substitution patterns present in your sequences set, that is proportion on nucleotides composition (ATGC), substitution frequency between transitions and tranvertions (purines/purines, purines/pyrimidines), and heterogeneity among sites (not all sites change at the same rate, in protein coding sequences the third position mutate faster than first), then it is important run a model test (such as jmodeltest) to know which is the best model for a specific sets of sequences (JC, K2 TN93, GTR…), then you can use  this model to phylogeny reconstruction based on distances NJ,  ML, or Bayes.

Perform more than one phylogenetic approach is recommendable to know if the result are consistence with different approximations, since this kind of analysis have not real repetitions, phylogenetic analysis with more than one approaches is one alternative  to access accuracy in the reconstruction.

Hi Nico,

Here my two cents ... I fully agree with Mathew's point of view about methods. However, I'd suggest to start your project by looking for your family at publicly available databases like PhylomeDB, eggNog, Enseml, OMA, etc. It will give you a first glance of how your gene family looks like. If data stored at those databases don't help you out to answer your question, I'd take data from there, combine it with yours and then reconstruct a Multiple Sequence Alignment, which depending on the conservation degree might be worth to trim, and then make a phylogenetic tree - depending on the number of sequences and the precision of you are looking at, you can use ML or Bayesian approach - or try something much faster but perhaps a bit less accurate than ML/Bayesian > Fasttree.

Since ML/Bayesian/Fasttree are all probabilistic models, it is important to choose the right model (either using program such as jModelTest or ProtTest) or allowing to the program to choose the best one (e.g. Mr. Bayes).

Hope it helps,

S

maybe these two paper might help you:

http://www.pnas.org/content/112/7/2058

http://arxiv.org/abs/1602.08268

The main idea of this new approach is as follows

[1] identify pairs of "putative" ortholog gene pairs,

[2] on this classification it is possible to construct respecitve gene trees

[3] from those  gene-tree information the "local" topology of the species can

     can be derived

[4] construct species tree.

Important here is that you need orthology-inference tools, that is an estimate of your putative orthologs/paralogs. Nevertheless, this approach does not need an explicit statistical underlying model to infer the gene and species trees + its reconciliation.

If you want to use tree-based orthology inference methods that rely on the reconciliation of a constructed gene tree (without event-labeling) from an

alignment of homologous sequences and a given species tree, then you need to know the species trees -- a hard task to my opinion, since for many scenarios

the species tree is not known with confidence and, in addition, all practical

issues that complicate phylogenetic inference (e.g. variability of duplication rates, mistaken homology, or HGT) limit the accuracy of both the gene and the species trees.