I am building phylogenies of the same species (4 subspecies) to identify genetic differentiation between the subspecies. I wondered if anyone has any tips for improving topology, or reducing unresolved polytomies in MrBayes?
I guess you selected the proper evolutionary model for your phylogenetic analysis, if so the polytomies that you are seeing are 1) the result of the lack of variability on the genes that your are working with 2) The lack of differentiation among your samples (or subspecies). If the problem is/are your gene/genes, then you should try to use other genes (i.e., mitochondrial and nuclear genes).
Another solution is using a different software like BayesPhylogenies (http://www.evolution.reading.ac.uk/SoftwareMain.html) and see if by increasing the number of Gamma parameters your likelihood increases and you get a "better" result.
(Easy answer: add more genes [sometimes not so easy]).
Great question: this is both a fascinating philosophical as well as practical issue simultaneously, and your challenge is to tap deep into your training as a phylogeneticist (and a biologist!) to come up with the appropriate answer for your system. Remember, there are in theory both hard as well as soft polytomies in nature. Determining which you are presented with is in a sense, first and foremost a matter of sampling depth, both in terms of genomic sampling and geographic coverage (plus number of individuals).
Sometimes it is instructive to consider extremes along the spectrum of perspectives (!), i.e. imagine you've compared genomic data for all specimens and you still can only recover an unresolved polytomy: would you be satisfied that it's hard, i.e. biologically real? Probably. But is that practical approach? Probably NOT. On the otherhand, had you sampled "293 bp of COI" gene fragment (as is often done much to my phylogenetic chagrin) for dozens of individuals of each subspecies, and you recovered the same pattern, would you stop and conclude a hard polytomy?
These are extremes. But I think most of us have a "rule of thumb" concept that if your sampling includes multiple character sets (mtDNA AND nuclear markers), sufficient statistical power in terms of # of phylogenetically informative sites and number of individuals included, as well as variety of types of genes (i.e. ribosomal AND structural, or protein coding), and you have taken into account gamma correction and invariant sites and selected an appropriate optimality criterion, and you are seeing a polytomy, then that's very cool and I think it is rational to conclude your polytomy is real. Time for taxonomic revision!
The polytomy could be real for a variety of reasons, such as incomplete lineage sorting and/or adaptive radiation (a relatively short period of changes followed by longer period of relative stasis). Brendan's answer is useful, although more genes will not always help and may in some cases be positively mis-leading. But his last paragraph offers a useful metric by which to judge for yourself.
When I encounter big polytomies among closely-related terminals, I always run an uncorrected pairwise distance analysis on my sequences to see how many of them are identical. Deleting all but one of a string of identical sequences will surely speed up your phylogenetic analyses.
If you are using molecular methods, a good choice is to add another mithocondrial gene. They are good to solve recent divergences. Sometimes, it's impossible to solve some politomies without being tendentious with your results.
You need to do certain things. I dont know which system you are working on, ie. insects, plants, vetrebrates, etc. COI may not be useful to see vaiation within the spcies. Try control region (CR) also called D-loop in mammals of mt region along with full length COI.
Before running Mr. bayes, run Modeltest in conjection with PAUP for both genes and change the setting in Bayes or PAUP for you likehood test. I hope this will give you good results, ITS region may also be tried.