There could be a lot of reasons.

First, the alignment itself might be messy in some of them. e.g. the SSU can often have "messy" alignments due to the hypervariable regions, so using Silva for the alignment is beneficial as it uses secondary structure information in its alignments.

Second, have you used a partitioned model? The combination of genes you describe are highly different in variability and function. Also, using a codon model for the protein genes, can be beneficial. In any case, with a partitioned model you should use the best substitution model for each gene.

About your COI, do you include the whole gene or just the coding regions? Introns will add noise to your alignment. If that's the case you can also try using the protein sequence. You can combine prots and DNA in a partitioned model.

Gappy regions and compositionally biased positions in the alignments can also be source of problems. Take a look to BMGE or Gblocks

There could be a lot of reasons.

First, the alignment itself might be messy in some of them. e.g. the SSU can often have "messy" alignments due to the hypervariable regions, so using Silva for the alignment is beneficial as it uses secondary structure information in its alignments.

Second, have you used a partitioned model? The combination of genes you describe are highly different in variability and function. Also, using a codon model for the protein genes, can be beneficial. In any case, with a partitioned model you should use the best substitution model for each gene.

About your COI, do you include the whole gene or just the coding regions? Introns will add noise to your alignment. If that's the case you can also try using the protein sequence. You can combine prots and DNA in a partitioned model.

Gappy regions and compositionally biased positions in the alignments can also be source of problems. Take a look to BMGE or Gblocks

There are many good supermatrices of insect DNA already constructed, such as the one used in these papers:

Regier JC, Zwick A. Sources of signal in 62 protein-coding nuclear genes for higher-level phylogenetics of arthropods. PLoS One. 2011;6(8):e23408. doi: 10.1371/journal.pone.0023408. Epub 2011 Aug 4. PubMed PMID:21829732 PubMed Central PMCID: PMC3150433.

Regier JC, Shultz JW, Zwick A, Hussey A, Ball B, Wetzer R, Martin JW, Cunningham CW. Arthropod relationships revealed by phylogenomic analysis of nuclear protein-coding sequences. Nature. 2010 Feb 25;463(7284):1079-83. doi: 10.1038/nature08742. Epub 2010 Feb 10. PubMed PMID: 20147900

Those papers were attempting to sort out the evolution of not only hexapod insects but also arachnids and other arthropods. But there are also dozens of similar papers focused on insects in general or classes of insects such as ants, or wasps or butterflies or beetles.

Anyway, the point is that it may be far more productive to make additions to an existing supermatrix that has already been studied than to create a new one from scratch. Mixing nuclear genes with mitochondrial genes is not necessarily a problem but it can bring up many issues because the mitochondrial DNA evolves some ten-fold faster than nuclear DNA. Mitochondrial DNA can be very useful for studying close relationships between species that have evolved from each other in the past 1 million to 50 million years, while nuclear DNA can be more useful at deeper timescales of hundreds of millions of years.

A species obviously can't evolve with a stop codon in a critical mitochondrial or nuclear protein-coding gene, so there must be some trouble such as not recognizing that the mitochondria use an alternative genetic code.

Many papers have been published using quite sparse supermatrices with less than 50% of the data filled in and the rest represented by dash characters or even a mix of dashes, NNNs and ???s characters. There has been some study of how the missing data impacts the results, but in general rather good results have been obtained even with quite sparse data.

I don't think its a good idea to do phylogeny based on highly variable regions of DNA, conservative regions is preferable! Also, I know that different softwares neglect gaps when draw phylogenic tree (Mega6, for instance).

Thank you all for your answers and suggestions!

Dear Xabier. Yes I use a partition model. About COI, I use partial alignments, but to be honest I am not sure if introns are excluded in the sequences I have.

Arthur Burzynski, ha ha, true! I knew that. That's why I shouldn't post before a cup of coffee!

I got the from GenBank, but since we talk about COI, my previous post is irrelevant.