When we download thousands of sequences for databases, we often pick multiple copies of the same sequences which are submitted by different group or sometimes different strains. How can we curate the data set, so that it contains only one representation of each sequence for quantifying and comparative studies ?
Always good to investigate other established approaches first. NCBI RefSeq database is a good start:
http://nar.oxfordjournals.org/content/35/suppl_1/D61.full
Vince
With Jalview one can easily remove duplicates and near-duplicates (highly similar sequences). See page 51 "Redundancy Removal" in the tutorial:
http://www.jalview.org/tutorial/TheJalviewTutorial_screen.pdf
Hashsum is a good idea.
If you are familiar with perl , you can use the hash function to do this.
But if you don't, I think excel is a simple and effective way to do this.
You may try CD-HIT, an excellent tool for clustering molecular sequences. It lets you choose at which level of sequence similarity you want to group your sequences; for example, you may want to keep just one representative sequence of those that are 100% identical (or 99%, or 90% or whatever cutoff you want to choose). Easy to use, easy to understand, I certainly recommend it!
http://weizhong-lab.ucsd.edu/cd-hit/
Hello how to investigate the toxicity
of micozole on humane feminine
As other people have mentioned..
Uclust and Cd-hit are good programs to use
If you download from datasets from the EBI you can also specify the redundancy e.g. 50%, 90% or 100%
The NCBInr database contains redundancy so will need to be uclust'ed or cd-hit'ed
if u have an identifier like uniprot accession or any accession, by which u give them identity, Use remove duplicates option in excel 2007 or higher. dont use this option on sequence data as limit is 256 characters only
Try this tool: http://emboss.bioinformatics.nl/cgi-bin/emboss/skipredundant?_pref_hide_optional=1
- Badges
- Science topic
- Similar topics
- Bioinformatics
- Bioinformatics and Computational Biology