NCBI Genbank hosts enormous numbers of COI sequences. However, it is not explicit how many species have their COI being sequenced, since there is also so many multiple sequences from one single species. I followed the keywords provided by Kwong et al (2012) to pull out the metazoan COI sequences (link provided below), with which I can obtain ~900,000 sequences, but cannot get the number of species information.
http://onlinelibrary.wiley.com/doi/10.1111/j.1096-0031.2012.00408.x/abstract
roughly today
using coi as gene name 827315 (793652 longer than 400 nt, 63103 >=800 nt, 36445 >=1000 nt, 7531 >=1500 nt)
in metazoa 815890
in viridiplantae 175
If you tell me which clade you are interested in, I can easily pull out
species\tnbr_sequences
And if I can then provide a link to download a fasta file of those (or gb file)
Cheers
"You will never be certain the obtained number is 100% correct "
agreed, thats why is wrote: "roughly"
- To retrieve sequences, I use my own programs (or via the eutils), not entrez which is not precise enough most of the time.
- then I filter on clade and minimal length.
- then I cluster sequences at some given % of similarity (needleman-wunsch, not blast).
knowing a number of manually confirmed sequences :
=>
- identify proper cluster(s), extract sequences & other infos from the genbank or embl entry.
- if new searches required in the future => try to estimate the best % of similarity to try to get every proper sequence but no non-target sequence in a single cluster (not always feasible).
Of course, this will need writing some lines in ... python of course
Best
I set my threshold for COI sequence length as 400:20.000, because I also want to include mitochondrial genome sequences in my local COI database. With the keywords given in the provided paper, I am able to get 900,499 sequences. I have not used the database as it is, but narrowed it down to ~700,000 sequences with setting certain thresholds.
I am not interested in a particular clade though, what I would like to obtain is the number of species within the 900,499 metazoan COI sequences that are submitted to NCBI. May I ask how you are pulling out the species @Richard Christen? This is something I am really curious about, and it would be useful to know. Many thanks!
Another tricky thing with my database is that, the sequences all have unique ids (ie different GI numbers). As a simple example, when I type coi in the search engine of NCBI genbank (please find the link below), the first three sequences belong to the same Escarpia sp. I checked my final local database with grep command, and I can see that all three of them are included. Hence, I am afraid if I use the provided sort | unique |wc -l, I am not going to be able to get unique sequences. I either have gi number or gi and gb identifiers in my output file. I am not quite sure whether I could get species information from this file.
http://www.ncbi.nlm.nih.gov/nuccore/?term=coi
Again this are rough estimates for reasons explained previously
And species is taken from the DEFINITION line, which was faster for me but less accurate. I could provide more accurate, but I have little time presently
total sequences: 827315
total species: 91694
sequences >1000 occurences/species
All results in attached file
43690 Lepidoptera sp.
3014 Bemisia tabaci
2347 Balanus glandula
2291 Ichneumonidae sp.
1970 Argiope bruennichi
1948 Antaeotricha sp.
1867 Elachistidae gen.
1522 Anomura sp.
1478 Blepharoneura sp.
1437 Belvosia sp.
1431 Epiphyas postvittana
1412 Hydroporus sp.
1391 Promachocrinus kerguelensis
1306 Hymenoptera sp.
1292 Astraptes sp.
1240 Brachyura sp.
1227 Stenoma sp.
1180 Culex quinquefasciatus
1100 Hemiceras sp.
1081 Carollia perspicillata
1061 Caridea sp.
1042 Microgastrinae voucher
1010 Homo sapiens
That is a great help, thanks. Btw, using these keywords ( [COI(Gene Name) OR ‘‘cytochrome oxidase subunit 1’’(Gene Name) OR ‘‘cytochrome c oxidase subunit 1’’(Gene Name) OR ‘‘cytochrome c oxidase subunit I’’(Gene Name) OR ‘‘cytochrome oxidase subunit I’’(Gene Name) OR COX1(Gene Name)]) would provide you the missed out COI sequences as well, in case you would need a more comprehensive coverage. That is how I was able to derive roughly 100,000 more sequences than your search results.
You probably used some scripting for getting these results. Is there any common command that I can use for this purpose or have you written your own code? Because, I would need the updated information on the species coverage, eventually.
yes I have my own programs to query and analyse the public databases.
With the keywords given above
total sequences: 945159
total species: 103475
- Badges
- Science topic