currently using fastq-dump of sratool kit, but it is taking long time. I have to download really large data of bacterial genomes, any alternative ???
Go through SRA's ftp site to download sra files. You can use commands curl or wget via command line. Check out the SRA handbook.
http://www.ncbi.nlm.nih.gov/books/NBK47528/?report=reader
Go through SRA's ftp site to download sra files. You can use commands curl or wget via command line. Check out the SRA handbook.
http://www.ncbi.nlm.nih.gov/books/NBK47528/?report=reader
ftp site and SRA tool kit are good options, but even you need good net speed as well
Hello! Ramani. I would like know how can I use aspera connect to download all the reads sequences from a bacterial specie. I have been trying to install aspera but I can´t.
This question comes back often...
Use parallel-fastq-dump Here is a (https://lifebit.page.link/93tG) step-by-step guide to quickly download several samples using AWS services. In this example it took 2h to process more than 40 samples (70 GB) for around $40. --> download --> align --> read counts
Hope this helps
How to use NCBI SRA toolkit effectively to download datatsets from SRA: Read this post https://reneshbedre.github.io/blog/fqutil.html
Alain Coletta parallel-fastq-dump is only for python version 3.0 or above.
I would recommend downloading .sra file using aspera (it is the fastest i know as of now) and converting .sra to fastq using fastq-dump. I still feel the latter is kind of slow process but have not found better alternatives. If there is any, for python 2.0 version platform, let me know. Thanks in advance
Using aspera connect on linux OS, bellow the command line example to download the run number SRR7854698 :
ascp -i ~/.aspera/connect/etc/asperaweb_id_dsa.openssh -k1 -QTr -l100m [email protected]:/sra/sra-instant/reads/ByRun/sra/SRR/SRR785/SRR7854698/SRR7854698.sra /disitnation_dir/
for example, I retrieved a 32 Gb file in 50 min with an average of ~ 90Mb/s.
You will need to use fastq-dump from SRAtools kit to convert .sra to .fastq.
Using fastq-dump , it only fetchs the srrxxx.sra.cache files in the fold of /Users/ncbi/public/sra/.
Any suggestion would be very much appreciated...
It's a very old question, but this can save a lot of time and also it works pretty well
https://ewels.github.io/sra-explorer/
Thanks, Eric.
It is very useful. The FTP and/HTTP may lost a connection sometimes...
There is a new addition to sratoolkit that can use multiple threads and it splits the download into batches and then merges them once download finishes.
https://github.com/ncbi/sra-tools/wiki/HowTo:-fasterq-dump
Fatih , it ( fasterq-dump )should work well, but doesn’t want to sometimes)
I also ran into problems with fasterq-dump too with really large files.
Here is the error-proof way and you can even resume your download if it stops due to a connection problem by simply running the same command again.
Do not use fastq-dump or fasterq-dump for downloading. Use prefetch which is also from sratoolkit. Here is an example command:
- prefetch --max-size 100000000 SRR8371669
You need to define a maximum size because it has 20GB default limit for some reason.
Next, convert the downloaded file with .sra extension to fastq using fastq-dump locally:
- fastq-dump SRR8371669/SRR8371669.sra --split-3
Eric JC Gálvez gave the best answer. I will only supplement. You need to copy the SRA URL on this site ( https://ewels.github.io/sra-explorer/ ) and download Download Master (or analog for Linux). prefetch always let me down and I was tormented with this team.
Sergey, I never had problems with prefetch so far, works like a charm and very fast too. My sratoolkit version is 2.10.0 and I see an update related to prefetch in their log for 2.10.x versions, just in case..
check for batch download with multithreading https://www.reneshbedre.com/blog/ncbi_sra_toolkit.html
- Badges
- Science topic
- Similar topics
- Bioinformatics
- Bioinformatics and Computational Biology