Currently, I am analysing relatively big data sets, which requires to use of High-Performance Computing System (HPC-system). I am trying to run a de-novo assembly using spades-gcc/3.10.1. However, the run has failed with the message (You need approx. 251.691GB of free RAM to assemble your dataset). Is there any way to ask for using less RAM? Or
how to reduce the requirement by tweaking this job?
What is the limit of RAM you can use on that system? The default RAM limit for spades is set to 250GB, but you can set it higher using the -m parameter. If 250GB really is your hard limit on the system, you may try to select another kmer length. This will have the largest impact on memory consumption. You may also save some buffer memory if you use less threads. Depending on your dataset, downsampling your data may do the trick too. Last but not least, there's more memory efficient assemblers than spades out there, if you're flexible.
Thanks Simon! I am open to using anything that supports my data. My cleaned and trimmed data size is about 260Gb as fastq files. I also tried velvet, which is not supporting as well in my system. Is there assembly software that can use less RAM (e.g 120 RAM)?
I'd need more information on the type of data you want to assemble here to give you an advice for an efficient assembler. If working on microbial data, Skesa is probably the method of choice as it can scale the RAM consumption according to your limitations. Having 260GB of sequencing data, I guess you're dealing with either a large genome or a metagenomic sample. In either case, I'd advise you to check your real RAM limitations on that system first and set the -m parameter of spades to the highest value available.
- Badges
- Science topic