Hi Nikita,

It really depends on exactly what sort of analysis you are doing on your VCF-formatted data. Is memory or runtime the issue?

If memory is the issue, you can index your file using tabix, which (if your application supports it) can help with accessing data without having to read the entire file into memory. If you are using something like read.table() in R, you might also consider traversing the data line by line (i.e. without putting the entire file into memory).

If runtime is the main issue, if you can analyze the file in chunks that is of course an option which would allow you to take advantage of parallel processing. I have generally found it more efficient to work in Python, but YMMV depending on the application (e.g., does it require 'reinventing the wheel'? If so it might not be worth the effort). Again, it really depends on exactly what you are trying to do whether or not there is a more efficient way of going about it.

Tyler

I want to expand the vcf and remove certain elements from the file as a part of preprocessing for further analysis. @Tyler Chafin

Unless you are filtering in some way that will require information from multiple rows, there shouldn't be any reason then that you couldn't process the file line-by-line, without needing to read the entire thing in to memory. Since it sounds like you are essentially writing a new VCF file at the same time, you should probably write that line-by-line as well so you don't end up having to store all of the new contents in memory as well.

Depending on exactly what sort of information you are extracting, bcftools/ vcftools might be able to do it.

Tyler Chafin Thank you for your suggestions. Following are the codes i used for normal vcf file;

evcf