Map GTF to memory

[zhuchcn]

Description

Instead of reading the entire GTF file into memory, we now create a map of gene/transcript ID and the location in the file (as a pointer) and only keep the map in memory. This reduces memory usage of callVariant significantly. Peak memory of callVairant is now a little over 8GB. For more complex samples, 12 GB of memory should be save. This means we can run more samples per node simultaneously (probably double it).

One thing to note is the index files are now different. There are not two additional files *_gene.idx and *_tx.idx. Also the GTF file now has to be uncompressed.

8GB peak memory usage is still kind of big but there is still room for optimization. The next big part of memroy usage is the genome, which is now still read into memory, which can be optimized into a memory-mapped file, too.

Closes #371

Checklist

• This PR does NOT contain PHI or germline genetic data. A repo may need to be deleted if such data is uploaded. Disclosing PHI is a major problem.
• This PR does NOT contain molecular files, compressed files, output files such as images (e.g. .png, .jpeg), .pdf, .RData, .xlsx, .doc, .ppt, or other non-plain-text files. To automatically exclude such files using a .gitignore file, see here for example.
• I have read the code review guidelines and the code review best practice on GitHub check-list.
• The name of the branch is meaningful and well formatted following the standards, using [AD_username (or 5 letters of AD if AD is too long)]-[brief_description_of_branch].
• I have added the major changes included in this pull request to the CHANGELOG.md under the next release version or unreleased, and updated the date.
• All test cases passed locally.

[lydiayliu]

I wonder how it would be for the worst case samples that I was seeing that was 32G+ in memory usage (hence had to switch to running only 1 sample on 1 F32...). Would it be a -8G decrease or actually halving?

[zhuchcn]

In general the memory usage should reduced for 8GB. For the 32 GB case, did you see it before or after #739 ? This fix should have reduced a big amount of memory

[lydiayliu]

sorry what are the expect outputs from generateIndex now after this update? Is the vignette updated?

[zhuchcn]

The output of generateIndex is changed but that's not something that users should worry about. As long as the correct --index-dir is given, all index files should be loaded correctly. We didn't mention what files are generated by generateIndex in vignette, but might be good to update the doc page for generateIndex itself. I'll push to #762 shortly