CNV consensus (7 of 6): Remove duplicated coordinates and NULLs

[nhatduongnn]

Purpose/implementation Section

Remove duplicated coordinates and NULLs in the result file

What GitHub issue does your pull request address?

#128

• The dependencies required to run the code in this pull request have been added to the project Dockerfile.
• This analysis has been added to continuous integration.
• This analysis is recorded in the table in analyses/README.md.

[jashapiro]

Hi @fingerfen. This looks good. I suggested sorting the coordinates before writing, and a minor reversion to fix, but that's about it.

Thank you for your contributions!

[nhatduongnn]

@jashapiro Thanks for catching the reversion. That was not on purpose. Thank you for your effort and quick responses. Also, since we are finished here, I just want to revisit the 3rd (3 of n) PR that I made. Recall you wanted to see if Bedtools Subtract could be a quicker alternative to my Python script. Do you think we should go back at this point to take a look and make changes to that? If I recall correctly, that would be the only part left to improve in this pipeline.

[jashapiro]

This looks good, with just a fix to potential sorting error to add (and a format suggestion). The sorting fix would mean rerunning the last step though to generate new output.

On the topic of updating step 3, yes, I think that is worth doing (in a separate PR). I looked back at some test code I had written, and it looks like you should be able to use:

"bedtools subtract -N -a {input.bedfile} -b {input.bad_list} -f 0.5 > {output.filtered}"

in place of your get_rid_bad_segments.py script, assuming the input files are actually bedfiles, which should just mean replacing space delimiters with tabs in the intermediate files (making them true bedfiles rather than pseudo-bed).

[jaclyn-taroni]

@fingerfen - it looks like the last addition before you merged in the master branch (7b500c4) failed on the copy number consensus step: https://circleci.com/gh/AlexsLemonade/OpenPBTA-analysis/2390

Here's the full output: build_2390_step_116_container_0.txt

[nhatduongnn]

I just tested the line break fix that @jashapiro just did and it worked on my local. I am not sure why it is failing on CI at the momment.

[jashapiro]

I just tested the line break fix that @jashapiro just did and it worked on my local. I am not sure why it is failing on CI at the momment.

It didn't really fail... I just merged in master before it finished running.

[jashapiro]

@fingerfen I am just waiting on the update to cnv_consensus.tsv to approve this and get it merged in.

[jashapiro]

@fingerfen I'm guessing the last commit (f72938a) was to respond to my request for the updated version, but that one doesn't seem to be the latest... it still uses sample instead of Biospecimen; I'm guessing it is just because you didn't rerun the merge step that sets the header names.

[nhatduongnn]

@jashapiro I apologize. I didn't re-run the merging step. I just re-run it and this new file should have Biospecimen instead of sample

[jashapiro]

Looks good! We'll merge as soon as tests finish again.