Planned data release: V14 #432

jharenza · 2020-01-13T19:22:28Z

What data file(s) does this issue pertain to?

additional files to be generated

What release are you using?

v13

Put a link to the relevant analysis here.

#431

Put your question or report your issue here.

Add matrix of yes/no presence of TP53, SUFU, and PTCH1 germline variants for Medulloblastoma SHH TP53 subtypes #378 medulloblastoma subtyping
Update GISTIC file using consensus CNV file Updated analysis: rerun GISTIC with consensus SEG file #453
add BS_NMJPX8VQ to CNVkit seg file

jaclyn-taroni · 2020-01-18T12:58:04Z

Consensus SEG file generated as part of #441 - analyses/copy_number_consensus_call/results/pbta-cnv-consensus.seg.

cansavvy · 2020-01-21T16:03:25Z

The updated Lancet WXS BED file downstream changes have been propagated and now there are updated files on S3 that should be included in the v14 release:

The updated/renamed intersection BED files:
- https://open-pbta.s3.amazonaws.com/data/snv-consensus-bed/intersect_cds_lancet_strelka_mutect_WGS.bed
- https://open-pbta.s3.amazonaws.com/data/snv-consensus-bed/intersect_cds_lancet_WXS.bed
The updated snv consensus zip file snv-consensus-20200121.zip (Note the only file of the three files within here that has changed is pbta-snv-consensus_snv_tmb_coding_only.tsv, but for consistency's sake, I uploaded the data in the same way and format that the other editions were.
- https://open-pbta.s3.amazonaws.com/data/snv-consensus/snv-consensus-20200121.zip

jaclyn-taroni · 2020-01-27T23:32:01Z

New fusion summary files as of #478: https://github.com/AlexsLemonade/OpenPBTA-analysis/tree/master/analyses/fusion-summary/results

jharenza · 2020-01-28T14:59:02Z

The updated Lancet WXS BED file downstream changes have been propagated and now there are updated files on S3 that should be included in the v14 release:

The updated/renamed intersection BED files:

https://open-pbta.s3.amazonaws.com/data/snv-consensus-bed/intersect_cds_lancet_strelka_mutect_WGS.bed

https://open-pbta.s3.amazonaws.com/data/snv-consensus-bed/intersect_cds_lancet_WXS.bed

The updated snv consensus zip file snv-consensus-20200121.zip (Note the only file of the three files within here that has changed is pbta-snv-consensus_snv_tmb_coding_only.tsv, but for consistency's sake, I uploaded the data in the same way and format that the other editions were.

https://open-pbta.s3.amazonaws.com/data/snv-consensus/snv-consensus-20200121.zip

@cansavvy I can download the zip file, but not the first two - getting access denied.

jharenza · 2020-01-28T15:06:17Z

cansavvy · 2020-01-28T15:21:35Z

Sorry about that @jharenza, apparently after I updated them I didn't realize they wouldn't be public access unless I set it that way. I've set them both to public. You should be good.

jharenza · 2020-01-28T15:23:09Z

all good, thanks @cansavvy !

jashapiro · 2020-01-28T16:38:34Z

I'm going to suggest that cnv-consensus.tsv be removed from the data download. It is present in copy_number_consensus_call/results and I don't expect much use of it, now that pbta-cnv-consensus.seg.gz is available. (I also suggest using the .gz file for the download as it is much smaller!)

jharenza · 2020-01-28T18:08:21Z

Not going to include germline data in this release. See #451 #431 .

jharenza · 2020-01-28T23:05:40Z

@jashapiro - are we using cnv-consensus.tsv for any current analyses? Eg - gain/loss info, or is that all being derived from GISTIC at this point?

jaclyn-taroni · 2020-01-28T23:16:36Z

@jharenza there's a module for mapping status to gene symbols, cytobands, etc. focal-cn-file-preparation. It is being updated to use the consensus SEG file in #479.

jashapiro · 2020-01-30T21:24:16Z

@jharenza before finalizing gistic-results-consensus.zip can you check that the base folder enclosed has the same name? The current pbta-cnv-cnvkit-gistic.zip has a base folder named 2019-12-10-gistic-results-cnvkit which is confusing and seems like a recipe for broken code on updates.

jharenza · 2020-01-30T21:51:46Z

@jashapiro yep! i attached the files here with dates to keep track of a version since we were running them a few times, but was planning to name them similar to the current naming scheme:
pbta-cnv-cnvkit-gistic.zip
pbta-cnv-consensus-gistic.zip
Does that work?

jashapiro · 2020-01-30T21:53:20Z

Yes, that is fine. It was just surprising to me that when the unzipped the folder name was not the same as the zip file.

jharenza · 2020-01-30T21:56:05Z

Ahh, I see what you're saying - yep, will fix both of those to be generic without dates.

jharenza · 2020-02-03T15:09:56Z

Hi @cansavvy and @jaclyn-taroni - these files have changed names since the last release - do you want me to change to what is current or keep named how we had them?
New file and name for intersect_exon_lancet_strelka_mutect_WGS.bed -> intersect_cds_lancet_strelka_mutect_WGS.bed

No new file, keep old name? intersect_exon_WXS.bed
No new file, keep old name? intersect_strelka_mutect_WGS.bed
New file: intersect_cds_lancet_WXS.bed
New file and name for pbta-snv-consensus-mutation-tmb-coding.tsv -> pbta-snv-consensus_snv_tmb_coding_only.tsv

Is this correct?

cansavvy · 2020-02-03T15:14:59Z

Yes, this is correct. I will change the code to get rid of the underscores for future releases. intersect_exon_WXS.bed although it didn't change, I would still change the filename to better reflect its contents. (I mistakenly called it exon in its filename originally, but it really is based on coding sequences.)

jharenza · 2020-02-03T15:15:59Z

So for that one, should it be intersect_exon_WXS.bed -> intersect_cds_WXS.bed?

jaclyn-taroni · 2020-02-03T15:16:27Z

Can we change the new file pbta-snv-consensus_snv_tmb_coding_only.tsv to use underscores pbta-snv-consensus-mutation-tmb-coding.tsv so we don't have breaking changes please?

jharenza · 2020-02-03T15:19:26Z

@jaclyn-taroni do you mean hyphens or use all underscores? Current filename is the mixed one. Old was only hyphens. Please send the exact name you want :)

jaclyn-taroni · 2020-02-03T15:20:22Z

Sorry that should have been to not use underscores. The file name should be pbta-snv-consensus-mutation-tmb-coding.tsv.

jharenza · 2020-02-03T15:21:00Z

OK, sure, and get rid of the only to keep it as it was?

jaclyn-taroni · 2020-02-03T15:21:46Z

Correct, I want to keep it as it was to minimize breaking changes.

jharenza · 2020-02-03T20:37:35Z

Also, updating pbta-histologies.tsv to include new molecular_subtype for #249 and #251 tumors.

cansavvy · 2020-02-04T14:45:22Z

@jharenza

So for that one, should it be intersect_exon_WXS.bed -> intersect_cds_WXS.bed?

Sorry, I misunderstood/lost track of this. This file is no longer necessary, the only WXS bed should be intersect_cds_lancet_WXS.bed

@cansavvy

per @cansavvy [comment](#432 (comment))

jharenza · 2020-02-04T19:20:15Z

got it, thanks!

jharenza · 2020-02-05T20:00:12Z

TCGA lancet file, release notes, and md5 are updated on s3 now.

jharenza · 2020-02-06T01:21:52Z

I also had to update pbta-histologies.tsv - found an inconsistency when I created one sample's new broad histology. Bo will update in the AM.

jharenza · 2020-02-06T14:58:33Z

@cansavvy and @jaclyn-taroni - this is all ready now and downloads pass. sorry about that!

jaclyn-taroni · 2020-02-06T14:59:28Z

@jharenza can you post the expected checksum so I can make sure I have what we want to include in testing please?

jharenza · 2020-02-06T15:00:37Z

md5sum.txt

@cansavvy

* add v14 release docs -update `release-notes.md` -update `data-files-description.md` -update `data-formats.md` * Update download-data.sh add new folder for V14 to download scipt * remove intersect_cds_WXS.bed per @cansavvy [comment](#432 (comment)) * add intersect_cds_lancet.bed and description from @cansavvy [comments](#507 (comment)) * Update release-notes.md - add removal of polyA+stranded samples that were still in file in v13 * Update data-formats.md add more information on gistic output files, to replace PR [#456](#456) * Reorganize derived CN section and make formatting consistent * Add links to relevant subtyping modules * Update release-notes.md readme update for upcoming lancet MAF per issue [here](#512) * Update doc/release-notes.md yup, nice catch Co-Authored-By: Jaclyn Taroni <jaclyn.n.taroni@gmail.com> * Update release-notes.md fix embryonal broad histology Co-authored-by: Jaclyn Taroni <jaclyn.n.taroni@gmail.com>

jharenza · 2020-02-08T02:06:01Z

closed via #507

jharenza added data planned data release labels Jan 13, 2020

This was referenced Jan 18, 2020

Discussion: consensus CN files for downstream analyses #392

Closed

Updated analysis: rerun GISTIC with consensus SEG file #453

Closed

jharenza mentioned this issue Jan 23, 2020

Missing cnvkit data #472

Closed

jharenza mentioned this issue Jan 28, 2020

kallisto stranded file has extra column #474

Closed

jharenza mentioned this issue Jan 30, 2020

Updated analysis: run GISTIC with new CNVkit seg file #491

Closed

This was referenced Feb 2, 2020

Update CI subset files for v14 #496

Closed

Updated analysis: generate cytoband copy number status file for consumption #497

Closed

Updated analysis: rerun sex prediction from RNA-seq with v14 data #500

Open

jharenza mentioned this issue Feb 3, 2020

add v14 release docs #507

Merged

4 tasks

jharenza pushed a commit that referenced this issue Feb 4, 2020

remove intersect_cds_WXS.bed

4805825

per @cansavvy [comment](#432 (comment))

jaclyn-taroni mentioned this issue Feb 6, 2020

Updates required for v14 CI files #515

Merged

jharenza closed this as completed Feb 8, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Planned data release: V14 #432

Planned data release: V14 #432

jharenza commented Jan 13, 2020 •

edited

Loading

jaclyn-taroni commented Jan 18, 2020

cansavvy commented Jan 21, 2020

jaclyn-taroni commented Jan 27, 2020

jharenza commented Jan 28, 2020

jharenza commented Jan 28, 2020 •

edited

Loading

cansavvy commented Jan 28, 2020

jharenza commented Jan 28, 2020

jashapiro commented Jan 28, 2020

jharenza commented Jan 28, 2020

jharenza commented Jan 28, 2020

jaclyn-taroni commented Jan 28, 2020

jashapiro commented Jan 30, 2020

jharenza commented Jan 30, 2020

jashapiro commented Jan 30, 2020

jharenza commented Jan 30, 2020

jharenza commented Feb 3, 2020

cansavvy commented Feb 3, 2020

jharenza commented Feb 3, 2020

jaclyn-taroni commented Feb 3, 2020

jharenza commented Feb 3, 2020 •

edited

Loading

jaclyn-taroni commented Feb 3, 2020

jharenza commented Feb 3, 2020

jaclyn-taroni commented Feb 3, 2020

jharenza commented Feb 3, 2020

cansavvy commented Feb 4, 2020 •

edited

Loading

jharenza commented Feb 4, 2020

jharenza commented Feb 5, 2020

jharenza commented Feb 6, 2020

jharenza commented Feb 6, 2020

jaclyn-taroni commented Feb 6, 2020

jharenza commented Feb 6, 2020

jharenza commented Feb 8, 2020

Planned data release: V14 #432

Planned data release: V14 #432

Comments

jharenza commented Jan 13, 2020 • edited Loading

What data file(s) does this issue pertain to?

What release are you using?

Put a link to the relevant analysis here.

Put your question or report your issue here.

jaclyn-taroni commented Jan 18, 2020

cansavvy commented Jan 21, 2020

jaclyn-taroni commented Jan 27, 2020

jharenza commented Jan 28, 2020

jharenza commented Jan 28, 2020 • edited Loading

cansavvy commented Jan 28, 2020

jharenza commented Jan 28, 2020

jashapiro commented Jan 28, 2020

jharenza commented Jan 28, 2020

jharenza commented Jan 28, 2020

jaclyn-taroni commented Jan 28, 2020

jashapiro commented Jan 30, 2020

jharenza commented Jan 30, 2020

jashapiro commented Jan 30, 2020

jharenza commented Jan 30, 2020

jharenza commented Feb 3, 2020

cansavvy commented Feb 3, 2020

jharenza commented Feb 3, 2020

jaclyn-taroni commented Feb 3, 2020

jharenza commented Feb 3, 2020 • edited Loading

jaclyn-taroni commented Feb 3, 2020

jharenza commented Feb 3, 2020

jaclyn-taroni commented Feb 3, 2020

jharenza commented Feb 3, 2020

cansavvy commented Feb 4, 2020 • edited Loading

jharenza commented Feb 4, 2020

jharenza commented Feb 5, 2020

jharenza commented Feb 6, 2020

jharenza commented Feb 6, 2020

jaclyn-taroni commented Feb 6, 2020

jharenza commented Feb 6, 2020

jharenza commented Feb 8, 2020

jharenza commented Jan 13, 2020 •

edited

Loading

jharenza commented Jan 28, 2020 •

edited

Loading

jharenza commented Feb 3, 2020 •

edited

Loading

cansavvy commented Feb 4, 2020 •

edited

Loading