Non-Ensembl GTF testing #110

hoelzer · 2020-12-08T18:34:27Z

I'm testing the workflow for genome/annotation that are not derived from Ensembl. But still, the GTF follows the Ensembl structure (gene, transcript, exon).

The test worked surprisingly well :) I just run into a general issue with a plotting function that failed because after p-value filtering fewer genes were remaining than defined by ntop.

Please leave this PR open: I will check the output and improve some of the scripts to plot still meaningful output even if we can not extract information such as gene_biotype from the GTF. Also, we don't want to link to Ensembl with a GTF that does not have matching IDs.

…e size

Marie lataretu/issue109

hoelzer · 2020-12-16T17:18:35Z

@MarieLataretu actually I think that's all for now. So this branch works with my custom GTF-styled annotation file. Sure, there can be more generalized and such, but I think it's fine for now. If you agree, we can merge (how? should we first merge the current master into that branch? o_O)

MarieLataretu · 2021-01-05T21:55:45Z

bin/deseq2.R

 plot.heatmap.top_fc <- function(out.dir, resFold, trsf_data, trsf_type, ntop, pcutoff='', samples.info=df.samples.info, genes.info=df.gene.anno) {
-  selected.ensembl.ids <- row.names(resFold[order(resFold$log2FoldChange, decreasing=TRUE), ])[1:ntop]
+  # check how many elements are in the dataframe
+  # if less elements are in the dataframe than selected by ntop, reduce ntop
+  if (length(resFold$log2FoldChange) < ntop) {
+    ntop = length(resFold$log2FoldChange)
+  }
+  if (ntop > 1) {
+    selected.ensembl.ids <- row.names(resFold[order(resFold$log2FoldChange, decreasing=TRUE), ])[1:ntop]


I fixed that also in #123 ☺️

I fixed that also in #123

:D

MarieLataretu · 2021-01-06T10:54:19Z

I just run the test profile (so normal Ensembl annotation):

fixed a small bug in the reporting tools ruby script
renamed the refactored reporting tools table to ..._extended.html, because it's more informative
deseq2 tables look good 👍

non-Ensembl test is scheduled for today.

(how? should we first merge the current master into that branch? o_O)

My first guess would be to merge this branch into master (there were no changes in the master in this module).
We should be careful thought, if we merge #123 into the master before. Maybe we should even merge #123 into this branch before and test, because there are some changes in the annotation preparation

hoelzer · 2021-01-06T13:29:38Z

(how? should we first merge the current master into that branch? o_O)

My first guess would be to merge this branch into master (there were no changes in the master in this module).
We should be careful thought, if we merge #123 into the master before. Maybe we should even merge #123 into this branch before and test, because there are some changes in the annotation preparation

Ah I see. Just checked #123 and agree, we could merge the changes from #123 into this branch #110, check if everything still works (I can also do a test again with my customized GTF file) and then merge into master?

MarieLataretu · 2021-01-07T14:47:15Z

So this branch works with my custom GTF-styled annotation file.

What is the difference to a normal Ensembl annotation? Right now we still expect 'geneandexonwithgene_id`s in the description, do we?

hoelzer · 2021-01-07T15:22:01Z

So this branch works with my custom GTF-styled annotation file.

What is the difference to a normal Ensembl annotation? Right now we still expect 'geneandexonwithgene_id`s in the description, do we?

yep, the main difference I checked was that there is ne ENSxxxxxxx ID. So my structure of the GTF is still a valid hierarchical GTF and looks like this:

contig_19_segment0_pilon_pilon	ID	gene	17	11637	.	-	.	gene_id "IDG1"; transcript_id "IDT1.1"; annotation_tag "hybrid_flye_pilon_0"; hmm_matches "NA"; homologies "NA"; gene_name "IDG1";
contig_19_segment0_pilon_pilon	ID	transcript	17	11637	.	-	.	gene_id "IDG1"; transcript_id "IDT1.1"; annotation_tag "hybrid_flye_pilon_0"; hmm_matches "NA"; homologies "NA"; 
contig_19_segment0_pilon_pilon	ID	exon	17	6319	.	-	.	gene_id "IDG1"; transcript_id "IDT1.1"; exon_id "IDE1.1.1"; 
contig_19_segment0_pilon_pilon	ID	exon	6474	11637	.	-	.	gene_id "IDG1"; transcript_id "IDT1.1"; exon_id "IDE1.1.2";

So I think that at least any valid GTF formatted file with a gene and exon feature and a gene_id in the descr column should work.

MarieLataretu · 2021-01-07T15:43:16Z

Ah I see. Just checked #123 and agree, we could merge the changes from #123 into this branch #110, check if everything still works (I can also do a test again with my customized GTF file) and then merge into master?

We now can test on #123, because git is confusing sometimes

…ab/rnaflow into MarieLataretu/issue116

Marie lataretu/issue116

MarieLataretu · 2021-01-12T14:26:40Z

These three commands run without errors, the reprotingTools tables look fine and random checks of the output also 👍

nextflow run main.nf -w work -profile test,local,conda -resume --max_cores 3 --softlink_results --featurecounts_additional_params '-t exon -g transcript_id' --feature_id_type 'ensembl_transcript_id'

nextflow run main.nf -w work -profile local,conda,test -resume --max_cores 3 --softlink_results --featurecounts_additional_params '-t exon -g exon_id' --feature_id_type 'ensembl_exon_id' --output results/exon

nextflow run main.nf -w work -profile local,conda,test -resume --max_cores 3 --softlink_results

From my side we can merge into master!

hoelzer · 2021-01-12T14:30:56Z

yeah! I scrolled the changes again - from my side please merge! Then we also do a new release?

And should we reply to Ahmed (#116) again? I think this issue thread started the whole transcript/exon input change

MarieLataretu · 2021-01-12T14:48:04Z

yeah! I scrolled the changes again - from my side please merge! Then we also do a new release?

Okay! Looks like a minor release to me.

And should we reply to Ahmed (#116) again? I think this issue thread started the whole transcript/exon input change

Yes, at least the non-Ensembl part should work!

hoelzer added 3 commits December 8, 2020 19:31

test GTF not from ensembl, fix plot bug due to smaller input datafram…

4d15c6f

…e size

add description to extended CSV/EXCEL table

185fb50

Merge pull request #114 from hoelzer-lab/MarieLataretu/issue109

dfad754

Marie lataretu/issue109

hoelzer mentioned this pull request Dec 13, 2020

Extend final CSV/XSLX tabe #117

Open

hoelzer added 4 commits December 13, 2020 14:06

add more information to the final csv table

635abd3

improve check for empty data frame

c311c64

fix if gene_name is NA

b09b0de

fix refac reportingtools table bug

c002f48

hoelzer requested a review from MarieLataretu December 16, 2020 17:17

MarieLataretu added 4 commits December 18, 2020 13:33

new param for id type

a1bc68f

customized annotation prep

46ec63c

rename variabl

f4872fa

rename variable

7acd6b4

MarieLataretu reviewed Jan 5, 2021

View reviewed changes

fixed bug

be44bcf

hoelzer mentioned this pull request Jan 6, 2021

Marie lataretu/issue116 #123

Merged

renamed variable

f1bbe24

Merge branch 'nonmodel' into MarieLataretu/issue116

23f0b6f

MarieLataretu and others added 6 commits January 7, 2021 16:45

fixed bug

e0e2ab4

add golden hamster to reporting tools table extension

9db971a

unique details, transcript info

f83d56c

fix reportingtools html for transcripts

84e3007

id sanity check

79d946c

Merge branch 'MarieLataretu/issue116' of https://github.com/hoelzer-l…

1faab14

…ab/rnaflow into MarieLataretu/issue116

hoelzer and others added 2 commits January 9, 2021 15:23

fix html extend table for exons

f6b7573

Merge pull request #123 from hoelzer-lab/MarieLataretu/issue116

7bfdeea

Marie lataretu/issue116

MarieLataretu self-assigned this Jan 12, 2021

Merge branch 'master' into nonmodel

19dbc25

MarieLataretu merged commit c3ee7b3 into master Jan 12, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Non-Ensembl GTF testing #110

Non-Ensembl GTF testing #110

hoelzer commented Dec 8, 2020

hoelzer commented Dec 16, 2020

MarieLataretu Jan 5, 2021

hoelzer Jan 6, 2021

MarieLataretu commented Jan 6, 2021 •

edited

Loading

hoelzer commented Jan 6, 2021

MarieLataretu commented Jan 7, 2021

hoelzer commented Jan 7, 2021

MarieLataretu commented Jan 7, 2021

MarieLataretu commented Jan 12, 2021

hoelzer commented Jan 12, 2021

MarieLataretu commented Jan 12, 2021

Non-Ensembl GTF testing #110

Non-Ensembl GTF testing #110

Conversation

hoelzer commented Dec 8, 2020

hoelzer commented Dec 16, 2020

MarieLataretu Jan 5, 2021

Choose a reason for hiding this comment

hoelzer Jan 6, 2021

Choose a reason for hiding this comment

MarieLataretu commented Jan 6, 2021 • edited Loading

hoelzer commented Jan 6, 2021

MarieLataretu commented Jan 7, 2021

hoelzer commented Jan 7, 2021

MarieLataretu commented Jan 7, 2021

MarieLataretu commented Jan 12, 2021

hoelzer commented Jan 12, 2021

MarieLataretu commented Jan 12, 2021

MarieLataretu commented Jan 6, 2021 •

edited

Loading