-
Notifications
You must be signed in to change notification settings - Fork 4
Appendix: CPU, Wall Time and Max RAM
Yaobo Xu edited this page Aug 20, 2019
·
3 revisions
cgpRna container (v2.3.3) was run using a human RNA-Seq sample (127 million raw read pairs) with 16/8/4 cores to evaluate its resource requirements. The stats are below. We used the three scripts in Use the Docker container.
Based on the stats, we recommend using 4 cores for the mapping_qc_stat script, a single core for bigwig_htseq script and 8 cores for the infuse pipeline script. However, the infuse pipeline is able to take advantage of more cores.
| Step name | 16core | 8core | 4core | |
|---|---|---|---|---|
| star_mapping_prepare | CPU | 99% | 99% | 99% |
| Wall time (hh:mm:ss) | 00:20:05 | 00:19:57 | 00:19:24 | |
| Max resident size (kbytes) | 145,004 | 145,016 | 144,784 | |
| star_mapping_star | CPU | 281% | 301% | 268% |
| Wall time (hh:mm:ss) | 01:17:01 | 01:09:04 | 01:17:05 | |
| Max resident size (kbytes) | 36,665,756 | 34,778,312 | 34,775,952 | |
| bammarkduplicates2 | CPU | 418% | 346% | 263% |
| Wall time (hh:mm:ss) | 00:16:40 | 00:19:14 | 00:25:21 | |
| Max resident size (kbytes) | 490,732 | 486,676 | 485,552 | |
| bamindex | CPU | 99% | 99% | 99% |
| Wall time (hh:mm:ss) | 00:10:55 | 00:10:42 | 00:10:49 | |
| Max resident size (kbytes) | 141,116 | 141,100 | 141,312 | |
| bam_stats_sampleBam | CPU | 99% | 99% | 99% |
| Wall time (hh:mm:ss) | 00:05:03 | 00:04:48 | 00:04:57 | |
| Max resident size (kbytes) | 9,096 | 9,096 | 8,972 | |
| bam_stats_transcriptomeBam | CPU | 99% | 99% | 99% |
| Wall time (hh:mm:ss) | 00:10:27 | 00:10:20 | 00:10:35 | |
| Max resident size (kbytes) | 22,800 | 22,728 | 22,804 | |
| split_bam | CPU | 99% | 99% | 99% |
| Wall time (hh:mm:ss) | 00:53:22 | 00:50:25 | 00:52:35 | |
| Max resident size (kbytes) | 65,656 | 65,536 | 65,632 | |
| geneBody_coverage | CPU | 100% | 100% | 100% |
| Wall time (hh:mm:ss) | 00:20:12 | 00:19:22 | 00:20:16 | |
| Max resident size (kbytes) | 113,016 | 113,072 | 113,060 | |
| read_distribution | CPU | 100% | 100% | 100% |
| Wall time (hh:mm:ss) | 00:19:22 | 00:18:47 | 00:19:54 | |
| Max resident size (kbytes) | 955,236 | 955,068 | 955,300 | |
| process_qcstats | CPU | 100% | 100% | 100% |
| Wall time (hh:mm:ss) | 00:00:00 | 00:00:00 | 00:00:00 | |
| Max resident size (kbytes) | 38,936 | 38,824 | 38,592 | |
| paste_stats | CPU | 0% | 0% | ?% |
| Wall time (hh:mm:ss) | 00:00:00 | 00:00:00 | 00:00:00 | |
| Max resident size (kbytes) | 1,968 | 1,852 | 1,836 | |
| Total run time | 03:53:06 | 03:42:39 | 04:00:56 | |
| Single threaded time | 02:19:26 | 02:14:21 | 02:18:30 |
| Step name | 16core | 8core | 4core | |
|---|---|---|---|---|
| bamToBw_bamToBw.time | CPU | 458% | 361% | 274% |
| Wall time (hh:mm:ss) | 00:10:35 | 00:13:06 | 00:17:08 | |
| Max resident size (kbytes) | 182,784 | 119,132 | 119,028 | |
| bamToBw_generateBw.time | CPU | 99% | 99% | 99% |
| Wall time (hh:mm:ss) | 00:01:02 | 00:01:03 | 00:01:04 | |
| Max resident size (kbytes) | 189,360 | 191,088 | 189,304 | |
| bamcollate2.time | CPU | 99% | 99% | 99% |
| Wall time (hh:mm:ss) | 00:18:38 | 00:18:28 | 00:19:03 | |
| Max resident size (kbytes) | 246,492 | 246,432 | 246,668 | |
| htset-count.time | CPU | 100% | 100% | 100% |
| Wall time (hh:mm:ss) | 02:55:01 | 02:49:29 | 02:57:27 | |
| Max resident size (kbytes) | 202,472 | 202,496 | 202,268 | |
| Total run time | 3:25:16 | 3:22:06 | 3:34:42 | |
| Single threaded time | 03:14:41 | 03:09:00 | 03:17:34 |
| Step name | 16core | 8core | 4core | |
|---|---|---|---|---|
| tophat_fusion_bamtofastq.time | CPU | 99% | 95% | 99% |
| Wall time (hh:mm:ss) | 00:18:04 | 00:19:06 | 00:17:42 | |
| Max resident size (kbytes) | 145,932 | 146,016 | 145,932 | |
| tophat_fusion_tophatfusion.time | CPU | 880% | 524% | 315% |
| Wall time (hh:mm:ss) | 05:25:23 | 06:15:08 | 09:25:52 | |
| Max resident size (kbytes) | 13,514,432 | 8,815,976 | 6,402,420 | |
| tophat_fusion_split.time | CPU | 37% | 35% | 42% |
| Wall time (hh:mm:ss) | 00:00:01 | 00:00:01 | 00:00:01 | |
| Max resident size (kbytes) | 31,316 | 31,160 | 31,316 | |
| tophat_fusion_tophatpost.time | CPU | 273% | 234% | 197% |
| Wall time (hh:mm:ss) | 02:47:47 | 03:11:08 | 03:51:30 | |
| Max resident size (kbytes) | 12,826,512 | 12,826,472 | 12,826,052 | |
| tophat_fusion_filter.time | CPU | 98% | 69% | 86% |
| Wall time (hh:mm:ss) | 00:00:08 | 00:00:13 | 00:00:10 | |
| Max resident size (kbytes) | 31,112 | 31,072 | 31,084 | |
| tophat_fusion_strand.time | CPU | 94% | 93% | 93% |
| Wall time (hh:mm:ss) | 00:00:02 | 00:00:02 | 00:00:02 | |
| Max resident size (kbytes) | 31,296 | 31,340 | 31,192 | |
| defuse_fusion_prepare.time | CPU | 98% | 97% | 98% |
| Wall time (hh:mm:ss) | 00:08:44 | 00:09:12 | 00:09:07 | |
| Max resident size (kbytes) | 144,180 | 143,800 | 143,888 | |
| defuse_fusion_merge.time | CPU | 100% | 99% | 100% |
| Wall time (hh:mm:ss) | 00:00:00 | 00:00.2 | 00:00:00 | |
| Max resident size (kbytes) | 29,716 | 29,480 | 29,788 | |
| defuse_fusion_defuse.time | CPU | 1159% | 668% | 370% |
| Wall time (hh:mm:ss) | 06:15:08 | 10:18:23 | 18:41:36 | |
| Max resident size (kbytes) | 6,124,448 | 6,124,588 | 6,124,900 | |
| defuse_fusion_filter.time | CPU | 99% | 99% | 99% |
| Wall time (hh:mm:ss) | 00:30:59 | 00:31:58 | 00:32:54 | |
| Max resident size (kbytes) | 29,844 | 29,572 | 29,704 | |
| defuse_filters.time | CPU | 98% | 98% | 99% |
| Wall time (hh:mm:ss) | 00:00:00 | 00:00:00 | 00:00:00 | |
| Max resident size (kbytes) | 17,924 | 17,872 | 17,732 | |
| star_fusion_prepare.time | CPU | 95% | 99% | 95% |
| Wall time (hh:mm:ss) | 00:18:14 | 00:18:02 | 19:00.4 | |
| Max resident size (kbytes) | 145,876 | 145,764 | 145,612 | |
| star_fusion_star.time | CPU | 379% | 369% | 323% |
| Wall time (hh:mm:ss) | 00:30:49 | 00:31:17 | 00:35:21 | |
| Max resident size (kbytes) | 35,415,304 | 34,777,956 | 34,775,664 | |
| star_fusion_starfusion.time | CPU | 99% | 99% | 99% |
| Wall time (hh:mm:ss) | 00:20:34 | 00:20:14 | 00:21:30 | |
| Max resident size (kbytes) | 5,120,724 | 5,121,156 | 5,120,948 | |
| star_fusion_filter.time | CPU | 98% | 99% | 99% |
| Wall time (hh:mm:ss) | 00:01:15 | 00:01:18 | 00:01:21 | |
| Max resident size (kbytes) | 29,952 | 29,808 | 29,740 | |
| compare_overlapping_fusions.time | CPU | 81% | 80% | 91% |
| Wall time (hh:mm:ss) | 00:01:01 | 00:01:04 | 00:00:58 | |
| Max resident size (kbytes) | 70,472 | 70,424 | 70,560 | |
| Total run time | 16:38:12 | 21:57:06 | 34:17:06 | |
| Single threaded time | 1:39:05 | 1:41:11 | 1:42:47 |