Code

Download pipelines for Tower#2247

Code

Merged

MatthiasZepper merged 43 commits intonf-core:devnf-core/tools:devfrom MatthiasZepper:DownloadForTowerMatthiasZepper/tools:DownloadForTowerCopy head branch name to clipboardJun 1, 2023

+1,340-579

Commits

Commits on May 25, 2023

• Add -t / --tower option to 'nf-core download'.

  MatthiasZepper

  committed
  f9d07ef

  Copy full SHA for f9d07ef

• Intermediate commit

  MatthiasZepper

  committed
  e94dce0

  Copy full SHA for e94dce0

• Implement logic for the Tower download in DownloadWorkflow:download_workflow()

  MatthiasZepper

  committed
  cba336b

  Copy full SHA for cba336b

• Extend ModulesRepo:setup_local_repo() with a cache_only bool, so we can use ModuleRepo as superclass to the new WorkflowRepo.

  MatthiasZepper

  committed
  15c0588

  Copy full SHA for 15c0588

• Create WorkflowRepo subclass of ModuleRepo and initialise local clone.

  MatthiasZepper

  committed
  2be88c4

  Copy full SHA for 2be88c4

• TypeError: HEAD is a detached symbolic reference as it points to ...

  MatthiasZepper

  committed
  d72169d

  Copy full SHA for d72169d

• Split history ./modules/modules_repo.py to synced_repo.py

  MatthiasZepper

  committed
  a6b3492

  Copy full SHA for a6b3492

• Split history ./modules/modules_repo.py to synced_repo.py

  MatthiasZepper

  committed
  8984a63

  Copy full SHA for 8984a63

• Split history ./modules/modules_repo.py to synced_repo.py

  MatthiasZepper

  committed
  33d0381

  Copy full SHA for 33d0381

• Duplication of ModulesRepo to SyncedRepo done.

  MatthiasZepper

  committed
  caef187

  Copy full SHA for caef187

• Strip ModulesRepo class of the methods moved to new superclass.

  MatthiasZepper

  committed
  2367ae7

  Copy full SHA for 2367ae7

• Rebase to current dev.

  MatthiasZepper

  committed
  f5f0df2

  Copy full SHA for f5f0df2

• Local caching of the repo works now.

  MatthiasZepper

  committed
  f852159

  Copy full SHA for f852159

• Started implementing the config download.

  MatthiasZepper

  committed
  af4754e

  Copy full SHA for af4754e

• Started to implement the multiple revision selection for the Tower download.

  MatthiasZepper

  committed
  3bc97c5

  Copy full SHA for 3bc97c5

• Rewrite get_revision_hash() function to accomodate multiple revisions.

  MatthiasZepper

  committed
  e17a8e9

  Copy full SHA for e17a8e9

• The 2nd revivial of the config choice. Now available for archives with only one revision.

  MatthiasZepper

  committed
  ecaabf8

  Copy full SHA for ecaabf8

• Inclusion of the revision in the output file name is problematic with the new ability to download multiple revisions at once. This resulted in loooooooong filenames.

  MatthiasZepper

  committed
  6d04ec8

  Copy full SHA for 6d04ec8

• Allow multiple instances of the -r argument. Needed for scripted download. Ultimately, this now means that I also have to implement multiple version downloads for the classic download. Just downloa…

  Show description for 7642e4f
  MatthiasZepper

  committed
  7642e4f

  Copy full SHA for 7642e4f

• Finished updating the prompts for the dialogues.

  MatthiasZepper

  committed
  7f93edb

  Copy full SHA for 7f93edb

• Converted the self.wf_download_url into a dict.

  MatthiasZepper

  committed
  12bf942

  Copy full SHA for 12bf942

• Enable multi-revision classic download.

  MatthiasZepper

  committed
  2ff62f3

  Copy full SHA for 2ff62f3

• Small tweaks to ensure that tools doesn't bail out if there is no symlink from singularity to apptainer on the system.

  MatthiasZepper

  committed
  986f791

  Copy full SHA for 986f791

• Initialise the Git repo clone of the workflow.

  MatthiasZepper

  committed
  6f95829

  Copy full SHA for 6f95829

• WorkflowRepo attributes and functions.

  MatthiasZepper

  committed
  760fcaa

  Copy full SHA for 760fcaa

• Finished the Tower download branch.

  MatthiasZepper

  committed
  c381776

  Copy full SHA for c381776

• Minor tweaks to the container download functionality.

  MatthiasZepper

  committed
  526a26e

  Copy full SHA for 526a26e

• Updating docs and changelog, fixing linting errors.

  MatthiasZepper

  committed
  f4b9e67

  Copy full SHA for f4b9e67

• Hopefully fixed the existing tests. New ones still need to be written.

  MatthiasZepper

  committed
  2bf14bd

  Copy full SHA for 2bf14bd

• Refactor the CLI commands for the Singularity Cache Dir

  MatthiasZepper

  committed
  8de588e

  Copy full SHA for 8de588e

• Readme updates for the new remote Singularity cache feature.

  MatthiasZepper

  committed
  d729bde

  Copy full SHA for d729bde

• Add interactive check in retry for parsing the index.

  MatthiasZepper

  committed
  0f58c29

  Copy full SHA for 0f58c29

• Incorporating some suggestions by @mashehu.

  MatthiasZepper

  committed
  6294d74

  Copy full SHA for 6294d74

• Apply suggestions from code review @mashehu

  Show description for 8d327a4
  

  MatthiasZepper

  and

  mashehu

  committed
  8d327a4

  Copy full SHA for 8d327a4

• Writing additional tests for the --tower download functionality.

  MatthiasZepper

  committed
  340c519

  Copy full SHA for 340c519

• Move alterations from Version 2.8 (which this PR didn't make anymore) to Version 2.9dev.

  MatthiasZepper

  committed
  f599237

  Copy full SHA for f599237

• Adding the info about remote containers to the summary log rather than showing it separately.

  MatthiasZepper

  committed
  2518a4b

  Copy full SHA for 2518a4b

• Moved the notification about remote containers to summary_log.

  MatthiasZepper

  committed
  4f390be

  Copy full SHA for 4f390be

• Apply suggestions from code review

  Show description for f8e5068
  

  MatthiasZepper

  and

  mashehu

  committed
  f8e5068

  Copy full SHA for f8e5068

• Fixes suggested by @mirpedrol during review. Thanks!

  MatthiasZepper

  committed
  e512878

  Copy full SHA for e512878

• @mashehu suggested that downloading the containers should not be optional for Tower downloads. Given that there is the option to provide the list of remote containers to skip their download, I agre…

  Show description for 315b9a3
  MatthiasZepper

  committed
  315b9a3

  Copy full SHA for 315b9a3

Commits on May 26, 2023

• Bugfix: WorkflowRepo.tidy_tags() did indeed only tidy tags. However, revisions may also be branches. Therefore, I rewrote this function to account for revisions that are not releases.

  MatthiasZepper

  committed
  6a806ee

  Copy full SHA for 6a806ee

Commits on May 31, 2023

• Merge branch 'dev' into DownloadForTower

  MatthiasZepper

  authored
  259afa6

  Copy full SHA for 259afa6