tool to query ftplinks from bioimage archive #1345

sunyi000 · 2023-10-31T14:17:36Z

A tool to query ftplinks for study from bioimage archive

inputs: mode, path
output: all images from accession plus optional ftp links
@beatrizserrano

tools/image_processing/bia-ftplinks/ftpLink.txt

bgruening · 2023-10-31T14:29:12Z

tools/image_processing/bia-ftplinks/biaftplink.xml

+    <expand macro="requirements" />
+    <command detect_errors="aggressive">
+	    <![CDATA[
+		#for $study in $study_name.split(","):


I guess this tool is a good step forward, but wouldn't it be better to query the BIA API?

i thought the curl command actually queries the api..I haven't found other api documentation yet..

The BIA API seems to be the BioStudies one: https://www.ebi.ac.uk/biostudies/help

Yes, its an API. Sorry I was not concrete here. I think its a good first step. I was just wondering if there is no API abstraction library that we should use. But maybe there is none.

thanks both. Yes I found that biostudies API. It seems more complicated than just a tool for getting FTP links.

bgruening · 2023-10-31T16:52:13Z

tools/image_processing/bia-ftplinks/biaftplink.xml

+    <help>
+        <![CDATA[
+            **What it does**
+	    This tool finds the ftp link for a study in the Bioimage Archive


Suggested change

This tool finds the ftp link for a study in the Bioimage Archive

This tool finds all FTP links associated to a study in the Bioimage Archive.

bgruening · 2023-10-31T16:52:40Z

tools/image_processing/bia-ftplinks/macros.xml

+    <token name="@VERSION@">0.1.0</token>
+    <xml name="requirements">
+        <requirements>
+	    <requirement type="package" version="1.20.3">wget</requirement>


why is wget needed?

ah, initially I was thinking of after getting the ftp links then download all the images using wget -r. so the tool will be about download images from bia.

Then I see there is already a lftp tool so I removed the download images part.

what do you suggest? A tool download images from bia,or a tool like this just output ftp links? If this is better I will remove wget from requirements

Thanks

bgruening

Looks good to me @sunyi000

One question, what should a user do with the FTP links? Should this tool offer at least the option to download them and store them in a collection?

sunyi000 · 2023-10-31T17:31:49Z

Looks good to me @sunyi000

One question, what should a user do with the FTP links? Should this tool offer at least the option to download them and store them in a collection?

Do you mean offering an option to download images with the links? For example,if it is checked,then the output of this tool will be ftp links and images. If it is unchecked,just output links?
I think this maybe more useful?

sunyi000 · 2023-12-06T17:24:13Z

now it downloads all images :)
thank you both @beatrizserrano @bgruening

bgruening · 2023-12-06T17:30:44Z

Thanks @sunyi000!

Do you want to create a PR to add this tool to the EU instance as well? https://github.com/usegalaxy-eu/usegalaxy-eu-tools

* added tool to query ftplink from bia * multiple values * add help * add .shed.yml;remove multiple from param * remove wrong file * updates to the tool * Update biaftplink.xml * Update .shed.yml * Update biaftplink.xml --------- Co-authored-by: Yi Sun <ysun@embl.de> Co-authored-by: Björn Grüning <bjoern@gruenings.eu>

* woundhealing tool * added gitlab citation * fixing typo * remove unwanted file * add .shed.yml * added location tag in tests * clean test data * update shed.yml * Fix two bugs with `split_file_to_collection` (#1358) * fix two bugs * allow chunksize to be greater than the number of records in the input file without failing * allow an empty file to be split without an error * bump version number * Update Nanopolish to v0.14 (#1359) * fix lint error + version upgrade * Explicit untar of .tar.xz * Adapt tests to multithreading and code refactoring * Add RBPBench tool wrapper (#1360) * add wrapper files * add macros * modify select_rbps command * add fasta_indexes.loc * fix data_table and genome fasta path * new conda version with wrapper script * new conda version with wrapper script * remove python_template_version --------- Co-authored-by: Pavankumar Videm <pavanvidem@gmail.com> * update rbpbench tool-data conf (#1362) * Add palette options to convert (#1364) * tool to query ftplinks from bioimage archive (#1345) * added tool to query ftplink from bia * multiple values * add help * add .shed.yml;remove multiple from param * remove wrong file * updates to the tool * Update biaftplink.xml * Update .shed.yml * Update biaftplink.xml --------- Co-authored-by: Yi Sun <ysun@embl.de> Co-authored-by: Björn Grüning <bjoern@gruenings.eu> * update v08 to fix meme version check error (#1366) * add mavedb data source tool (#1367) * add mavedb data source tool * Rename shed.yml to .shed.yml * change category * remove duplicate category * change tool ID to mavedb_importer * Update tools/mave_tools/mavedb/.shed.yml * Update mavedb.xml --------- Co-authored-by: Björn Grüning <bjoern@gruenings.eu> * Change query params according mavedb requirements (#1368) * add mavedb data source tool * Rename shed.yml to .shed.yml * change category * remove duplicate category * change tool ID to mavedb_importer * Update tools/mave_tools/mavedb/.shed.yml * Update mavedb.xml * change param names for mavedb requirements * remove comment --------- Co-authored-by: Björn Grüning <bjoern@gruenings.eu> Co-authored-by: Björn Grüning <bjoern.gruening@gmail.com> * update to v081 (#1369) * gm updates (#1373) * gm updates * fix linting * Update compare.xml * Integrate another bioMLmodel (biomd0000001076) (#1371) * Add tool BIOMD0000001076 * update tool desc * update tool desc * add version * update * update test * remove print * add creator * update tool * update doi * Update tools/biomodelsML/biomodels_BIOMD0000001076.xml Co-authored-by: Björn Grüning <bjoern@gruenings.eu> * remove xlsx * test bibtext to fix test * revert to type=doi * remove creator tag * update profile * update embedded script name --------- Co-authored-by: Björn Grüning <bjoern@gruenings.eu> * fix typo in param documentation (#1374) * bump intarna 3.4.0 (#1378) * Updating tools/cpat from version 3.0.4 to 3.0.5 (#1379) Co-authored-by: planemo-autoupdate <planemo-autoupdate@users.noreply.github.com> * Updating tools/image_processing/graphicsmagick from version 1.3.40 to 1.3.42 (#1380) Co-authored-by: planemo-autoupdate <planemo-autoupdate@users.noreply.github.com> * Cool to hic (#1382) * adding first tested and working generated tool from https://github.com/fubar2/nftoolmaker It actually does work. Sort of. Hope this is useful to someone. * fixes suggested by bgruening for the PR - all adjustments will now apply to all future generated tools :) * add yara mapper ToolFactory - NOT from nf-core. Allows single ended fastq or fasta to be mapped, or a pair of forward and reverse - both must be fastq * small test files still work :) * single quotes for $runme - disappeared mysteriously :( * adding panaroo * Finally fixed the '$runme' suggestion and got the sanitisers not to. * fix profile="22.05" - need to update galaxyxml to fix this and the stdio replacement... * cool_to_hic tool * add cool_to_hic * cleanup other branch * fix bogus gfastats fix * fix gfastats * and again * fix yaml * fix so allows cool/mcool/scool - untested as I got no data... * need test data. * fixes from review * fix bogus help transition * Update .shed.yml * some small cleanups * Update .shed.yml * lets call it juicebox_hic --------- Co-authored-by: Björn Grüning <bjoern@gruenings.eu> * Add juicebox_hic format and citation (#1383) * adding first tested and working generated tool from https://github.com/fubar2/nftoolmaker It actually does work. Sort of. Hope this is useful to someone. * fixes suggested by bgruening for the PR - all adjustments will now apply to all future generated tools :) * add yara mapper ToolFactory - NOT from nf-core. Allows single ended fastq or fasta to be mapped, or a pair of forward and reverse - both must be fastq * small test files still work :) * single quotes for $runme - disappeared mysteriously :( * adding panaroo * Finally fixed the '$runme' suggestion and got the sanitisers not to. * fix profile="22.05" - need to update galaxyxml to fix this and the stdio replacement... * add citation and link to github add juicebox_hic as the name of the output * fix branch again * update CI * Add TAPscan (#1357) * add tapscan tool * update shed file for this repo * add edam topic and creator * compress domains file * AGAT - Add new feature (functional annotation) (#1377) * add new feature * small modifications * some modification * small changes * small modifications * small modification * small modification --------- Co-authored-by: Bjoern Gruening <bjoern.gruening@gmail.com> * update CI * update CI * Tapscan update (#1388) * add new updated tapscan files * update wrapper and test data * change fiji-mopholibj version * small cleanups * Update woundhealing.xml --------- Co-authored-by: Yi Sun <ysun@embl.de> Co-authored-by: Simon Bray <32272674+simonbray@users.noreply.github.com> Co-authored-by: Tunc Kayikcioglu <19364610+tuncK@users.noreply.github.com> Co-authored-by: Michael Uhl <uhl.michael@gmx.de> Co-authored-by: Pavankumar Videm <pavanvidem@gmail.com> Co-authored-by: Helena <hexylena@galaxians.org> Co-authored-by: Björn Grüning <bjoern@gruenings.eu> Co-authored-by: Polina Polunina <55543056+PlushZ@users.noreply.github.com> Co-authored-by: Björn Grüning <bjoern.gruening@gmail.com> Co-authored-by: Anup Kumar <kumara@informatik.uni-freiburg.de> Co-authored-by: Linelle <48773001+abueg@users.noreply.github.com> Co-authored-by: Martin Raden <mmann@informatik.uni-freiburg.de> Co-authored-by: gxydevbot <72021197+gxydevbot@users.noreply.github.com> Co-authored-by: planemo-autoupdate <planemo-autoupdate@users.noreply.github.com> Co-authored-by: Ross Lazarus <fubar2@users.noreply.github.com> Co-authored-by: Saskia Hiltemann <shiltemann@users.noreply.github.com> Co-authored-by: Rlibouban <127295521+rlibouba@users.noreply.github.com>

Yi Sun added 3 commits October 31, 2023 12:08

added tool to query ftplink from bia

89700cb

multiple values

dea57ed

add help

8e0a52d

bgruening reviewed Oct 31, 2023

View reviewed changes

Yi Sun added 2 commits October 31, 2023 15:29

add .shed.yml;remove multiple from param

4fbe55e

remove wrong file

d33add0

bgruening reviewed Oct 31, 2023

View reviewed changes

sunyi000 and others added 3 commits December 6, 2023 14:06

Merge branch 'master' into bia-ftplinks

67f54ca

updates to the tool

72c468a

Update biaftplink.xml

fecd82e

bgruening added 2 commits December 6, 2023 18:28

Update .shed.yml

5ddd56c

Update biaftplink.xml

a2a00b1

bgruening approved these changes Dec 6, 2023

View reviewed changes

bgruening merged commit 04eb6bf into bgruening:master Dec 6, 2023
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tool to query ftplinks from bioimage archive #1345

tool to query ftplinks from bioimage archive #1345

sunyi000 commented Oct 31, 2023 •

edited

Loading

bgruening Oct 31, 2023

sunyi000 Oct 31, 2023

beatrizserrano Oct 31, 2023

bgruening Oct 31, 2023

sunyi000 Oct 31, 2023

bgruening Oct 31, 2023

bgruening Oct 31, 2023

sunyi000 Oct 31, 2023 •

edited

Loading

bgruening left a comment

sunyi000 commented Oct 31, 2023

sunyi000 commented Dec 6, 2023 •

edited

Loading

bgruening commented Dec 6, 2023

	This tool finds the ftp link for a study in the Bioimage Archive
	This tool finds all FTP links associated to a study in the Bioimage Archive.

tool to query ftplinks from bioimage archive #1345

tool to query ftplinks from bioimage archive #1345

Conversation

sunyi000 commented Oct 31, 2023 • edited Loading

bgruening Oct 31, 2023

Choose a reason for hiding this comment

sunyi000 Oct 31, 2023

Choose a reason for hiding this comment

beatrizserrano Oct 31, 2023

Choose a reason for hiding this comment

bgruening Oct 31, 2023

Choose a reason for hiding this comment

sunyi000 Oct 31, 2023

Choose a reason for hiding this comment

bgruening Oct 31, 2023

Choose a reason for hiding this comment

bgruening Oct 31, 2023

Choose a reason for hiding this comment

sunyi000 Oct 31, 2023 • edited Loading

Choose a reason for hiding this comment

bgruening left a comment

Choose a reason for hiding this comment

sunyi000 commented Oct 31, 2023

sunyi000 commented Dec 6, 2023 • edited Loading

bgruening commented Dec 6, 2023

sunyi000 commented Oct 31, 2023 •

edited

Loading

sunyi000 Oct 31, 2023 •

edited

Loading

sunyi000 commented Dec 6, 2023 •

edited

Loading