Skip to content

Latest commit

 

History

History
108 lines (68 loc) · 7.98 KB

DEPENDENCIES.md

File metadata and controls

108 lines (68 loc) · 7.98 KB

Dependencies

Back to Team Agreement

This is the full story. The TL;DR is here.

skaardb is primarily a Go repo that also generates Javascript (JS) code with Dart bindings, as well as frugal-generated messaging API code (currently, Go, Dart, and Python).

To ease dependency management, flexible SemVer ranges are recommended for all Workiva repo's with version >=1.0.0. Please comment exceptional cases appropriately.

Go

Make sure dep is installed.

A subset of the Go code is transpiled to JS using GopherJS. GopherJS is consumed by skaardb and installed as an executable. The main complication here is that GopherJS does not support being vendored, so the Makefile targets accommodate this to ease development while still tracking GopherJS using standard dependency management tools.

Code dependencies and build tools are vendored in the vendor directory and committed to this repo. To vendor the dependencies and install the build tools specified by the Gopkg.toml and/or Gopkg.lock, run

$ make vendor

This will add any missing dependencies to the vendor directory, generate the Gopkg.lock file if it is missing, and also prune unneeded items. The vendor directory (without GopherJS) and the lock file are committed to the repo. Certain vendored files are also not committed via specification in the gitignore file.

Because of the dependencies that make vendor can change, make vendor also conservatively runs make build-tools, make gopherjs, and make generate-thrift-all.

See the Usage section of the dep README and the spec for the Gopkg.toml. In particular, if a dependency is new, then save the associated import statement(s) to at least one Go source file before running make vendor to avoid premature pruning by make vendor.

IMPORTANT

If you use the dep tool directly, such as dep ensure -update [some/repo/I/need/to/update], then be sure to run make vendor when you are done to make sure everything else is brought up to date.

The vendor directory is checked for accuracy, but not updated, in Smithy CI, so make sure to commit the correct and up-to-date vendored dependencies (by running make vendor before pushing).

Dart

Dart code is in the lib directory and dependency management is the standard one provided by pub. The version control in the pubspec.yaml must be compatible with publishing skaardb to to Workiva's pub server for Dart client consumption. The pubspec.lock is currently committed to the repo.

IMPORTANT

pub get will pull Dart dependencies in Smithy CI, and in some edge cases will change dependencies specified in the pubspec.lock.

Parsimony, Frugal, GopherJS, and Other Build Tools

Build tools such as parsimony, frugal, and gopherjs are installed on the GOPATH (with versions taken from the Gopkg.lock) using

$ make build-tools

Note that—

  • GopherJS currently cannot be vendored, and so make build-tools installs it on the GOPATH at the commit specified in the Gopkg.toml. (Note that indirect dependencies of GopherJS installed on the GOPATH by go get -u github.com/gopherjs/gopherjs are not version controlled.)
  • Other critical build tools (e.g., frugal) are specified in the Gopkg.toml, vendored, and installed by make build-tools from skaarbd's vendor directory. (Build tools like frugal also appear as an imported library.)
  • Certain non-critical 3rd-party build tools are not specified in the Gopkg.toml (e.g., go2xunit), and their versions are tracked by the Smithy builder version that includes them.

Updating GopherJS-Generated Javascript Code

The GopherJS-generated JS code needs to be updated when changes are made to transpiled Go code.

  1. Ensure the branch is up-to-date with master:
$ git pull origin master
  1. Build the GopherJS-generated Javascript code:
$ make gopherjs

The GopherJS transpiled JS code and its Dart bindings are found here. Note that make gopherjs also runs the constpiler that generates Dart bindings from Go constants.

IMPORTANT

The GopherJS-generated .js and .js.map files for the Dart bindings are not committed to the Git repository, so if you wish to consume the bindings from your local copy of skaardb, you'll need to run the make gopherjs command.

If you want to commit the GopherJS-generated JS files so that you can consume the bindings from a Git branch, you can force add the files to override the .gitignore settings for them: git add -f lib/*.js*.

Adding More Methods to the Dart Bindings (bindings.go case)

  1. Write a top-level method in bindings.go.
  2. Add an entry to the bindings map in the bindings.go main method.
  3. Update the GopherJS-generated javascript code (see above).
  4. Write a public API in the dart code that uses context['skaardbBindings'].callMethod() to leverage the method(s) written in bindings.go that were transpiled into javascript. Use the existing methods in bindings.dart as a model.
  5. Merge the PR to skaardb. This will update the Dart pub package on the subsequent tag.
  6. Consume the new tagged version of skaardb in your Dart package.

For advice on consumption by Dart, see dart-bindings/README.md.

IMPORTANT

You don't wan't to break consumers' builds or block them from consuming new functionality, so be sure to provide an adequate deprecation period for breaking changes, following Workiva's Wdesk Client-side Versioning Policy.

Updating Frugal-Generated Code

Frugal-generated code is managed using Workiva's parsimony tool, as specified in the parsimony.yml. We track the version of parsimony in the Gopkg.lock and vendor it like other build tools.

To generate code for all languages, run

$ make generate-thrift-all

In particular, the frugal-generated Go code is committed to the repo here, and the frugal-generated Dart code is included in the Dart bindings for client consumption. Python code is used for testing, for example, in "thin" load testing clients. Java generated code is built, packaged, and made available for installation into JVM projects via artifactory.

Building Java Locally

In order to build and package the generated Java code locally, you will need to set up your maven environment with credentials to our internal artifactory server. This process is documented on the Dev Portal.

IMPORTANT

Frugal-generated code is not updated in Smithy CI, so make sure you updated and commit the generated code when needed. Also, be sure to follow Workiva's Wdesk Client-side Versioning Policy that prohibits tightly coupled server+client releases. Deprecate wisely to avoid coupled releases and to accommodate "rollback" combinations of new/old server against new/old client.