Let cache.name override cache slug name we compute

[BanzaiMan]

So that the jobs within a build can share caches arbitrarily.

It is not possible to put this change in
Travis::Build::Script#cache_slug, since multiple modules may call
this with super; in such cases the end cache name will be different
from what is given in cache.name.

[BanzaiMan]

See https://staging.travis-ci.org/BanzaiMan/travis_staging_test/builds/667279 for an example build.

[BanzaiMan]

This resolves travis-ci/travis-ci#7590.

[pat-s]

@BanzaiMan any ETA of this feature?

[meatballhat]

For documentation/clarity/maybe-just-for-me, can you list what part of the cache object key will remain consistent even when using cache.name?

[BanzaiMan]

@meatballhat

All other cache mechanics remain.

The example configuration used in the test above is:

cache:
  bundler: true
  name: shared

stages:
  - set up cache
  - test
rvm:
  - '2.4'
  - '2.3'
  
jobs:
  include:
　　　- stage: set up cache

This runs the "Set up cache" stage first (which creates shared.tgz), which will be used by the two jobs in the "Test" stage.

The configurations that need change are those of the form:

cache: bundler

but the change would be consistent with other cache configurations, in that it will be transformed into:

cache:
  bundler: true
  name: shared

The caveat with cache corruption remains. This needs to be documented well.

[meatballhat]

@BanzaiMan Do I understand correctly that each cache will continue to be created under a repository-specific "namespace", regardless of the given cache.name? Do any extra safeguards need to be added to prevent path traversal, e.g.:

cache:
  name: "../../../dastardly-nefarious"

[meatballhat]

@BanzaiMan ooooh, and what about URI-encoding the name?

cache:
  name: "very/special?!cache&name%"

[BanzaiMan]

@meatballhat cache_slug (computed, or overridden with name) is passed to a Travis::Build::Script::DirectoryCache::Base object, which computes the S3-compat signature for casher to use. It is not a path to a file on the file system. It is conceivable that we end up uploading a cache to, say, s3://bucket/path/to/../../nefarious.tgz, but that file will be downloaded as fetch.tgz, and expanded.

[BanzaiMan]

It is worth testing, for sure.

[BanzaiMan]

@meatballhat We normalize the cache names:

travis-build/lib/travis/build/script/shared/directory_cache/base.rb

Line 230 in 80a0271

args.map! { |arg| arg.to_s.gsub(/[^\w\.\_\-]+/, '') }

[BanzaiMan]

See this test: https://staging.travis-ci.org/BanzaiMan/travis_staging_test/jobs/715139#L449

[meatballhat]

@BanzaiMan Yep, my concern wasn't with local file path traversal, but potentially poisoning or stealing a cache from a different repository.

[BanzaiMan]

It just occurred to me. Another consideration we might add for the future is the read_only attribute, so that the job downloads but does not upload the cache.

[mosyp]

Hello, imagine I have first stage with 2 concurrent jobs. Later I have second stages which aggregates caches from first stage. Does this PR will help me when merged or there's another already working solution?

[pat-s]

Whats the status here @BanzaiMan? :)

[gobetti]

Hello @BanzaiMan , how can we help to have this merged? Thanks!

[BanzaiMan]

We'll have a more comprehensive look at this problem.