Document preserving artifacts between stages

[svenfuchs]

From travis-ci/beta-features#11 (comment)

We'd want to check how this compares to existing documentation, and potentially add this as a separate solution.

---

The following trick can be used to preserve artifacts between stages: enable caching, and store the built artifacts inside the cached directory.

Don't forget to clear the cache directory at the end of the last stage, so that it does not waste space on Travis's infrastructure. You might want to also clear the cache directory at the beginning of the first stage, in case some stale data was not discarded, e.g. after a partially failed build.

It is possible to check in the second and subsequent stages that the cache was populated by the same build by storing "$TRAVIS_BUILD_ID" in the cache and then comparing the value.

language: c
sudo: false

cache:
  directories:
    - ~/mycache

jobs:
  include:
    - stage: phase1
      install:
      - mv ~/mycache ~/mycache-discard
      - mkdir ~/mycache
      - echo "$TRAVIS_BUILD_ID"
      - echo "$TRAVIS_BUILD_ID" > ~/mycache/travis_build_id
      - echo "build stuff" > ~/mycache/artifact.bin
      script:
      - true
    - stage: phase2
      install:
      - if test "$TRAVIS_BUILD_ID" != `cat ~/mycache/travis_build_id`; then travis_terminate 1; fi
      - rm -f ~/mycache/travis_build_id
      - echo "Do something with ~/mycache/artifact.bin, e.g. publish it to gh-pages"
      - mv ~/mycache ~/mycache-discard
      - mkdir ~/mycache # clear the cache
      script:
      - true

Note that the Travis documentation seems to hint that the cache is not updated as part of cron builds, and manually restarted builds. The trick should hopefully work for pull requests, though. Also, I'm not sure how this will behave when a stage contains multiple concurrent jobs which could in principle all populate the cache. Apparently, it is possible to explicitly specify the cache name with an environment variable CACHE_NAME=FOOBAR.

[stale]

Thanks for contributing to this issue. As it has been 90 days since the last activity, we are automatically closing the issue. This is often because the request was already solved in some way and it just wasn't updated or it's no longer applicable. If that's not the case, please do feel free to either reopen this issue or open a new one. We'll gladly take a look again! You can read more here: https://blog.travis-ci.com/2018-03-09-closing-old-issues

[jgebal]

To avoid race condition for cache and purging cache data that might be still needed (when using matrix or concurrent builds) I would use time-based cache purging. Something like:

  install:
    #delete all files in cache that are older than 5 days
    - find ~/mycache/ -mtime +5 -exec rm {} \;