How to use Volta in CI

[dherman]

I had a great conversation with @chancancode about using Volta in CI. The main takeaway was we should have instructions in the docs for how to use Volta in CI. But it turned out to be nontrivial and we should spend some time polishing the user experience and testing it out in popular CI systems (Travis, Circle, Azure Pipelines, and GitHub Actions). We should also have doc pages or sections for each of these popular CI systems.

Some of the challenges Godfrey ran into:

• In Travis, the setup command curl | bash silently fails sometimes because of something about how Travis does download caching.
• Since installing Volta requires opening a new shell to get the persisted changes to the environment, you can't install it and use it in the same shell. What should we recommend here?
  • Manually make the env changes in CI?
  • Some kind of CI installer that spits out commands to eval or source in the CI yaml?
  • Should we move the volta executable to VOLTA_HOME/bin?
  • Can we eliminate the function wrapper soon?
• How should we interact with build matrices? It would be nice to be able to choose different tool versions via VOLTA_* env vars, but we should think about whether this kind of precedence algorithm (overriding local config with an env var) could be problematic.

For reference, here's a fairly minimal travis config Godfrey is using for his project, where he got Volta working.

[dustinsoftware]

An issue we noticed on Windows that would prevent us from using volta on our teamcity-based build agents: if any script runs volta install node@<version>, it will change the version for already spawned processes, which could break other builds that had started before the votla install command runs.

For example, if a build script invokes node multiple times, but suddenly the version changes, it could result in unexpected build failures.

A workaround could be that build scripts explicitly point to a versioned Node directory after it has been installed. The PATH could also be modified for the current session so that scripts can be sure the version of node that was just installed is the one that's going to be used for all commands in the current session.

[charlespierce]

@dustinsoftware Out of curiosity, what is the use case for calling volta install node@<version> multiple times in the build agent? If you have the version pinned in your projects, Volta will automatically download the right version when you run a command.

Alternatively, you can use the volta fetch node@<version> command, which will do the download but not make any changes to the default version that would affect other processes.

[dherman]

I just noticed rustup has a precedence algorithm that involves env vars in a way that would work well for CI matrices, which gives me some confidence that that approach could work for Volta. Unlike rustup, though, I think there would need to be more than one variable: one for each dimension that should be independently controlled. Probably we’d want variables like VOLTA_NODE, VOLTA_NPM, VOLTA_YARN.

[mydea]

FYI I just spent some time getting volta to work on CircleCI. After some experimentation, I arrived at the following setup which seems to work as expected now:

commands:
  setup-node-version:
    description: Install volta & ensure consistent node & yarn versions are installed
    steps:
      - run:
          name: Install volta
          command: |
            curl https://get.volta.sh | bash
            echo 'export VOLTA_HOME=$HOME/.volta' >> $BASH_ENV
            echo 'export PATH=$VOLTA_HOME:$PATH' >> $BASH_ENV
      - run:
          name: Pin node@10
          command: volta install node@10
      - run:
          name: Pin yarn@1.19
          command: volta install yarn@1.19

Note that I had to add both $VOLTA_HOME to the path. I didn't have to add e.g. $VOLTA_HOME/bin or similar, though. The following steps are all using the correct node/yarn version, as far as I can tell.

Somewhat related, on Github Actions I didn't need to do anything but curl https://get.volta.sh | bash to get it to work - nice!

[rwjblue]

For folks wanting to use volta with GitHub actions, I created https://github.com/rwjblue/setup-volta (see usage details in README). I hope to be able to move that repo into the volta-cli org, but that may need an RFC?

[mydea]

Nice! I played around with this myself (turns out, it didn't "just work" on Github, it just so happened by accident that the default installed yarn/node versions matched the ones I had setup, so it seemed like Volta was working 🤕 ), with a much more "basic" implementation - although it does some other things as well (setting up problem matchers, ..): https://github.com/mydea/actions-ember-testing - Might just remove the volta part from it and use yours, as it seems more thorough :)

[rwjblue]

FYI - The GitHub action has been moved into this org:

• https://github.com/volta-cli/action
• https://github.com/marketplace/actions/volta-cli-action

[glasser]

fyi, @mydea 's comment above isn't quite right: the PATH-setting line should say $VOLTA_HOME/bin instead of just $VOLTA_HOME.

[spiderhands]

Any updates? Did anyone set up volta in the bitbucket pipelines?

[florinasavei]

I was also struggling with this in and Azure DevOps pipeline.
It seems that the Bash Task has an option called bashEnvValue as described in the docs where you can point it to the profile

So this worked for me in Azure DevOps:

steps:
 
- task: Bash@3
  inputs:
    targetType: 'inline'
    script: 'curl https://get.volta.sh | bash'
    bashEnvValue: '~/.profile'

- task: Bash@3
  inputs:
    targetType: 'inline'
    script: |
      volta install node
    bashEnvValue: '~/.profile'

- task: Bash@3
  inputs:
    targetType: 'inline'
    script: |
      volta install yarn
    bashEnvValue: '~/.profile'

[shreejan-regmi01]

@spiderhands This worked for me in bitbucket pipelines:

- step:
          caches:
            - node
          name: "Install frontend dependencies and start dev server"
          script:
            - curl https://get.volta.sh | bash
            - export VOLTA_HOME=$HOME/.volta
            - export PATH=$VOLTA_HOME/bin:$PATH
            - volta install node
            - npm install
            - node --version
            - node server.js &

[Vinnl]

Oh, just realised it might be useful for people here to know that GitHub's own setup-node Action added support for Volta a while ago: https://github.com/actions/setup-node/releases/tag/v3.5.0

[jameelmoses]

@shreejan-regmi01 what image are you using as a base for this? the only two things i need in my pipeline are volta and aws cli

@spiderhands This worked for me in bitbucket pipelines:

- step:
          caches:
            - node
          name: "Install frontend dependencies and start dev server"
          script:
            - curl https://get.volta.sh | bash
            - export VOLTA_HOME=$HOME/.volta
            - export PATH=$VOLTA_HOME/bin:$PATH
            - volta install node
            - npm install
            - node --version
            - node server.js &

[shreejan-regmi01]

@shreejan-regmi01 what image are you using as a base for this? the only two things i need in my pipeline are volta and aws cli

@spiderhands This worked for me in bitbucket pipelines:

- step:
          caches:
            - node
          name: "Install frontend dependencies and start dev server"
          script:
            - curl https://get.volta.sh | bash
            - export VOLTA_HOME=$HOME/.volta
            - export PATH=$VOLTA_HOME/bin:$PATH
            - volta install node
            - npm install
            - node --version
            - node server.js &

I used the image atlassian/default-image:3 that's mentioned in their documentation here https://support.atlassian.com/bitbucket-cloud/docs/use-docker-images-as-build-environments/

[jorismak]

I use the first 3 lines of that script in my bitbucket pipelines.
The projects all have their versions pinned, so I don't have to 'install' anything.

But the download fails sometimes, with a 404. And then the entire pipeline fails of course.

Is that because of rate limiting? Or does the pipeline run at the same time 'volta latest' gets changed and it's just an unlucky timing issue?

Is there some other way the volta people would like to do this? Because this can hammer their download URL quite a bit if all the runs of everyone start with downloading the script.

[Pazns]

I planned to use Volta in my CI, to ease experimentation across dev teams, but as I understand it, it will lead to every single CI job downloading Node/Npm at each start.
As it is running over Kubernetes pods, I would have mounted a volume from the local node into the pod into a Volta download cache folder, to at least mitigate the issue, and share the raw archives.
I don't know where this folder is, or if this is even possible as Volta does low-level stuff with hard-links.

Is there a way to make Volta efficient in CI ?

[jorismak]

Most ci/cd systems i know of have caching methods you can define. For instance , i can mark any folder as a cache , which means that folder is placed as is in Amy ci/cd job, unless it isn't used for over a week, in that case nothing gets restored so you start fresh. Mark your ~/.volta directory (i think , typing this from memory ) as cacheable , or save and restore it another way with every ci/cd job and it should be fine ?

…

On Thu, Feb 23, 2023, 20:43 Pazns ***@***.***> wrote: I planned to use Volta in my CI, to ease experimentation across dev teams, but as I understand it, it will lead to every single CI job downloading Node/Npm at each start. As it is running over Kubernetes pods, I would have mounted a volume from the local node into the pod into a Volta download cache folder, to at least mitigate the issue, and share the raw archives. I don't know where this folder is, or if this is even possible as Volta does low-level stuff with hard-links. Is there a way to make Volta efficient in CI ? — Reply to this email directly, view it on GitHub <#43 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALMD2DSHSIYL4J6NINWJ4H3WY64UZANCNFSM4IOX4TGQ> . You are receiving this because you commented.Message ID: ***@***.***>