Skip to content
This repository has been archived by the owner on Sep 11, 2024. It is now read-only.

Use Buildkite for CI #2788

Merged
merged 11 commits into from
Mar 15, 2019
Merged

Use Buildkite for CI #2788

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
618b2b1
Add basic BuildKite :pipeline:
turt2live Mar 13, 2019
fd23be9
Remove Travis (CI)
turt2live Mar 13, 2019
378bf68
Force the scripts to be executable in BuildKite
turt2live Mar 13, 2019
311c81f
Disable end-to-end tests for now
turt2live Mar 13, 2019
c63c6fc
Linter doesn't need downstream projects
turt2live Mar 14, 2019
bdc94f3
Fix Chrome for Karma
turt2live Mar 14, 2019
cab5f7a
Use the right branch for fetching dependencies
turt2live Mar 14, 2019
d8a5590
Better emoji is the most important part of CI
turt2live Mar 15, 2019
5b68a70
Remove uneeded CHROME_BIN env setter
turt2live Mar 15, 2019
365c8e9
Fetch downstream dependencies from as many places as possible
turt2live Mar 15, 2019
6d7703d
Re-introduce `yarn test` and fix comments
turt2live Mar 15, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions .buildkite/pipeline.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
steps:
- label: ":eslint: Lint"
command:
- "yarn install"
- "yarn lintwithexclusions"
plugins:
- docker#v3.0.1:
image: "node:10"

# - label: ":chains: End-to-End Tests"
# command:
# # TODO: Remove hacky chmod for BuildKite
# - "chmod +x ./scripts/ci/*.sh"
# - "chmod +x ./scripts/*"
# - "sudo apt-get install build-essential python2.7-dev libffi-dev python-pip python-setuptools sqlite3 libssl-dev python-virtualenv libjpeg-dev libxslt1-dev"
# - "./scripts/ci/install-deps.sh"
# - "./scripts/ci/end-to-end-tests.sh"
# plugins:
# - docker#v3.0.1:
# image: "node:10"

- label: ":karma: Tests"
command:
# Install chrome
- "wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -"
- "sh -c 'echo \"deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main\" >> /etc/apt/sources.list.d/google.list'"
- "apt-get update"
- "apt-get install -y google-chrome-stable"
# Run tests
# TODO: Remove hacky chmod for BuildKite
- "chmod +x ./scripts/ci/*.sh"
- "chmod +x ./scripts/*"
- "./scripts/ci/install-deps.sh"
Copy link
Collaborator

@jryans jryans Mar 15, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still feels wasteful to duplicate so much of the installation... I can't tell from the BK log how long each step takes right now, since the log get truncated...

Speaking of that, the biggest factor in our log length seems to be Webpack's --progress output. Can we change Webpack options used in CI so that the log shows up start to finish in BK?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The log gets truncated for the web view only - as mentioned in the error, there's a download button for when your build explodes and you want to see the whole log. Historically, our stuff breaks before it would reach the point of truncation.

- "./scripts/ci/unit-tests.sh"
env:
CHROME_BIN: "/usr/bin/google-chrome-stable"
turt2live marked this conversation as resolved.
Show resolved Hide resolved
plugins:
- docker#v3.0.1:
image: "node:10"
propagate-environment: true

- label: "🔧 Riot Tests"
command:
# Install chrome
- "wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -"
- "sh -c 'echo \"deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main\" >> /etc/apt/sources.list.d/google.list'"
- "apt-get update"
- "apt-get install -y google-chrome-stable"
# Run tests
# TODO: Remove hacky chmod for BuildKite
- "chmod +x ./scripts/ci/*.sh"
- "chmod +x ./scripts/*"
- "./scripts/ci/install-deps.sh"
- "./scripts/ci/riot-unit-tests.sh"
env:
CHROME_BIN: "/usr/bin/google-chrome-stable"
plugins:
- docker#v3.0.1:
image: "node:10"
propagate-environment: true
38 changes: 0 additions & 38 deletions .travis.yml
View file
Open in desktop

This file was deleted.

3 changes: 2 additions & 1 deletion karma.conf.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,9 +135,10 @@ module.exports = function (config) {
],

customLaunchers: {
'ChromeHeadless': {
'VectorChromeHeadless': {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still use the Vector prefix that much, now that it's called Riot? Also, what's the purpose of the custom name here in any case?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm using Vector in the company sense here, not in the product sense. ChromeHeadless is an actual browser version and seems to confuse Buildkite (but not Travis CI for some reason), so I've essentially named it New Vector's ChromeHeadless Browser v10: Spectacular feature release for CI environments without using so many words.

base: 'Chrome',
flags: [
'--no-sandbox',
// See https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md
'--headless',
'--disable-gpu',
Expand Down
4 changes: 2 additions & 2 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@
"lintwithexclusions": "eslint --max-warnings 0 --ignore-path .eslintignore.errorfiles src test",
"clean": "rimraf lib",
"prepare": "yarn clean && yarn build && git rev-parse HEAD > git-revision.txt",
"test": "karma start --single-run=true --browsers ChromeHeadless",
"test": "karma start --single-run=true --browsers VectorChromeHeadless",
"test-multi": "karma start"
},
"dependencies": {
Expand Down Expand Up @@ -131,7 +131,7 @@
"flow-parser": "^0.57.3",
"jest-mock": "^23.2.0",
"karma": "^4.0.1",
"karma-chrome-launcher": "^0.2.3",
"karma-chrome-launcher": "^2.2.0",
"karma-cli": "^1.0.1",
"karma-junit-reporter": "^2.0.0",
"karma-logcapture-reporter": "0.0.1",
Expand Down
2 changes: 1 addition & 1 deletion scripts/travis/build.sh → scripts/ci/build.sh
100755 → 100644
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/bash
#
# script which is run by the travis build (after `yarn test`).
# script which is run by the CI build (after `yarn test`).
#
# clones riot-web develop and runs the tests against our version of react-sdk.

Expand Down
4 changes: 2 additions & 2 deletions scripts/travis/end-to-end-tests.sh → scripts/ci/end-to-end-tests.sh
100755 → 100644
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/bash
#
# script which is run by the travis build (after `yarn test`).
# script which is run by the CI build (after `yarn test`).
#
# clones riot-web develop and runs the tests against our version of react-sdk.

Expand All @@ -9,7 +9,7 @@ set -ev
RIOT_WEB_DIR=riot-web
REACT_SDK_DIR=`pwd`

scripts/travis/build.sh
scripts/ci/build.sh
# run end to end tests
scripts/fetchdep.sh matrix-org matrix-react-end-to-end-tests master
pushd matrix-react-end-to-end-tests
Expand Down
0 scripts/travis/install-deps.sh → scripts/ci/install-deps.sh
100755 → 100644
View file
Open in desktop
File renamed without changes.
4 changes: 2 additions & 2 deletions scripts/travis/riot-unit-tests.sh → scripts/ci/riot-unit-tests.sh
100755 → 100644
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
#!/bin/bash
#
# script which is run by the travis build (after `yarn test`).
# script which is run by the CI build (after `yarn test`).
#
# clones riot-web develop and runs the tests against our version of react-sdk.

set -ev

RIOT_WEB_DIR=riot-web

scripts/travis/build.sh
scripts/ci/build.sh
pushd "$RIOT_WEB_DIR"
yarn test
popd
10 changes: 10 additions & 0 deletions scripts/ci/unit-tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/bin/bash
#
# script which is run by the CI build (after `yarn test`).
#
# clones riot-web develop and runs the tests against our version of react-sdk.

set -ev

scripts/ci/build.sh
yarn test
5 changes: 3 additions & 2 deletions scripts/fetchdep.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,11 @@ clone() {
fi
}


# Try the PR author's branch in case it exists on the deps as well.
clone $TRAVIS_PULL_REQUEST_BRANCH
turt2live marked this conversation as resolved.
Show resolved Hide resolved
clone $BUILDKITE_BRANCH
# Try the target branch of the push or PR.
clone $TRAVIS_BRANCH
clone $BUILDKITE_PULL_REQUEST_BASE_BRANCH
Copy link
Collaborator

@jryans jryans Mar 15, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for bringing these back. The clone lines seem correct and are in the correct order, but the comments are mismatched ($BUILDKITE_BRANCH is the PR author's branch), so we should swap the comments. I believe Riot Web's fetch-develop.deps.sh has the right ordering for both code and comments.

# Try the current branch from Jenkins.
clone `"echo $GIT_BRANCH" | sed -e 's/^origin\///'`
# Use the default branch as the last resort.
Expand Down
10 changes: 0 additions & 10 deletions scripts/travis/unit-tests.sh
View file
Open in desktop

This file was deleted.

8 changes: 4 additions & 4 deletions yarn.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4065,10 +4065,10 @@ just-extend@^4.0.2:
resolved "https://registry.yarnpkg.com/just-extend/-/just-extend-4.0.2.tgz#f3f47f7dfca0f989c55410a7ebc8854b07108afc"
integrity sha512-FrLwOgm+iXrPV+5zDU6Jqu4gCRXbWEQg2O3SKONsWE4w7AXFRkryS53bpWdaL9cNol+AmR3AEYz6kn+o0fCPnw==

karma-chrome-launcher@^0.2.3:
version "0.2.3"
resolved "https://registry.yarnpkg.com/karma-chrome-launcher/-/karma-chrome-launcher-0.2.3.tgz#4c6d700d163a9d34c618efd87918be49e7a4a8c9"
integrity sha1-TG1wDRY6nTTGGO/YeRi+SeekqMk=
karma-chrome-launcher@^2.2.0:
version "2.2.0"
resolved "https://registry.yarnpkg.com/karma-chrome-launcher/-/karma-chrome-launcher-2.2.0.tgz#cf1b9d07136cc18fe239327d24654c3dbc368acf"
integrity sha512-uf/ZVpAabDBPvdPdveyk1EPgbnloPvFFGgmRhYLTDH7gEB4nZdSBk8yTU47w1g/drLSx5uMOkjKk7IWKfWg/+w==
dependencies:
fs-access "^1.0.0"
which "^1.2.1"
Expand Down