diff --git a/CHANGELOG-helmop.md b/CHANGELOG-helmop.md index 98321662a..144932d9d 100644 --- a/CHANGELOG-helmop.md +++ b/CHANGELOG-helmop.md @@ -1,3 +1,18 @@ +## 0.5.1 (2018-11-21) + +### Bug fixes + + - Helm releases will now stay put when an upgrade fails or the + Kubernetes API connectivity is flaky, instead of getting purged + [weaveworks/flux#1530](https://github.com/weaveworks/flux/pull/1530) + +### Thanks + +Thanks to @sfrique, @brantb and @squaremo for helping document the +issues leading to this bug fix, @stefanprodan for actually squashing +the bug and all others that may have gone unnoticed while writing this +release note. + ## 0.5.0 (2018-11-14) WARNING: this release of the Helm operator is not backward-compatible: diff --git a/CHANGELOG.md b/CHANGELOG.md index 7a4568323..134f19299 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,68 @@ This is the changelog for the Flux daemon; the changelog for the Helm operator is in [./CHANGELOG-helmop.md](./CHANGELOG-helmop.md). +## 1.8.2 (2018-12-19) + +This holiday season release fixes a handful of annoyances, and adds an +experimental `--watch` flag for following the progress of `fluxctl +release`. + +### Fixes + +- Respect proxy env entries for git operations + [weaveworks/flux#1556][#1556] +- Only push the "sync tag" when the synced revision has changed, + avoiding spurious notifications [weaveworks/flux#1605][#1605] +- Return any sync errors for workloads in the ListControllers API + [weaveworks/flux#1521][#1521] + +### Improvements + +- The experimental flag `fluxctl release --watch` shows the rollout + progress of workloads in the release [weaveworks/flux#1525][#1525] +- The example manifests now include resource requests, to help + Kubernetes with scheduling [weaveworks/flux#1541][#1541] +- We have a more comprehensive [example git + repo](https://github.com/weaveworks/flux-get-started), which is + mentioned consistently throughout the docs + [weaveworks/flux#1527][#1527] and [weaveworks/flux#1540][#1540]. +- Many clarifications and better structure in the docs + weaveworks/flux{[#1597], [#1595], [#1563], [#1555], [#1548], + [#1550], [#1549], [#1547], [#1508], [#1557]} +- Registry scanning produces far less log spam, and abandons scans as + soon as possible on being throttled [weaveworks/flux#1538][#1538] + +### Thanks + +Thanks to @Alien2150, @batpok, @bboreham, @brantb, @camilb, +@davidkarlsen, @dbluxo, @demikl, @dholbach, @dpgeekzero, @etos, +@hiddeco, @iandotmartin, @jakubbujny, @JeremyParker, @JimPruitt, +@johnraz, @kopachevsky, @kozejonaz, @leoblanc, @marccarre, +@marcincuber, @mgazza, @michalschott, @montyz, @ncabatoff, @nmaupu, +@Nogbit, @pdeveltere, @rampreethethiraj, @rndstr, @samisq, @scjudd, +@sfrique, @Smirl, @songsak2299, @squaremo, @stefanprodan, +@stephenmoloney, @Timer, @whereismyjetpack, @willnewby for +contributions in the period up to this release. + +[#1508]: https://github.com/weaveworks/flux/pull/1508 +[#1521]: https://github.com/weaveworks/flux/pull/1521 +[#1525]: https://github.com/weaveworks/flux/pull/1525 +[#1527]: https://github.com/weaveworks/flux/pull/1527 +[#1538]: https://github.com/weaveworks/flux/pull/1538 +[#1540]: https://github.com/weaveworks/flux/pull/1540 +[#1541]: https://github.com/weaveworks/flux/pull/1541 +[#1547]: https://github.com/weaveworks/flux/pull/1547 +[#1548]: https://github.com/weaveworks/flux/pull/1548 +[#1549]: https://github.com/weaveworks/flux/pull/1549 +[#1550]: https://github.com/weaveworks/flux/pull/1550 +[#1555]: https://github.com/weaveworks/flux/pull/1555 +[#1556]: https://github.com/weaveworks/flux/pull/1556 +[#1557]: https://github.com/weaveworks/flux/pull/1557 +[#1563]: https://github.com/weaveworks/flux/pull/1563 +[#1595]: https://github.com/weaveworks/flux/pull/1595 +[#1597]: https://github.com/weaveworks/flux/pull/1597 +[#1605]: https://github.com/weaveworks/flux/pull/1605 + ## 1.8.1 (2018-10-15) This release completes the support for `HelmRelease` resources as used @@ -47,7 +109,7 @@ range. If you run into difficulties relating to the `kubectl` version, ### Thanks -Thanks goes to @Ashiroq, @JimPruitt, @MansM, @Morriz, @Smirl, @Timer, +Thanks go to @Ashiroq, @JimPruitt, @MansM, @Morriz, @Smirl, @Timer, @aytekk, @bzon, @camilb, @claude-leveille, @demikl, @dholbach, @endrec, @foot, @hiddeco, @jrcole2884, @lelenanam, @marcusolsson, @mellena1, @montyz, @olib963, @rade, @rndstr, @sfitts, @squaremo, diff --git a/Makefile b/Makefile index f2cbfb411..86394d877 100644 --- a/Makefile +++ b/Makefile @@ -2,6 +2,7 @@ .PHONY: all release-bins clean realclean test integration-test check-generated SUDO := $(shell docker info > /dev/null 2> /dev/null || echo "sudo") + TEST_FLAGS?= include docker/kubectl.version @@ -40,8 +41,8 @@ clean: realclean: clean rm -rf ./cache -test: - PATH=${PATH}:${PWD}/bin go test ${TEST_FLAGS} $(shell go list ./... | grep -v "^github.com/weaveworks/flux/vendor" | sort -u) +test: build/helm + PATH=${PWD}/bin:${PWD}/build:${PATH} go test ${TEST_FLAGS} $(shell go list ./... | grep -v "^github.com/weaveworks/flux/vendor" | sort -u) build/.%.done: docker/Dockerfile.% mkdir -p ./build/docker/$* @@ -64,11 +65,13 @@ build/helm-operator: cmd/helm-operator/*.go CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o $@ $(LDFLAGS) -ldflags "-X main.version=$(shell ./docker/image-tag)" ./cmd/helm-operator build/kubectl: cache/kubectl-$(KUBECTL_VERSION) + mkdir -p build cp cache/kubectl-$(KUBECTL_VERSION) $@ strip $@ chmod a+x $@ build/helm: cache/helm-$(HELM_VERSION) + mkdir -p build cp cache/helm-$(HELM_VERSION) $@ strip $@ chmod a+x $@ @@ -76,16 +79,16 @@ build/helm: cache/helm-$(HELM_VERSION) cache/kubectl-$(KUBECTL_VERSION): docker/kubectl.version mkdir -p cache curl -L -o $(KUBECTL_TARGZ) "https://dl.k8s.io/$(KUBECTL_VERSION)/kubernetes-client-linux-amd64.tar.gz" - echo "$(KUBECTL_CHECKSUM) $(KUBECTL_TARGZ)" > "$(KUBECTL_TARGZ).checksum" - sha256sum -c $(KUBECTL_TARGZ).checksum + echo "$(KUBECTL_CHECKSUM) $(KUBECTL_TARGZ)" > "$(KUBECTL_TARGZ).checksum" + shasum -a 256 -c $(KUBECTL_TARGZ).checksum tar -C ./cache -xzf $(KUBECTL_TARGZ) kubernetes/client/bin/kubectl cp ./cache/kubernetes/client/bin/kubectl $@ cache/helm-$(HELM_VERSION): docker/helm.version mkdir -p cache curl -L -o $(HELM_TARGZ) "https://storage.googleapis.com/kubernetes-helm/helm-v$(HELM_VERSION)-linux-amd64.tar.gz" - echo "$(HELM_CHECKSUM) $(HELM_TARGZ)" > "$(HELM_TARGZ).checksum" - sha256sum -c "$(HELM_TARGZ).checksum" + echo "$(HELM_CHECKSUM) $(HELM_TARGZ)" > "$(HELM_TARGZ).checksum" + shasum -a 256 -c "$(HELM_TARGZ).checksum" tar -C ./cache -xzf $(HELM_TARGZ) linux-amd64/helm cp ./cache/linux-amd64/helm $@ diff --git a/README.md b/README.md index ae4e7d7b8..c468b881f 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ We believe in GitOps: - **You push code not containers.** Everything is controlled through pull requests. There is no learning curve for new devs, they just use your standard git PR process. The history in git allows you to recover - from any snapshot as you have an sequence of transactions. It is much + from any snapshot as you have a sequence of transactions. It is much more transparent to make operational changes by pull request, e.g. fix a production issue via a pull request instead of making changes to the running system. @@ -89,6 +89,16 @@ Get started by browsing through the documentation below: - [Frequently encountered issues](https://github.com/weaveworks/flux/labels/FAQ) - [Upgrading to Flux v1](/site/upgrading-to-1.0.md) +### Integrations + +As Flux is Open Source, integrations are very straight-forward. Here are +a few popular ones you might want to check out: + +- [Managing Helm releases the GitOps way](https://github.com/stefanprodan/gitops-helm) +- [OpenFaaS GitOps workflow with Flux](https://github.com/stefanprodan/openfaas-flux) +- [GitOps for Istio Canary deployments](https://github.com/stefanprodan/gitops-istio) +- [Fluxcloud to receive events from Flux](https://github.com/justinbarrick/fluxcloud) + ## Community & Developer information We welcome all kinds of contributions to Flux, be it code, issues you found, diff --git a/api/v6/api.go b/api/v6/api.go index b35fe94be..a95f2a4f3 100644 --- a/api/v6/api.go +++ b/api/v6/api.go @@ -36,6 +36,7 @@ type ControllerStatus struct { ReadOnly ReadOnlyReason Status string Rollout cluster.RolloutStatus + SyncError string Antecedent flux.ResourceID Labels map[string]string Automated bool diff --git a/chart/flux/CHANGELOG.md b/chart/flux/CHANGELOG.md new file mode 100644 index 000000000..bfccdef97 --- /dev/null +++ b/chart/flux/CHANGELOG.md @@ -0,0 +1,127 @@ +## 0.5.1 (2018-11-21) + +### Bug fixes + + - Removed CRD hook from chart + [weaveworks/flux#1536](https://github.com/weaveworks/flux/pull/1536) + +### Improvements + + - Updated Helm operator to `v0.5.1` + [weaveworks/flux#1536](https://github.com/weaveworks/flux/pull/1536) + - Updated chart README (removed Helm operator Git flags, fixed typos, + updated example repo and use the same Git URL format everywhere) + [weaveworks/flux#1527](https://github.com/weaveworks/flux/pull/1527) + +## 0.5.0 (2018-11-16) + +### Improvements + + - Updated Flux to `v1.8.1` and the Helm operator to `v0.5.0` + [weaveworks/flux#1522](https://github.com/weaveworks/flux/pull/1522) + - Adapted chart to new Helm operator CRD and args + [weaveworks/flux#1382](https://github.com/weaveworks/flux/pull/1382) + +## 0.4.1 (2018-11-04) + +### Bug fixes + + - Fixed indentation of `.Values.helmOperator.tls.caContent` + [weaveworks/flux#1484](https://github.com/weaveworks/flux/pull/1484) + +### Improvements + + - Updated Helm operator to `v0.4.0` + [weaveworks/flux#1487](https://github.com/weaveworks/flux/pull/1487) + - Added `--tiller-tls-hostname` Helm operator config flag to the chart + [weaveworks/flux#1484](https://github.com/weaveworks/flux/pull/1484) + - Include `valueFileSecrets` property in `helm-operator-crd.yaml` + [weaveworks/flux#1468](https://github.com/weaveworks/flux/pull/1468) + - Uniform language highlight on Helm chart README + [weaveworks/flux#1464](https://github.com/weaveworks/flux/pull/1463) + +## 0.4.0 (2018-10-25) + +### Bug fixes + + - Made maximum memcache item size configurable, fixes + `SERVER_ERROR object too large for cache` errors on large deployments + [weaveworks/flux#1453](https://github.com/weaveworks/flux/pull/1453) + - Fixed indentation of `aditionalArgs` + [weaveworks/flux#1417](https://github.com/weaveworks/flux/pull/1417) + +### Improvements + + - Updated Flux to `v1.8.0` and the Helm operator to `0.3.0` + [weaveworks/flux#1470](https://github.com/weaveworks/flux/pull/1470) + - Deprecated Flux `--registry-cache-expiry` config flag + [weaveworks/flux#1470](https://github.com/weaveworks/flux/pull/1470) + - Added and documented multiple values (s.a. `nodeSelector`, + `extraEnvs`, `git.timeout`) + [weaveworks/flux#1469](https://github.com/weaveworks/flux/pull/1469) + [weaveworks/flux#1446](https://github.com/weaveworks/flux/pull/1446) + [weaveworks/flux#1416](https://github.com/weaveworks/flux/pull/1416) + - Made it possible to enable Promotheus annotations + [weaveworks/flux#1462](https://github.com/weaveworks/flux/pull/1462) + +## 0.3.4 (2018-09-28) + +### Improvements + + - Updated Flux to `v1.7.1` + [weaveworks/flux#1405](https://github.com/weaveworks/flux/pull/1405) + - Custom SSH keys for Flux and Helm operator are now allowed + [weaveworks/flux#1391](https://github.com/weaveworks/flux/pull/1391) + +## 0.3.3 (2018-09-18) + +### Improvements + + - Updated Flux to `v1.7.0` and the Helm operator to `v0.2.1` + [weaveworks/flux#1368](https://github.com/weaveworks/flux/pull/1368) + - Added memcached verbose option + [weaveworks/flux#1350](https://github.com/weaveworks/flux/pull/1350) + - Allow overrides of `.kube/config` + [weaveworks/flux#1342](https://github.com/weaveworks/flux/pull/1342) + - Documentation improvements + [weaveworks/flux#1357](https://github.com/weaveworks/flux/pull/1357) + +## 0.3.2 (2018-08-31) + +### Improvements + + - Updated Flux to `v1.6.0` + [weaveworks/flux#1330](https://github.com/weaveworks/flux/pull/1330) + - Made the Helm operator CRD creation optional + [weaveworks/flux#1311](https://github.com/weaveworks/flux/pull/1311) + +## 0.3.0 (2018-08-24) + +### Improvements + + - Updated Helm operator to `v0.2.0` + [weaveworks/flux#1308](https://github.com/weaveworks/flux/pull/1308) + - Added Flux git label and registry options + [weaveworks/flux#1305](https://github.com/weaveworks/flux/pull/1305) + - Removed `.Values.git.gitPath` value + [weaveworks/flux#1305](https://github.com/weaveworks/flux/pull/1305) + - Documented how to use a private Git host + [weaveworks/flux#1299](https://github.com/weaveworks/flux/pull/1299) + - Added option to opt-in to logging of release diffs + [weaveworks/flux#1271](https://github.com/weaveworks/flux/pull/1272) + +## 0.2.2 (2018-08-09) + +### Bug fixes + + - Fixed indentation of `.Values.ssh.known_hosts` + [weaveworks/flux#1246](https://github.com/weaveworks/flux/pull/1246) + +### Improvements + + - Updated Flux to `v1.5.0` + [weaveworks/flux#1279](https://github.com/weaveworks/flux/pull/1279) + - Added openAPIV3Schema validation to Helm CRD + [weaveworks/flux#1253](https://github.com/weaveworks/flux/pull/1253) + - Fix markdown typo in README + [weaveworks/flux#1248](https://github.com/weaveworks/flux/pull/1248) diff --git a/chart/flux/Chart.yaml b/chart/flux/Chart.yaml index 78122ae56..fa7f090d3 100644 --- a/chart/flux/Chart.yaml +++ b/chart/flux/Chart.yaml @@ -1,8 +1,8 @@ apiVersion: v1 -appVersion: "1.8.0" +appVersion: "1.8.1" description: Flux is a tool that automatically ensures that the state of a cluster matches what is specified in version control name: flux -version: 0.5.0-beta +version: 0.5.3 home: https://weave.works sources: - https://github.com/weaveworks/flux @@ -10,4 +10,4 @@ maintainers: - name: stefanprodan email: stefan@weave.works engine: gotpl -icon: https://landscape.cncf.io/logos/flux.svg +icon: https://landscape.cncf.io/logos/weave-flux.svg diff --git a/chart/flux/README.md b/chart/flux/README.md index 8cd5ff180..7b8987b94 100755 --- a/chart/flux/README.md +++ b/chart/flux/README.md @@ -14,10 +14,10 @@ a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) p ### Kubernetes -Kubernetes >= v1.9 is recommended. Kubernetes v1.8 (the first to support +Kubernetes >= v1.10 is recommended. Kubernetes v1.8 (the first to support Custom Resources) appears to have problems with repeated application of custom resources (see https://github.com/kubernetes/kubernetes/issues/53379). -This means fluxd can fail to apply changes to FluxHelmRelease resources. +This means fluxd can fail to apply changes to HelmRelease resources. ### Helm @@ -27,24 +27,18 @@ until it can find one. # Git repo - - One repo containing both desired release state information and Charts - themselves. - - Release state information in the form of Custom Resources manifests is - located under a particular path ("releaseconfig" by default; can be - overriden). - - Charts are colocated under another path ("charts" by default; can be - overriden). Charts are subdirectories under the charts path. + - One repo containing cluster config (i.e., Kubernetes YAMLs) and zero or more git repos containing Charts themselves. + - Charts can be co-located with config in the git repo, or be from Helm repositories. - Custom Resource namespace reflects where the release should be done. Both the Helm release and its corresponding Custom Resource will live in this namespace. - - Example of a test repo: https://github.com/weaveworks/flux-helm-test + - Example of a test repo: https://github.com/weaveworks/flux-get-started ## Installation We put together a simple [Get Started guide](../../site/helm-get-started.md) which takes about 5-10 minutes to follow. -You will have a fully working Flux installation deploying workloads to your -cluster. +You will have a fully working Flux installation deploying workloads to your cluster. ## Installing Flux using Helm @@ -56,11 +50,13 @@ Add the weaveworks repo: helm repo add weaveworks https://weaveworks.github.io/flux ``` -#### To install the chart with the release name `flux`: +#### To install the chart with the release name `flux` + +Replace `weaveworks/flux-get-started` with your own git repository and run helm install: ```sh $ helm install --name flux \ ---set git.url=ssh://git@github.com/weaveworks/flux-example \ +--set git.url=git@github.com:weaveworks/flux-get-started \ --namespace flux \ weaveworks/flux ``` @@ -69,6 +65,7 @@ weaveworks/flux ```sh helm install --name flux \ +--set git.url=git@github.com:weaveworks/flux-get-started \ --set token=YOUR_WEAVE_CLOUD_SERVICE_TOKEN \ --namespace flux \ weaveworks/flux @@ -76,10 +73,19 @@ weaveworks/flux #### To install Flux with the Helm operator: +Apply the Helm Release CRD: + +```sh +kubectl apply -f https://raw.githubusercontent.com/weaveworks/flux/master/deploy-helm/flux-helm-release-crd.yaml +``` + +Install Flux with Helm: + ```sh $ helm install --name flux \ ---set git.url=ssh://git@github.com/weaveworks/flux-helm-test \ +--set git.url=git@github.com:weaveworks/flux-get-started \ --set helmOperator.create=true \ +--set helmOperator.createCRD=false \ --namespace flux \ weaveworks/flux ``` @@ -110,6 +116,7 @@ using an alternate mechanism. ```sh YOUR_GIT_HOST=your_git_host.example.com + YOUR_GIT_USER=your_git_user KNOWN_HOSTS='domain ssh-rsa line1 domain ecdsa-sha2-line2 domain ssh-ed25519 line3' @@ -117,7 +124,8 @@ using an alternate mechanism. helm install \ --name flux \ --set helmOperator.create=true \ - --set git.url="ssh://git@${YOUR_GIT_HOST}:weaveworks/flux-helm-test.git" \ + --set helmOperator.createCRD=false \ + --set git.url="git@${YOUR_GIT_HOST}:${YOUR_GIT_USER}/flux-get-started" \ --set-string ssh.known_hosts="${KNOWN_HOSTS}" \ --namespace flux \ chart/flux @@ -129,11 +137,13 @@ using an alternate mechanism. ```sh YOUR_GIT_HOST=your_git_host.example.com + YOUR_GIT_USER=your_git_user helm install \ --name flux \ --set helmOperator.create=true \ - --set git.url="ssh://git@${YOUR_GIT_HOST}:weaveworks/flux-helm-test.git" \ + --set helmOperator.createCRD=false \ + --set git.url="git@${YOUR_GIT_HOST}:${YOUR_GIT_USER}/flux-get-started" \ --set-file ssh.known_hosts=/tmp/flux_known_hosts \ --namespace flux \ chart/flux @@ -144,10 +154,11 @@ The [configuration](#configuration) section lists all the parameters that can be #### Setup Git deploy At startup Flux generates a SSH key and logs the public key. -Find the SSH public key with: +Find the SSH public key by installing [fluxctl](../../site/fluxctl.md) and +running: ```sh -kubectl -n flux logs deployment/flux | grep identity.pub | cut -d '"' -f2 +fluxctl identity ``` In order to sync your cluster state with GitHub you need to copy the public key and @@ -209,15 +220,11 @@ The following tables lists the configurable parameters of the Weave Flux chart a | `helmOperator.tag` | Helm operator image tag | `` | `helmOperator.pullPolicy` | Helm operator image pull policy | `IfNotPresent` | `helmOperator.updateChartDeps` | Update dependencies for charts | `true` +| `helmOperator.git.pollInterval` | Period at which to poll git repo for new commits | `git.pollInterval` +| `helmOperator.git.timeout` | Duration after which git operations time out | `git.timeout` | `helmOperator.chartsSyncInterval` | Interval at which to check for changed charts | `3m` | `helmOperator.chartsSyncTimeout` | Timeout when checking for changed charts | `1m` | `helmOperator.extraEnvs` | Extra environment variables for the Helm operator pod | `[]` -| `helmOperator.git.url` | URL of git repo with Helm charts | `git.url` -| `helmOperator.git.branch` | Branch of git repo to use for Helm charts | `master` -| `helmOperator.git.chartsPath` | Path within git repo to locate Helm charts (relative path) | `charts` -| `helmOperator.git.pollInterval` | Period at which to poll git repo for new commits | `git.pollInterval` -| `helmOperator.git.timeout` | Duration after which git operations time out | `git.timeout` -| `helmOperator.git.secretName` | Kubernetes secret with the SSH private key | None | `helmOperator.logReleaseDiffs` | Helm operator should log the diff when a chart release diverges (possibly insecure) | `false` | `helmOperator.tillerNamespace` | Namespace in which the Tiller server can be found | `kube-system` | `helmOperator.tls.enable` | Enable TLS for communicating with Tiller | `false` @@ -232,7 +239,7 @@ Specify each parameter using the `--set key=value[,key=value]` argument to `helm ```sh $ helm upgrade --install --wait flux \ ---set git.url=ssh://git@github.com/stefanprodan/podinfo \ +--set git.url=git@github.com:stefanprodan/k8s-podinfo \ --set git.path=deploy/auto-scaling,deploy/local-storage \ --namespace flux \ weaveworks/flux @@ -244,6 +251,6 @@ Update Weave Flux version with: ```sh helm upgrade --reuse-values flux \ ---set image.tag=1.7.1 \ +--set image.tag=1.8.1 \ weaveworks/flux ``` diff --git a/chart/flux/templates/NOTES.txt b/chart/flux/templates/NOTES.txt index 98683c0c0..aa7acd329 100644 --- a/chart/flux/templates/NOTES.txt +++ b/chart/flux/templates/NOTES.txt @@ -1,19 +1,10 @@ -1. Get the application URL by running these commands: -{{- if contains "NodePort" .Values.service.type }} - export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ template "flux.fullname" . }}) - export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}") - echo http://$NODE_IP:$NODE_PORT -{{- else if contains "LoadBalancer" .Values.service.type }} - NOTE: It may take a few minutes for the LoadBalancer IP to be available. - You can watch the status of by running 'kubectl get svc -w {{ template "flux.fullname" . }}' - export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ template "flux.fullname" . }} -o jsonpath='{.status.loadBalancer.ingress[0].ip}') - echo http://$SERVICE_IP:{{ .Values.service.port }} -{{- else if contains "ClusterIP" .Values.service.type }} - export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app={{ template "flux.name" . }},release={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}") - kubectl -n {{ .Release.Namespace }} port-forward $POD_NAME 8080:3030 & -{{- end }} +Get the Git deploy key by either (a) running -2. Get the Git deploy key by running these commands: - export FLUX_POD=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app={{ template "flux.name" . }},release={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}") - kubectl -n {{ .Release.Namespace }} logs $FLUX_POD | grep identity.pub | cut -d '"' -f2 + kubectl -n {{ .Release.Namespace }} logs deployment/{{ .Release.Name }} | grep identity.pub | cut -d '"' -f2 + +or by (b) installing fluxctl through +https://github.com/weaveworks/flux/blob/master/site/fluxctl.md#installing-fluxctl +and running: + + fluxctl identity diff --git a/chart/flux/templates/deployment.yaml b/chart/flux/templates/deployment.yaml index d9d6c34c7..e2197f954 100644 --- a/chart/flux/templates/deployment.yaml +++ b/chart/flux/templates/deployment.yaml @@ -23,17 +23,17 @@ spec: app: {{ template "flux.name" . }} release: {{ .Release.Name }} spec: - {{- if .Values.serviceAccount.create }} serviceAccountName: {{ template "flux.serviceAccountName" . }} - {{- end }} volumes: - name: kubedir configMap: name: {{ template "flux.fullname" . }}-kube-config + {{- if .Values.ssh.known_hosts }} - name: sshdir configMap: name: {{ template "flux.fullname" . }}-ssh-config defaultMode: 0600 + {{- end }} - name: git-key secret: {{- if .Values.git.secretName }} @@ -56,9 +56,11 @@ spec: volumeMounts: - name: kubedir mountPath: /root/.kubectl + {{- if .Values.ssh.known_hosts }} - name: sshdir mountPath: /root/.ssh readOnly: true + {{- end }} - name: git-key mountPath: /etc/fluxd/ssh readOnly: true diff --git a/chart/flux/templates/helm-operator-crd.yaml b/chart/flux/templates/helm-operator-crd.yaml index 6d7f7720f..ca8f85b16 100644 --- a/chart/flux/templates/helm-operator-crd.yaml +++ b/chart/flux/templates/helm-operator-crd.yaml @@ -11,7 +11,7 @@ metadata: release: {{ .Release.Name }} heritage: {{ .Release.Service }} annotations: - "helm.sh/hook": crd-install + "helm.sh/resource-policy": keep spec: group: flux.weave.works names: @@ -35,6 +35,9 @@ spec: releaseName: type: string pattern: "^[a-z0-9]([-a-z0-9]*[a-z0-9])?$" + timeout: + type: integer + format: int64 valueFileSecrets: type: array items: @@ -81,7 +84,7 @@ metadata: release: {{ .Release.Name }} heritage: {{ .Release.Service }} annotations: - "helm.sh/hook": crd-install + "helm.sh/resource-policy": keep spec: group: helm.integrations.flux.weave.works names: diff --git a/chart/flux/templates/helm-operator-deployment.yaml b/chart/flux/templates/helm-operator-deployment.yaml index c458c8131..9d393c63d 100644 --- a/chart/flux/templates/helm-operator-deployment.yaml +++ b/chart/flux/templates/helm-operator-deployment.yaml @@ -1,4 +1,3 @@ -{{- $gitPollInterval := default .Values.git.pollInterval .Values.helmOperator.git.pollInterval }} {{- $gitTimeout := default .Values.git.timeout .Values.helmOperator.git.timeout }} {{- if .Values.helmOperator.create -}} apiVersion: apps/v1beta2 @@ -26,14 +25,14 @@ spec: app: {{ template "flux.name" . }}-helm-operator release: {{ .Release.Name }} spec: - {{- if .Values.serviceAccount.create }} serviceAccountName: {{ template "flux.serviceAccountName" . }} - {{- end }} volumes: + {{- if .Values.ssh.known_hosts }} - name: sshdir configMap: name: {{ template "flux.fullname" . }}-ssh-config defaultMode: 0600 + {{- end }} - name: git-key secret: {{- if .Values.helmOperator.git.secretName }} @@ -61,10 +60,12 @@ spec: image: "{{ .Values.helmOperator.repository }}:{{ .Values.helmOperator.tag }}" imagePullPolicy: {{ .Values.helmOperator.pullPolicy }} volumeMounts: + {{- if .Values.ssh.known_hosts }} - name: sshdir mountPath: /root/.ssh/known_hosts subPath: known_hosts readOnly: true + {{- end }} - name: git-key mountPath: /etc/fluxd/ssh readOnly: true @@ -79,7 +80,6 @@ spec: {{- end }} {{- end }} args: - - --git-poll-interval={{ $gitPollInterval }} - --git-timeout={{ $gitTimeout }} - --charts-sync-interval={{ .Values.helmOperator.chartsSyncInterval }} - --update-chart-deps={{ .Values.helmOperator.updateChartDeps }} diff --git a/chart/flux/templates/memcached.yaml b/chart/flux/templates/memcached.yaml index 0a8dbf56b..c13a3fecc 100755 --- a/chart/flux/templates/memcached.yaml +++ b/chart/flux/templates/memcached.yaml @@ -23,7 +23,7 @@ spec: spec: containers: - name: memcached - image: memcached:1.4.25 + image: {{ .Values.memcached.repository }}:{{ .Values.memcached.tag }} imagePullPolicy: IfNotPresent args: - -m {{ .Values.memcached.maxMemory }} # Maximum memory to use, in megabytes. 64MB is default. diff --git a/chart/flux/templates/ssh.yaml b/chart/flux/templates/ssh.yaml index b75c87f52..70b8d1bdf 100644 --- a/chart/flux/templates/ssh.yaml +++ b/chart/flux/templates/ssh.yaml @@ -1,3 +1,4 @@ +{{- if .Values.ssh.known_hosts -}} apiVersion: v1 kind: ConfigMap metadata: @@ -13,3 +14,4 @@ data: {{ .Values.ssh.known_hosts }} {{- end }} {{- end }} +{{- end -}} diff --git a/chart/flux/values.yaml b/chart/flux/values.yaml index c79ee8296..608de3889 100644 --- a/chart/flux/values.yaml +++ b/chart/flux/values.yaml @@ -7,7 +7,7 @@ replicaCount: 1 image: repository: quay.io/weaveworks/flux - tag: 1.8.0 + tag: 1.8.1 pullPolicy: IfNotPresent service: @@ -18,7 +18,7 @@ helmOperator: create: false createCRD: true repository: quay.io/weaveworks/helm-operator - tag: 0.4.0 + tag: 0.5.1 pullPolicy: IfNotPresent # Update dependencies for charts updateChartDeps: true @@ -54,15 +54,10 @@ helmOperator: nodeSelector: {} tolerations: [] affinity: {} - resources: {} - # If you do want to specify resources, uncomment the following - # lines, adjust them as necessary, and remove the curly braces after 'resources:'. - # limits: - # cpu: 50m - # memory: 150Mi - # requests: - # cpu: 20m - # memory: 100Mi + resources: + requests: + cpu: 50m + memory: 64Mi rbac: # Specifies whether RBAC resources should be created @@ -75,15 +70,10 @@ serviceAccount: # If not set and create is true, a name is generated using the fullname template name: -resources: {} - # If you do want to specify resources, uncomment the following - # lines, adjust them as necessary, and remove the curly braces after 'resources:'. - # limits: - # cpu: 250m - # memory: 300Mi - # requests: - # cpu: 150m - # memory: 250Mi +resources: + requests: + cpu: 50m + memory: 64Mi nodeSelector: {} @@ -92,7 +82,7 @@ tolerations: [] affinity: {} git: - # URL of git repo with Kubernetes manifests; e.g. git.url=ssh://git@github.com/weaveworks/flux-example + # URL of git repo with Kubernetes manifests; e.g. git.url=ssh://git@github.com/weaveworks/flux-get-started url: "" # Branch of git repo to use for Kubernetes manifests branch: "master" @@ -134,6 +124,8 @@ registry: cacheExpiry: memcached: + repository: memcached + tag: 1.4.25 verbose: false maxItemSize: 1m maxMemory: 64 diff --git a/cmd/fluxctl/release_cmd.go b/cmd/fluxctl/release_cmd.go index 51b8d1c45..872096090 100644 --- a/cmd/fluxctl/release_cmd.go +++ b/cmd/fluxctl/release_cmd.go @@ -4,10 +4,14 @@ import ( "context" "fmt" "io" + "time" "github.com/spf13/cobra" "github.com/weaveworks/flux" + "github.com/weaveworks/flux/api/v11" + "github.com/weaveworks/flux/api/v6" + "github.com/weaveworks/flux/cluster" "github.com/weaveworks/flux/job" "github.com/weaveworks/flux/update" ) @@ -23,6 +27,7 @@ type controllerReleaseOpts struct { dryRun bool interactive bool force bool + watch bool outputOpts cause update.Cause @@ -57,6 +62,7 @@ func (opts *controllerReleaseOpts) Command() *cobra.Command { cmd.Flags().BoolVar(&opts.dryRun, "dry-run", false, "Do not release anything; just report back what would have been done") cmd.Flags().BoolVar(&opts.interactive, "interactive", false, "Select interactively which containers to update") cmd.Flags().BoolVarP(&opts.force, "force", "f", false, "Disregard locks and container image filters (has no effect when used with --all or --update-all-images)") + cmd.Flags().BoolVarP(&opts.watch, "watch", "w", false, "Watch rollout progress during release") // Deprecated cmd.Flags().StringSliceVarP(&opts.services, "service", "s", []string{}, "Service to release") @@ -81,6 +87,8 @@ func (opts *controllerReleaseOpts) RunE(cmd *cobra.Command, args []string) error switch { case len(opts.controllers) <= 0 && !opts.allControllers: return newUsageError("please supply either --all, or at least one --controller=") + case opts.watch && opts.dryRun: + return newUsageError("cannot use --watch with --dry-run") case opts.force && opts.allControllers && opts.allImages: return newUsageError("--force has no effect when used with --all and --update-all-images") case opts.force && opts.allControllers: @@ -90,6 +98,7 @@ func (opts *controllerReleaseOpts) RunE(cmd *cobra.Command, args []string) error } var controllers []update.ResourceSpec + if opts.allControllers { controllers = []update.ResourceSpec{update.ResourceSpecAll} } else { @@ -131,7 +140,7 @@ func (opts *controllerReleaseOpts) RunE(cmd *cobra.Command, args []string) error } if kind == update.ReleaseKindPlan { - fmt.Fprintf(cmd.OutOrStderr(), "Submitting dry-run release...\n") + fmt.Fprintf(cmd.OutOrStderr(), "Submitting dry-run release ...\n") } else { fmt.Fprintf(cmd.OutOrStderr(), "Submitting release ...\n") } @@ -153,12 +162,11 @@ func (opts *controllerReleaseOpts) RunE(cmd *cobra.Command, args []string) error return err } + result, err := awaitJob(ctx, opts.API, jobID) + if err != nil { + return err + } if opts.interactive { - result, err := awaitJob(ctx, opts.API, jobID) - if err != nil { - return err - } - spec, err := promptSpec(cmd.OutOrStdout(), result, opts.verbosity) spec.Force = opts.force if err != nil { @@ -166,7 +174,7 @@ func (opts *controllerReleaseOpts) RunE(cmd *cobra.Command, args []string) error return nil } - fmt.Fprintf(cmd.OutOrStderr(), "Submitting selected release...\n") + fmt.Fprintf(cmd.OutOrStderr(), "Submitting selected release ...\n") jobID, err = opts.API.UpdateManifests(ctx, update.Spec{ Type: update.Containers, Cause: opts.cause, @@ -179,7 +187,68 @@ func (opts *controllerReleaseOpts) RunE(cmd *cobra.Command, args []string) error opts.dryRun = false } - return await(ctx, cmd.OutOrStdout(), cmd.OutOrStderr(), opts.API, jobID, !opts.dryRun, opts.verbosity) + + err = await(ctx, cmd.OutOrStdout(), cmd.OutOrStderr(), opts.API, jobID, !opts.dryRun, opts.verbosity) + if !opts.watch || err != nil { + return err + } + + fmt.Fprintf(cmd.OutOrStderr(), "Monitoring rollout ...\n") + for { + completed := 0 + services, err := opts.API.ListServicesWithOptions(ctx, v11.ListServicesOptions{Services: result.Result.AffectedResources()}) + if err != nil { + return err + } + + for _, service := range services { + writeRolloutStatus(service, opts.verbosity) + + if service.Status == cluster.StatusReady { + completed++ + } + + if service.Rollout.Messages != nil { + fmt.Fprintf(cmd.OutOrStderr(), "There was a problem releasing %s:\n", service.ID) + for _, msg := range service.Rollout.Messages { + fmt.Fprintf(cmd.OutOrStderr(), "%s\n", msg) + } + return nil + } + } + + if completed == len(services) { + fmt.Fprintf(cmd.OutOrStderr(), "All controllers ready.\n") + return nil + } + + time.Sleep(2000 * time.Millisecond) + } +} + +func writeRolloutStatus(service v6.ControllerStatus, verbosity int) { + w := newTabwriter() + fmt.Fprintf(w, "CONTROLLER\tCONTAINER\tIMAGE\tRELEASE\tREPLICAS\n") + + if len(service.Containers) > 0 { + c := service.Containers[0] + fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%d/%d", service.ID, c.Name, c.Current.ID, service.Status, service.Rollout.Updated, service.Rollout.Desired) + if verbosity > 0 { + fmt.Fprintf(w, " (%d outdated, %d ready)", service.Rollout.Outdated, service.Rollout.Ready) + } + fmt.Fprintf(w, "\n") + for _, c := range service.Containers[1:] { + fmt.Fprintf(w, "\t%s\t%s\t\t\n", c.Name, c.Current.ID) + } + } else { + fmt.Fprintf(w, "%s\t\t\t%s\t%d/%d", service.ID, service.Status, service.Rollout.Updated, service.Rollout.Desired) + if verbosity > 0 { + fmt.Fprintf(w, " (%d outdated, %d ready)", service.Rollout.Outdated, service.Rollout.Ready) + } + fmt.Fprintf(w, "\n") + } + fmt.Fprintln(w) + w.Flush() } func promptSpec(out io.Writer, result job.Result, verbosity int) (update.ReleaseContainersSpec, error) { diff --git a/cmd/fluxd/main.go b/cmd/fluxd/main.go index 3c29cd150..f41a5fd4d 100644 --- a/cmd/fluxd/main.go +++ b/cmd/fluxd/main.go @@ -79,7 +79,7 @@ func main() { kubernetesKubectl = fs.String("kubernetes-kubectl", "", "Optional, explicit path to kubectl tool") versionFlag = fs.Bool("version", false, "Get version number") // Git repo & key etc. - gitURL = fs.String("git-url", "", "URL of git repo with Kubernetes manifests; e.g., git@github.com:weaveworks/flux-example") + gitURL = fs.String("git-url", "", "URL of git repo with Kubernetes manifests; e.g., git@github.com:weaveworks/flux-get-started") gitBranch = fs.String("git-branch", "master", "branch of git repo to use for Kubernetes manifests") gitPath = fs.StringSlice("git-path", []string{}, "relative paths within the git repo to locate Kubernetes manifests") gitUser = fs.String("git-user", "Weave Flux", "username to use as git committer") diff --git a/cmd/helm-operator/main.go b/cmd/helm-operator/main.go index ee521e1e1..ea0fcf5a5 100644 --- a/cmd/helm-operator/main.go +++ b/cmd/helm-operator/main.go @@ -49,8 +49,7 @@ var ( logReleaseDiffs *bool updateDependencies *bool - gitPollInterval *time.Duration - gitTimeout *time.Duration + gitTimeout *time.Duration listenAddr *string ) @@ -93,8 +92,10 @@ func init() { logReleaseDiffs = fs.Bool("log-release-diffs", false, "log the diff when a chart release diverges; potentially insecure") updateDependencies = fs.Bool("update-chart-deps", true, "Update chart dependencies before installing/upgrading a release") - gitPollInterval = fs.Duration("git-poll-interval", 5*time.Minute, "period on which to poll for changes to the git repo") + _ = fs.Duration("git-poll-interval", 0, "") gitTimeout = fs.Duration("git-timeout", 20*time.Second, "duration after which git operations time out") + + fs.MarkDeprecated("git-poll-interval", "no longer used; has been replaced by charts-sync-interval") } func main() { @@ -181,7 +182,7 @@ func main() { chartSync := chartsync.New(log.With(logger, "component", "chartsync"), chartsync.Polling{Interval: *chartsSyncInterval}, chartsync.Clients{KubeClient: *kubeClient, IfClient: *ifClient}, - rel, chartsync.Config{LogDiffs: *logReleaseDiffs, UpdateDeps: *updateDependencies}) + rel, chartsync.Config{LogDiffs: *logReleaseDiffs, UpdateDeps: *updateDependencies, GitTimeout: *gitTimeout}) chartSync.Run(shutdown, errc, shutdownWg) // OPERATOR - CUSTOM RESOURCE CHANGE SYNC ----------------------------------------------- diff --git a/daemon/daemon.go b/daemon/daemon.go index b2180ab6b..4884a11bf 100644 --- a/daemon/daemon.go +++ b/daemon/daemon.go @@ -136,12 +136,17 @@ func (d *Daemon) ListServicesWithOptions(ctx context.Context, opts v11.ListServi case service.IsSystem: readOnly = v6.ReadOnlySystem } + var syncError string + if service.SyncError != nil { + syncError = service.SyncError.Error() + } res = append(res, v6.ControllerStatus{ ID: service.ID, Containers: containers2containers(service.ContainersOrNil()), ReadOnly: readOnly, Status: service.Status, Rollout: service.Rollout, + SyncError: syncError, Antecedent: service.Antecedent, Labels: service.Labels, Automated: policies.Has(policy.Automated), diff --git a/daemon/daemon_test.go b/daemon/daemon_test.go index 5ea954b49..6cdaca4b6 100644 --- a/daemon/daemon_test.go +++ b/daemon/daemon_test.go @@ -290,7 +290,7 @@ func TestDaemon_ListImagesWithOptions(t *testing.T) { { name: "Override container field selection", opts: v10.ListImagesOptions{ - Spec: specAll, + Spec: specAll, OverrideContainerFields: []string{"Name", "Current", "NewAvailableImagesCount"}, }, expectedImages: []v6.ImageStatus{ @@ -320,7 +320,7 @@ func TestDaemon_ListImagesWithOptions(t *testing.T) { { name: "Override container field selection with invalid field", opts: v10.ListImagesOptions{ - Spec: specAll, + Spec: specAll, OverrideContainerFields: []string{"InvalidField"}, }, expectedImages: nil, diff --git a/daemon/loop.go b/daemon/loop.go index daaf02851..386baebd7 100644 --- a/daemon/loop.go +++ b/daemon/loop.go @@ -321,7 +321,7 @@ func (d *Daemon) doSync(logger log.Logger) (retErr error) { Error: n.Result.Error(), }, Spec: event.ReleaseSpec{ - Type: event.ReleaseContainersSpecType, + Type: event.ReleaseContainersSpecType, ReleaseContainersSpec: &spec, }, Cause: n.Spec.Cause, @@ -411,23 +411,23 @@ func (d *Daemon) doSync(logger log.Logger) (retErr error) { } // Move the tag and push it so we know how far we've gotten. - { - ctx, cancel := context.WithTimeout(ctx, gitOpTimeout) - err := working.MoveSyncTagAndPush(ctx, newTagRev, "Sync pointer") - cancel() - if err != nil { - return err - } - } - if oldTagRev != newTagRev { + { + ctx, cancel := context.WithTimeout(ctx, gitOpTimeout) + err := working.MoveSyncTagAndPush(ctx, newTagRev, "Sync pointer") + cancel() + if err != nil { + return err + } + } logger.Log("tag", d.GitConfig.SyncTag, "old", oldTagRev, "new", newTagRev) - ctx, cancel := context.WithTimeout(ctx, gitOpTimeout) - err := d.Repo.Refresh(ctx) - cancel() - return err + { + ctx, cancel := context.WithTimeout(ctx, gitOpTimeout) + err := d.Repo.Refresh(ctx) + cancel() + return err + } } - return nil } diff --git a/deploy-helm/flux-helm-release-crd.yaml b/deploy-helm/flux-helm-release-crd.yaml index b41682ce6..162aad095 100644 --- a/deploy-helm/flux-helm-release-crd.yaml +++ b/deploy-helm/flux-helm-release-crd.yaml @@ -26,6 +26,9 @@ spec: releaseName: type: string pattern: "^[a-z0-9]([-a-z0-9]*[a-z0-9])?$" + timeout: + type: integer + format: int64 valueFileSecrets: type: array items: diff --git a/deploy-helm/helm-operator-deployment.yaml b/deploy-helm/helm-operator-deployment.yaml index 47531e8ef..3513822dd 100644 --- a/deploy-helm/helm-operator-deployment.yaml +++ b/deploy-helm/helm-operator-deployment.yaml @@ -60,8 +60,12 @@ spec: # There are no ":latest" images for helm-operator. Find the most recent # release or image version at https://quay.io/weaveworks/helm-operator # and replace the tag here. - image: quay.io/weaveworks/helm-operator:0.5.0 + image: quay.io/weaveworks/helm-operator:0.5.1 imagePullPolicy: IfNotPresent + resources: + requests: + cpu: 50m + memory: 64Mi volumeMounts: # Include this if you need to mount a customised known_hosts or ssh_config # file; you'll also need the volume declared above. diff --git a/deploy/flux-deployment.yaml b/deploy/flux-deployment.yaml index 2bb744223..15d0f313c 100644 --- a/deploy/flux-deployment.yaml +++ b/deploy/flux-deployment.yaml @@ -54,8 +54,12 @@ spec: # There are no ":latest" images for flux. Find the most recent # release or image version at https://quay.io/weaveworks/flux # and replace the tag here. - image: quay.io/weaveworks/flux:1.7.0 + image: quay.io/weaveworks/flux:1.8.1 imagePullPolicy: IfNotPresent + resources: + requests: + cpu: 50m + memory: 64Mi ports: - containerPort: 3030 # informational volumeMounts: @@ -96,7 +100,7 @@ spec: - --ssh-keygen-dir=/var/fluxd/keygen # replace or remove the following URL - - --git-url=git@github.com:weaveworks/flux-example + - --git-url=git@github.com:weaveworks/flux-get-started - --git-branch=master # include these next two to connect to an "upstream" service diff --git a/git/operations.go b/git/operations.go index 732e306df..0f175c1e3 100644 --- a/git/operations.go +++ b/git/operations.go @@ -7,6 +7,7 @@ import ( "fmt" "io" "io/ioutil" + "os" "os/exec" "strings" @@ -18,6 +19,9 @@ import ( // If true, every git invocation will be echoed to stdout const trace = false +// Env vars that are allowed to be inherited from the os +var allowedEnvVars = []string{"http_proxy", "https_proxy", "no_proxy"} + func config(ctx context.Context, workingDir, user, email string) error { for k, v := range map[string]string{ "user.name": user, @@ -289,7 +293,16 @@ func execGitCmd(ctx context.Context, dir string, out io.Writer, args ...string) } func env() []string { - return []string{"GIT_TERMINAL_PROMPT=0"} + env := []string{"GIT_TERMINAL_PROMPT=0"} + + // include allowed env vars from os + for _, k := range allowedEnvVars { + if v, ok := os.LookupEnv(k); ok { + env = append(env, k+"="+v) + } + } + + return env } // check returns true if there are changes locally. diff --git a/integrations/apis/flux.weave.works/v1beta1/types.go b/integrations/apis/flux.weave.works/v1beta1/types.go index 4b4b42f12..e248cf0a5 100644 --- a/integrations/apis/flux.weave.works/v1beta1/types.go +++ b/integrations/apis/flux.weave.works/v1beta1/types.go @@ -2,7 +2,7 @@ package v1beta1 import ( "github.com/ghodss/yaml" - v1 "k8s.io/api/core/v1" + "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/helm/pkg/chartutil" @@ -65,11 +65,21 @@ type RepoChartSource struct { // FluxHelmReleaseSpec is the spec for a FluxHelmRelease resource // FluxHelmReleaseSpec type HelmReleaseSpec struct { - ChartSource `json:"chart"` - ReleaseName string `json:"releaseName,omitempty"` - + ChartSource `json:"chart"` + ReleaseName string `json:"releaseName,omitempty"` ValueFileSecrets []v1.LocalObjectReference `json:"valueFileSecrets,omitempty"` HelmValues `json:",inline"` + // Install or upgrade timeout in seconds + // +optional + Timeout *int64 `json:"timeout,omitempty"` +} + +// GetTimeout returns the install or upgrade timeout (defaults to 300s) +func (r HelmRelease) GetTimeout() int64 { + if r.Spec.Timeout == nil { + return 300 + } + return *r.Spec.Timeout } type HelmReleaseStatus struct { diff --git a/integrations/apis/flux.weave.works/v1beta1/zz_generated.deepcopy.go b/integrations/apis/flux.weave.works/v1beta1/zz_generated.deepcopy.go index c0a249777..b66b8c6de 100644 --- a/integrations/apis/flux.weave.works/v1beta1/zz_generated.deepcopy.go +++ b/integrations/apis/flux.weave.works/v1beta1/zz_generated.deepcopy.go @@ -165,6 +165,15 @@ func (in *HelmReleaseSpec) DeepCopyInto(out *HelmReleaseSpec) { copy(*out, *in) } in.HelmValues.DeepCopyInto(&out.HelmValues) + if in.Timeout != nil { + in, out := &in.Timeout, &out.Timeout + if *in == nil { + *out = nil + } else { + *out = new(int64) + **out = **in + } + } return } diff --git a/integrations/helm/chartsync/chartsync.go b/integrations/helm/chartsync/chartsync.go index fe6fc1499..ec8063747 100644 --- a/integrations/helm/chartsync/chartsync.go +++ b/integrations/helm/chartsync/chartsync.go @@ -88,6 +88,7 @@ type Config struct { ChartCache string LogDiffs bool UpdateDeps bool + GitTimeout time.Duration } func (c Config) WithDefaults() Config { @@ -264,7 +265,7 @@ func mirrorName(chartSource *fluxv1beta1.GitChartSource) string { func (chs *ChartChangeSync) maybeMirror(fhr fluxv1beta1.HelmRelease) { chartSource := fhr.Spec.ChartSource.GitChartSource if chartSource != nil { - if ok := chs.mirrors.Mirror(mirrorName(chartSource), git.Remote{chartSource.GitURL}, git.ReadOnly); !ok { + if ok := chs.mirrors.Mirror(mirrorName(chartSource), git.Remote{chartSource.GitURL}, git.Timeout(chs.config.GitTimeout), git.ReadOnly); !ok { chs.logger.Log("info", "started mirroring repo", "repo", chartSource.GitURL) } } @@ -324,7 +325,7 @@ func (chs *ChartChangeSync) reconcileReleaseDef(fhr fluxv1beta1.HelmRelease) { chartPath = filepath.Join(chartClone.export.Dir(), chartSource.Path) if chs.config.UpdateDeps { - if err := updateDependencies(chartPath); err != nil { + if err := updateDependencies(chartPath, ""); err != nil { chs.setCondition(&fhr, fluxv1beta1.HelmReleaseReleased, v1.ConditionFalse, ReasonDependencyFailed, err.Error()) chs.logger.Log("warning", "Failed to update chart dependencies", "namespace", fhr.Namespace, "name", fhr.Name, "error", err) return diff --git a/integrations/helm/chartsync/deps.go b/integrations/helm/chartsync/deps.go index 00e985fd2..085a7b38a 100644 --- a/integrations/helm/chartsync/deps.go +++ b/integrations/helm/chartsync/deps.go @@ -7,9 +7,26 @@ import ( "path/filepath" ) -func updateDependencies(chartDir string) error { +// helmHome is optional; if it's "", it's left to default +func updateDependencies(chartDir, helmhome string) error { var hasLockFile bool + // sanity check: does the chart directory exist + chartInfo, err := os.Stat(chartDir) + switch { + case err != nil: + return err + case !chartInfo.IsDir(): + return fmt.Errorf("chart path %s is not a directory", chartDir) + } + + // check if the requirements file exists + reqFilePath := filepath.Join(chartDir, "requirements.yaml") + reqInfo, err := os.Stat(reqFilePath) + if err != nil || reqInfo.IsDir() { + return nil + } + // We are going to use `helm dep build`, which tries to update the // dependencies in charts/ by looking at the file // `requirements.lock` in the chart directory. If the lockfile @@ -29,6 +46,9 @@ func updateDependencies(chartDir string) error { } cmd := exec.Command("helm", "repo", "update") + if helmhome != "" { + cmd.Args = append(cmd.Args, "--home", helmhome) + } out, err := cmd.CombinedOutput() if err != nil { return fmt.Errorf("could not update repo: %s", string(out)) diff --git a/integrations/helm/chartsync/deps_test.go b/integrations/helm/chartsync/deps_test.go new file mode 100644 index 000000000..bc19f5354 --- /dev/null +++ b/integrations/helm/chartsync/deps_test.go @@ -0,0 +1,51 @@ +package chartsync + +import ( + "io/ioutil" + "os" + "os/exec" + "testing" +) + +func Test_updateDependencies(t *testing.T) { + helmhome, err := ioutil.TempDir("", "flux-helm") + if err != nil { + t.Fatal(err) + } + defer os.RemoveAll(helmhome) + cmd := exec.Command("helm", "init", "--client-only", "--home", helmhome) + if err := cmd.Run(); err != nil { + t.Fatal(err) + } + + type args struct { + chartDir string + } + tests := []struct { + name string + args args + wantErr bool + }{ + { + name: "Chart without dependencies", + args: args{ + chartDir: "test/chart-without-deps", + }, + wantErr: false, + }, + { + name: "non-existent chart", + args: args{ + chartDir: "test/folder-doesnt-exist", + }, + wantErr: true, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if err := updateDependencies(tt.args.chartDir, helmhome); (err != nil) != tt.wantErr { + t.Errorf("updateDependencies() error = %v, wantErr %v", err, tt.wantErr) + } + }) + } +} diff --git a/integrations/helm/chartsync/test/chart-without-deps/.helmignore b/integrations/helm/chartsync/test/chart-without-deps/.helmignore new file mode 100644 index 000000000..f0c131944 --- /dev/null +++ b/integrations/helm/chartsync/test/chart-without-deps/.helmignore @@ -0,0 +1,21 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*~ +# Various IDEs +.project +.idea/ +*.tmproj diff --git a/integrations/helm/chartsync/test/chart-without-deps/Chart.yaml b/integrations/helm/chartsync/test/chart-without-deps/Chart.yaml new file mode 100644 index 000000000..a062526fb --- /dev/null +++ b/integrations/helm/chartsync/test/chart-without-deps/Chart.yaml @@ -0,0 +1,5 @@ +apiVersion: v1 +appVersion: "1.0" +description: A bare bones Helm chart for Kubernetes +name: chart-without-deps +version: 0.1.0 diff --git a/integrations/helm/chartsync/test/chart-without-deps/values.yaml b/integrations/helm/chartsync/test/chart-without-deps/values.yaml new file mode 100644 index 000000000..e69de29bb diff --git a/integrations/helm/release/release.go b/integrations/helm/release/release.go index 2983ae084..d0bfa6778 100644 --- a/integrations/helm/release/release.go +++ b/integrations/helm/release/release.go @@ -31,12 +31,10 @@ const ( // Release contains clients needed to provide functionality related to helm releases type Release struct { logger log.Logger - HelmClient *k8shelm.Client } type Releaser interface { - GetCurrent() (map[string][]DeployInfo, error) GetDeployedRelease(name string) (*hapi_release.Release, error) Install(dir string, releaseName string, fhr flux_v1beta1.HelmRelease, action Action, opts InstallOptions) (*hapi_release.Release, error) } @@ -60,7 +58,7 @@ func New(logger log.Logger, helmClient *k8shelm.Client) *Release { } // GetReleaseName either retrieves the release name from the Custom Resource or constructs a new one -// in the form : $Namespace-$CustomResourceName +// in the form : $Namespace-$CustomResourceName func GetReleaseName(fhr flux_v1beta1.HelmRelease) string { namespace := fhr.Namespace if namespace == "" { @@ -105,17 +103,16 @@ func (r *Release) canDelete(name string) (bool, error) { "PENDING_ROLLBACK": 8, */ status := rls.GetInfo().GetStatus() - r.logger.Log("info", fmt.Sprintf("Release [%s] status: %s", name, status.Code.String())) switch status.Code { case 1, 4: - r.logger.Log("info", fmt.Sprintf("Deleting release (%s)", name)) + r.logger.Log("info", fmt.Sprintf("Deleting release %s", name)) return true, nil case 2: - r.logger.Log("info", fmt.Sprintf("Release (%s) already deleted", name)) + r.logger.Log("info", fmt.Sprintf("Release %s already deleted", name)) return false, nil default: - r.logger.Log("info", fmt.Sprintf("Release (%s) with status %s cannot be deleted", name, status.Code.String())) - return false, fmt.Errorf("Release (%s) with status %s cannot be deleted", name, status.Code.String()) + r.logger.Log("info", fmt.Sprintf("Release %s with status %s cannot be deleted", name, status.Code.String())) + return false, fmt.Errorf("release %s with status %s cannot be deleted", name, status.Code.String()) } } @@ -139,7 +136,10 @@ func (r *Release) Install(chartPath, releaseName string, fhr flux_v1beta1.HelmRe return nil, fmt.Errorf("error statting path given for chart %s: %s", chartPath, err.Error()) } - r.logger.Log("info", "releaseName", releaseName, "action", action, "options", fmt.Sprintf("%+v", opts)) + r.logger.Log("info", fmt.Sprintf("processing release %s", releaseName), + "action", fmt.Sprintf("%v", action), + "options", fmt.Sprintf("%+v", opts), + "timeout", fmt.Sprintf("%vs", fhr.GetTimeout())) // Read values from given valueFile paths (configmaps, etc.) mergedValues := chartutil.Values{} @@ -179,22 +179,20 @@ func (r *Release) Install(chartPath, releaseName string, fhr flux_v1beta1.HelmRe k8shelm.ReleaseName(releaseName), k8shelm.InstallDryRun(opts.DryRun), k8shelm.InstallReuseName(opts.ReuseName), - /* - helm.InstallReuseName(i.replace), - helm.InstallDisableHooks(i.disableHooks), - helm.InstallTimeout(i.timeout), - helm.InstallWait(i.wait) - */ + k8shelm.InstallTimeout(fhr.GetTimeout()), ) if err != nil { r.logger.Log("error", fmt.Sprintf("Chart release failed: %s: %#v", releaseName, err)) - // if an install fails, purge the release and keep retrying - r.logger.Log("info", fmt.Sprintf("Deleting failed release: [%s]", releaseName)) - _, err = r.HelmClient.DeleteRelease(releaseName, k8shelm.DeletePurge(true)) - if err != nil { - r.logger.Log("error", fmt.Sprintf("Release deletion error: %#v", err)) - return nil, err + // purge the release if the install failed but only if this is the first revision + history, err := r.HelmClient.ReleaseHistory(releaseName, k8shelm.WithMaxHistory(2)) + if err == nil && len(history.Releases) == 1 && history.Releases[0].Info.Status.Code == hapi_release.Status_FAILED { + r.logger.Log("info", fmt.Sprintf("Deleting failed release: [%s]", releaseName)) + _, err = r.HelmClient.DeleteRelease(releaseName, k8shelm.DeletePurge(true)) + if err != nil { + r.logger.Log("error", fmt.Sprintf("Release deletion error: %#v", err)) + return nil, err + } } return nil, err } @@ -208,15 +206,7 @@ func (r *Release) Install(chartPath, releaseName string, fhr flux_v1beta1.HelmRe chartPath, k8shelm.UpdateValueOverrides(rawVals), k8shelm.UpgradeDryRun(opts.DryRun), - /* - helm.UpgradeRecreate(u.recreate), - helm.UpgradeForce(u.force), - helm.UpgradeDisableHooks(u.disableHooks), - helm.UpgradeTimeout(u.timeout), - helm.ResetValues(u.resetValues), - helm.ReuseValues(u.reuseValues), - helm.UpgradeWait(u.wait)) - */ + k8shelm.UpgradeTimeout(fhr.GetTimeout()), ) if err != nil { @@ -253,29 +243,6 @@ func (r *Release) Delete(name string) error { return nil } -// GetCurrent provides Chart releases (stored in tiller ConfigMaps) -// output: -// map[namespace][release name] = nil -func (r *Release) GetCurrent() (map[string][]DeployInfo, error) { - response, err := r.HelmClient.ListReleases() - if err != nil { - return nil, r.logger.Log("error", err) - } - r.logger.Log("info", fmt.Sprintf("Number of Chart releases: %d\n", response.GetCount())) - - relsM := make(map[string][]DeployInfo) - var depl []DeployInfo - - for _, r := range response.GetReleases() { - ns := r.Namespace - depl = relsM[ns] - - depl = append(depl, DeployInfo{Name: r.Name}) - relsM[ns] = depl - } - return relsM, nil -} - // annotateResources annotates each of the resources created (or updated) // by the release so that we can spot them. func (r *Release) annotateResources(release *hapi_release.Release, fhr flux_v1beta1.HelmRelease) error { @@ -296,8 +263,7 @@ func (r *Release) annotateResources(release *hapi_release.Release, fhr flux_v1be return err } -// fhrResourceID constructs a flux.ResourceID for a HelmRelease -// resource. +// fhrResourceID constructs a flux.ResourceID for a HelmRelease resource. func fhrResourceID(fhr flux_v1beta1.HelmRelease) flux.ResourceID { return flux.MakeResourceID(fhr.Namespace, "HelmRelease", fhr.Name) } diff --git a/internal_docs/releasing.md b/internal_docs/releasing.md index ba04f8885..1d35921ec 100644 --- a/internal_docs/releasing.md +++ b/internal_docs/releasing.md @@ -1,27 +1,131 @@ -# How to make a release of flux +# How to release Flux and the Helm operator -This process will create a new tagged release of flux, push dockerfiles and upload the `fluxctl` binary to GitHub releases. +The release process needs to do these things: + + - create a new release on GitHub, with a tag + - push Docker image(s) to quay.io + - possibly upload the `fluxctl` binaries to the GitHub release + - make sure the version is entered into the checkpoint database so + that up-to-date checks report back accurate information + +Much of this is automated, but it needs a human to turn the wheel. + +## Overview + +The Flux daemon and the Helm operator have separate releases, and use +different branches and tags. Flux daemon releases use just a semver +version, like `1.8.1`, and the Helm operator uses the prefix "helm", +e.g., `helm-0.5.0`. + +Each minor version has its own "release series" branch, from which +patch releases will be put together, called e.g., `release/1.8.x`, or +for the Helm operator, `release/helm-0.5.x`. + +The CircleCI script runs builds for tags, which push Docker images and +upload binaries. This is triggered by creating a release in GitHub, +which will create the tag. ## Requirements -- Circle CI must have a secret environmental variable called `GITHUB_TOKEN` which is a personal access token. + +- CircleCI must have a secret environmental variable called + `GITHUB_TOKEN` which is a personal access token. (This is almost + certainly already set up, but mentioned here in case it needs to be + reinstated.) ## Release process -1. Alter and commit the /CHANGELOG.md file to signify what has changed in this version. -2. Ensure example deployment is up-to-date. (Check image tags) -3 Push that to a PR and have it reviewed. - 3.1 Merge! -4. Create a new release: https://github.com/weaveworks/flux/releases/new -5. Fill in the version number for the name and tag. The version number should conform to [semver](semver.org); i.e., look like `1.2.3` (NB: no leading 'v'); optionally with a pre-release suffix, e.g., `1.0.0-beta` -6. Fill in the Description field (possibly a copy paste from the CHANGELOG.md) -7. Click "Publish release" -8. Add a new release to https://checkpoint-api.weave.works/admin +**Preparing the release PR** + +1. If the release is a new minor version, create a "release series" + branch and push it to GitHub. + +Depending on what is to be includeed in the release, you may need to +pick a point from which branch that is not HEAD of master. But +usually, it will be HEAD of master. + +2. From the release series branch, create _another_ branch for the + particular release. This will be what you submit as a PR. + +For example, + +```sh +git checkout release/1.8.x +git pull origin +git checkout -b release/1.8.1 +``` + +3. Put the commits you want in the release, into your branch + +If you just created a new release series branch, i.e., this is a +`x.y.0` patch release, you may already have what you need, because +you've just branched from master. + +If this is _not_ the first release on this branch, you will need to +either merge master, or cherry-pick commits from master, to get the +things you want in the release. + +4. Put an entry into the changelog + +For the Flux daemon, it's `CHANGELOG.md`; for the Helm operator, it's +`CHANGELOG-helmop.md`. Follow the format established, and commit your +change. + +If you cherry-picked commits, remember to only mention those changes. + +To compile a list of people (GitHub usernames) to thank, you can use a +script (if you have access to weaveworks/dx) or peruse the commits/PRs +merged/issues since the last release. There's no exact way to do +it. Be generous. + +5. Post the branch as a PR to the release series + +Push the patch release branch -- e.g., `release/1.8.1` -- to GitHub, +and create a PR from it. + +**Please note** You will need to change the branch the PR targets, +from `master` to the release series, e.g., `release/1.8.x`, while +creating the PR. + +Get the PR reviewed, and merge it. + +**Creating the release** + +6. [Create a release in GitHub](https://github.com/weaveworks/flux/releases/new) + +Use a tag name as explained above; semver for the flux daemon, `helm-` +then the semver for the Helm operator. + +Copy and paste the changelog entry. You may need to remove newlines +that have been inserted by your editor, so that it wraps nicely. + +Publishing the release will create the tag, and that will trigger CI +to build images and binaries. + +**After publishing the release** + +7. Put an entry in the checkpoint database + +Add a row to the [checkpoint +database](https://checkpoint-api.weave.works/admin) (or ask someone at +Weaveworks to do so). This is so that the up-to-date check will report +the latest available version correctly. + +8. Merge the release series branch back into master, so it has the + changelog entry. + +You can do this by creating a new PR in GitHub -- you don't need to +create any new branches, since you want to merge a branch that already +exists. -Circle will then run the build and upload the built binaries to the "Downloads" section of the release. +9. Consider updating the deploy manifest examples and the Helm + chart. -## Outputs +The example manifests are in [deploy](./deploy/) and +[deploy-helm](./deploy-helm/). Check the changes included in the +release, to see if arguments, volume mounts, etc., have changed. -The most recent binaries are always available at: https://github.com/weaveworks/flux/releases/latest +You can do these as additional PRs. Read on, for how to publish a new +Helm chart version. ## Helm chart release process diff --git a/registry/cache/warming.go b/registry/cache/warming.go index 78fa68dda..5ef1b0f1c 100644 --- a/registry/cache/warming.go +++ b/registry/cache/warming.go @@ -271,16 +271,21 @@ func (w *Warmer) warm(ctx context.Context, now time.Time, logger log.Logger, id // w.Burst, so limit the number of fetching goroutines to that. fetchers := make(chan struct{}, w.burst) awaitFetchers := &sync.WaitGroup{} - awaitFetchers.Add(len(toUpdate)) + + ctxc, cancel := context.WithCancel(ctx) + var once sync.Once + defer cancel() updates: for _, up := range toUpdate { select { - case <-ctx.Done(): + case <-ctxc.Done(): break updates case fetchers <- struct{}{}: } + awaitFetchers.Add(1) + go func(update update) { defer func() { awaitFetchers.Done(); <-fetchers }() @@ -291,13 +296,22 @@ func (w *Warmer) warm(ctx context.Context, now time.Time, logger log.Logger, id } // Get the image from the remote - entry, err := client.Manifest(ctx, imageID.Tag) + entry, err := client.Manifest(ctxc, imageID.Tag) if err != nil { if err, ok := errors.Cause(err).(net.Error); ok && err.Timeout() { // This was due to a context timeout, don't bother logging return } - errorLogger.Log("err", err, "ref", imageID) + + // abort the image tags fetching if we've been rate limited + if strings.Contains(err.Error(), "429") { + once.Do(func() { + errorLogger.Log("warn", "aborting image tag fetching due to rate limiting, will try again later") + cancel() + }) + } else { + errorLogger.Log("err", err, "ref", imageID) + } return } diff --git a/site/daemon.md b/site/daemon.md index 022e2e303..a761d63cc 100644 --- a/site/daemon.md +++ b/site/daemon.md @@ -33,6 +33,11 @@ Version controlling of cluster manifests provides reproducibility and a historic Deployment is pinned to a particular image tag. New deployment will not proceed upon triggered release. +# More information + +Setting up and configuring fluxd is discussed in our [standalone setup](./standalone-setup.md) +document. + # Flags fluxd requires setup and offers customization though a multitude of flags. @@ -44,7 +49,7 @@ fluxd requires setup and offers customization though a multitude of flags. |--kubernetes-kubectl | | optional, explicit path to kubectl tool| |--version | false | output the version number and exit | |**Git repo & key etc.** | || -|--git-url | | URL of git repo with Kubernetes manifests; e.g., `git@github.com:weaveworks/flux-example`| +|--git-url | | URL of git repo with Kubernetes manifests; e.g., `git@github.com:weaveworks/flux-get-started`| |--git-branch | `master` | branch of git repo to use for Kubernetes manifests| |--git-ci-skip | false | when set, fluxd will append `\n\n[ci skip]` to its commit messages | |--git-ci-skip-message | `""` | if provided, fluxd will append this to commit messages (overrides --git-ci-skip`) | @@ -55,16 +60,16 @@ fluxd requires setup and offers customization though a multitude of flags. |--git-label | | label to keep track of sync progress; overrides both --git-sync-tag and --git-notes-ref| |--git-sync-tag | `flux-sync` | tag to use to mark sync progress for this cluster (old config, still used if --git-label is not supplied)| |--git-notes-ref | `flux` | ref to use for keeping commit annotations in git notes| -|--git-poll-interval | `5 minutes` | period at which to fetch any new commits from the git repo | -|--git-timeout | `20 seconds` | duration after which git operations time out | +|--git-poll-interval | `5m` | period at which to fetch any new commits from the git repo | +|--git-timeout | `20s` | duration after which git operations time out | |**syncing** | | control over how config is applied to the cluster | -|--sync-interval | `5 minutes` | apply the git config to the cluster at least this often. New commits may provoke more frequent syncs | +|--sync-interval | `5m` | apply the git config to the cluster at least this often. New commits may provoke more frequent syncs | |**registry cache** | | (none of these need overriding, usually) | |--memcached-hostname | `memcached` | hostname for memcached service to use for caching image metadata| -|--memcached-timeout | `1 second` | maximum time to wait before giving up on memcached requests| +|--memcached-timeout | `1s` | maximum time to wait before giving up on memcached requests| |--memcached-service | `memcached` | SRV service used to discover memcache servers| -|--registry-cache-expiry | `1 hour` | Duration to keep cached registry tag info. Must be < 1 month.| -|--registry-poll-interval| `5 minutes` | period at which to poll registry for new images| +|--registry-cache-expiry | `1h` | Duration to keep cached registry tag info. Must be < 1 month.| +|--registry-poll-interval| `5m` | period at which to poll registry for new images| |--registry-rps | `200` | maximum registry requests per second per host| |--registry-burst | `125` | maximum number of warmer connections to remote and memcache| |--registry-insecure-host| [] | registry hosts to use HTTP for (instead of HTTPS) | @@ -81,4 +86,3 @@ fluxd requires setup and offers customization though a multitude of flags. |**SSH key generation** | | | |--ssh-keygen-bits | | -b argument to ssh-keygen (default unspecified)| |--ssh-keygen-type | | -t argument to ssh-keygen (default unspecified)| - diff --git a/site/faq.md b/site/faq.md index d7d81809e..07cca957e 100644 --- a/site/faq.md +++ b/site/faq.md @@ -3,6 +3,39 @@ title: Weave Flux FAQ menu_order: 60 --- +- [General questions](#general-questions) + * [What does Flux do?](#what-does-flux-do) + * [How does it automate deployment?](#how-does-it-automate-deployment) + * [How is that different from a bash script?](#how-is-that-different-from-a-bash-script) + * [Why should I automate deployment?](#why-should-i-automate-deployment) + * [I thought Flux was about service routing?](#i-thought-flux-was-about-service-routing) + * [Are there nightly builds I can run?](#are-there-nightly-builds-i-can-run) +- [Technical questions](#technical-questions) + * [Does it work only with one git repository?](#does-it-work-only-with-one-git-repository) + * [Do I have to put my application code and config in the same git repo?](#do-i-have-to-put-my-application-code-and-config-in-the-same-git-repo) + * [Is there any special directory layout I need in my git repo?](#is-there-any-special-directory-layout-i-need-in-my-git-repo) + * [Why does Flux need a git ssh key with write access?](#why-does-flux-need-a-git-ssh-key-with-write-access) + * [Does Flux automatically sync changes back to git?](#does-flux-automatically-sync-changes-back-to-git) + * [How do I give Flux access to an image registry?](#how-do-i-give-flux-access-to-an-image-registry) + * [How often does Flux check for new images?](#how-often-does-flux-check-for-new-images) + * [How often does Flux check for new git commits (and can I make it sync faster)?](#how-often-does-flux-check-for-new-git-commits-and-can-i-make-it-sync-faster) + * [How do I use my own deploy key?](#how-do-i-use-my-own-deploy-key) + * [Why are my images not showing up in the list of images?](#why-are-my-images-not-showing-up-in-the-list-of-images) + * [Why do my image tags appear out of order?](#why-do-my-image-tags-appear-out-of-order) + * [How do I use a private git host (or one that's not github.com, gitlab.com, or bitbucket.org)?](#how-do-i-use-a-private-git-host-or-one-thats-not-githubcom-gitlabcom-or-bitbucketorg) + * [Will Flux delete resources that are no longer in the git repository?](#will-flux-delete-resources-that-are-no-longer-in-the-git-repository) + * [Why does my CI pipeline keep getting triggered?](#why-does-my-ci-pipeline-keep-getting-triggered) + * [What is the "sync tag"; or, why do I see a `flux-sync` tag in my git repo?](#what-is-the-sync-tag-or-why-do-i-see-a-flux-sync-tag-in-my-git-repo) + * [Can I restrict the namespaces that Flux can see or operate on?](#can-i-restrict-the-namespaces-that-flux-can-see-or-operate-on) + * [Can I change the namespace Flux puts things in by default?](#can-i-change-the-namespace-flux-puts-things-in-by-default) + * [Can I temporarily make Flux ignore a deployment?](#can-i-temporarily-make-flux-ignore-a-deployment) +- [Flux Helm Operator questions](#flux-helm-operator-questions) + * [I'm using SSL between Helm and Tiller. How can I configure Flux to use the certificate?](#im-using-ssl-between-helm-and-tiller-how-can-i-configure-flux-to-use-the-certificate) + * [I've deleted a FluxHelmRelease file from Git. Why is the Helm release still running on my cluster?](#ive-deleted-a-fluxhelmrelease-file-from-git-why-is-the-helm-release-still-running-on-my-cluster) + * [I've manually deleted a Helm release. Why is Flux not able to restore it?](#ive-manually-deleted-a-helm-release-why-is-flux-not-able-to-restore-it) + * [I've uninstalled Flux and all my Helm releases are gone. Why is that?](#ive-uninstalled-flux-and-all-my-helm-releases-are-gone-why-is-that) + * [I have a dedicated Kubernetes cluster per environment and I want to use the same Git repo for all. How can I do that?](#i-have-a-dedicated-kubernetes-cluster-per-environment-and-i-want-to-use-the-same-git-repo-for-all-how-can-i-do-that) + ## General questions Also see [the introduction](/site/introduction.md). @@ -126,7 +159,7 @@ There are exceptions: - In some environments, authorisation provided by the platform is used instead of image pull secrets. Google Container Registry works this way, for example (and we have introduced a special case for it - so Flux will work there too). See below regarding ECR. + so Flux will work there too with image pull secrets). See below regarding ECR. To work around the exceptional cases, you can mount a docker config into the Flux container. See the argument `--docker-config` in @@ -213,23 +246,13 @@ happen: - Flux just hasn't fetched the image metadata yet. This may be the case if you've only just started using a particular image in a workload. - Flux can't get suitable credentials for the image repository. At - present, it looks at `imagePullSecret`s attached to workloads (but - not to service accounts; see - [weaveworks/flux#1043](https://github.com/weaveworks/flux/issues/1043)), - and a Docker config file if you mount one into the fluxd container + present, it looks at `imagePullSecret`s attached to workloads, + service accounts and a Docker config file if you mount one into the fluxd container (see the [command-line usage](./daemon.md)). - Flux doesn't know how to obtain registry credentials for ECR. A workaround is described in [weaveworks/flux#539](https://github.com/weaveworks/flux/issues/539#issuecomment-394588423) - - Flux doesn't yet understand what to do with image repositories that - have images for more than one architecture; see - [weaveworks/flux#741](https://github.com/weaveworks/flux/issues/741). At - present there's no workaround for this, if you are not in control - of the image repository in question (or you are, but you need to - have multi-arch manifests). - - Flux doesn't yet examine `initContainer`s when cataloguing the - images used by workloads. See - [weaveworks/flux#702](https://github.com/weaveworks/flux/issues/702) + - Flux excludes images with no suitable manifest (linux amd64) in manifestlist - Flux doesn't yet understand image refs that use digests instead of tags; see [weaveworks/flux#885](https://github.com/weaveworks/flux/issues/885). @@ -416,7 +439,7 @@ Flux doesn't delete resources, there is an [issue](https://github.com/weaveworks In order to delete a Helm release first remove the file from Git and afterwards run: ```yaml -kubectl delete fluxhelmrelease/my-release +kubectl delete helmrelease/my-release ``` The Flux Helm operator will receive the delete event and will purge the Helm release. @@ -428,7 +451,7 @@ You need to use the `helm delete --purge` option only then Flux will be able rei ### I've uninstalled Flux and all my Helm releases are gone. Why is that? -On `FluxHelmRelease` CRD deletion, Kubernetes will remove all `FluxHelmRelease` CRs triggering a Helm purge for each release created by Flux. +On `HelmRelease` CRD deletion, Kubernetes will remove all `HelmRelease` resources triggering a Helm purge for each release created by Flux. To avoid this you have to manually delete the Flux Helm Operator with `kubectl -n flux delete deployment/flux-helm-operator` before running `helm delete flux`. ### I have a dedicated Kubernetes cluster per environment and I want to use the same Git repo for all. How can I do that? @@ -436,14 +459,4 @@ To avoid this you have to manually delete the Flux Helm Operator with `kubectl - For each cluster create a Git branch in your config repo. When installing Flux set the Git branch using `--set git.branch=cluster-name` and set a unique label for each cluster `--set git.label=cluster-name`. -### I have a dedicated Git repo for my Helm charts. How can I point Flux Helm Operator to it? - -When installing Flux with Helm you can override the Operator Git settings using `--set helmOperator.git.url=`. - -If you are using GitHub you need to create a SSH key for Helm Operator: -* generate a SSH key named identity: `ssh-keygen -q -N "" -f ./identity` -* create a Kubernetes secret: `kubectl -n flux create secret generic helm-ssh --from-file=./identity` -* delete the private key: `rm ./identity` -* add `./identity.pub` as a read-only deployment key in your GitHub repo where the charts are -* set the secret name with `--set helmOperator.git.secretName=helm-ssh` diff --git a/site/fluxctl.md b/site/fluxctl.md index ab38640f8..875da86b1 100644 --- a/site/fluxctl.md +++ b/site/fluxctl.md @@ -3,6 +3,36 @@ title: Using Weave Flux menu_order: 40 --- +- [Installing fluxctl](#installing-fluxctl) + * [Mac OS](#mac-os) + * [Linux](#linux) + + [Arch Linux](#arch-linux) + * [Binary releases](#binary-releases) +- [Connecting fluxctl to the daemon](#connecting-fluxctl-to-the-daemon) + * [Flux API service](#flux-api-service) + * [Add an SSH deploy key to the repository](#add-an-ssh-deploy-key-to-the-repository) + + [1. Allow flux to generate a key for you](#1-allow-flux-to-generate-a-key-for-you) + + [2. Specify a key to use](#2-specify-a-key-to-use) +- [What is a Controller](#what-is-a-controller) +- [Viewing Controllers](#viewing-controllers) +- [Inspecting the Version of a Container](#inspecting-the-version-of-a-container) +- [Releasing a Controller](#releasing-a-controller) +- [Turning on Automation](#turning-on-automation) +- [Turning off Automation](#turning-off-automation) +- [Rolling back a Controller](#rolling-back-a-controller) +- [Locking a Controller](#locking-a-controller) +- [Releasing an image to a locked controller](#releasing-an-image-to-a-locked-controller) +- [Unlocking a Controller](#unlocking-a-controller) +- [Recording user and message with the triggered action](#recording-user-and-message-with-the-triggered-action) +- [Image Tag Filtering](#image-tag-filtering) + * [Filter pattern types](#filter-pattern-types) + + [Glob](#glob) + + [Semver](#semver) + + [Regexp](#regexp) + * [Actions triggered through `fluxctl`](#actions-triggered-through-fluxctl) + * [Errors due to author customization](#errors-due-to-author-customization) +- [Using Annotations](#using-annotations) + All of the features of Flux are accessible from within [Weave Cloud](https://cloud.weave.works). @@ -563,5 +593,44 @@ filtering annotations take the form [`regexp`](#regexp). Filter values use the same syntax as when the filter is configured using fluxctl. +Here's a simple but complete deployment file with annotations: + +``` +apiVersion: apps/v1 +kind: Deployment +metadata: + name: podinfo + namespace: demo + labels: + app: podinfo + annotations: + flux.weave.works/automated: "true" + flux.weave.works/tag.podinfod: semver:~1.3 +spec: + selector: + matchLabels: + app: podinfo + template: + metadata: + labels: + app: podinfo + spec: + containers: + - name: podinfod + image: stefanprodan/podinfo:1.3.2 + ports: + - containerPort: 9898 + name: http + command: + - ./podinfo + - --port=9898 +``` + +Things to notice: + +1. The annotations are made in `metadata.annotations`, not in `spec.template.metadata`. +2. The `flux.weave.works/tag.`... references the container name `podinfod`, this will change based on your container name. If you have multiple containers you would have multiple lines like that. +3. The value for the `flux.weave.works/tag.`... annotation should includes the filter pattern type, in this case `semver`. + Annotations can also be used to tell Flux to temporarily ignore certain manifests using `flux.weave.works/ignore: "true"`. Read more about this in the [FAQ](faq.md#can-i-temporarily-make-flux-ignore-a-deployment). diff --git a/site/get-started.md b/site/get-started.md index 9a29f315d..15c223704 100644 --- a/site/get-started.md +++ b/site/get-started.md @@ -3,6 +3,16 @@ title: Installing Weave Flux Manually menu_order: 10 --- +- [Get started with Flux](#get-started-with-flux) + * [Prerequisites](#prerequisites) + + [A Note on GKE with RBAC enabled](#a-note-on-gke-with-rbac-enabled) + * [Set up Flux](#set-up-flux) + * [Deploying Flux to the cluster](#deploying-flux-to-the-cluster) + * [Giving write access](#giving-write-access) + * [Committing a small change](#committing-a-small-change) + * [Confirm the change landed](#confirm-the-change-landed) + * [Conclusion](#conclusion) + # Get started with Flux This short guide shows a self-contained example of Flux and just @@ -19,19 +29,6 @@ You will need to have Kubernetes set up. For a quick local test, you can use `minikube` or `kubeadm`. Any other Kubernetes setup will work as well though. -When using a cluster in the cloud (e.g. GKE), use nodes with at least 2 CPU's. -When using nodes with only 1 CPU (like `n1-standard-1`), an upgrade -may be stuck with not enough CPU resources. This issue usually manifests itself -in the form of pods hanging in the PENDING state, which looks something like: - -```sh -$ kubectl describe pod/helloworld-... | tail -3 -Events: - Type Reason Age From Message - ---- ------ ---- ---- ------- - Warning FailedScheduling 3m (x37 over 13m) default-scheduler 0/2 nodes are available: 2 Insufficient cpu. -``` - ### A Note on GKE with RBAC enabled > If working on e.g. GKE with RBAC enabled, you will need to add a clusterrolebinding: @@ -64,7 +61,7 @@ $EDITOR deploy/flux-deployment.yaml ``` In our example we are going to use -[flux-example](https://github.com/weaveworks/flux-example). If you +[flux-get-started](https://github.com/weaveworks/flux-get-started). If you want to use that too, be sure to create a fork of it on Github and add the git URL to the config file above. @@ -87,15 +84,13 @@ watch kubectl get pods --all-namespaces ## Giving write access At startup Flux generates a SSH key and logs the public key. Find -the SSH public key with: +the SSH public key by installing [fluxctl](./fluxctl.md) and +runnning: ```sh -kubectl logs deployment/flux | grep identity.pub | cut -d '"' -f2 +fluxctl identity ``` -*Note:* If you have downloaded [fluxctl](./fluxctl.md) already, you can use -`fluxctl identity` as well. - In order to sync your cluster state with git you need to copy the public key and create a deploy key with write access on your GitHub repository. @@ -105,19 +100,19 @@ click on **Add deploy key**, give it a name, check **Allow write access**, paste the Flux public key and click **Add key**. (Or replace `YOURUSER` with your Github ID in this url: -`https://github.com/YOURUSER/flux-example/settings/keys/new` and +`https://github.com/YOURUSER/flux-get-started/settings/keys/new` and paste the key there.) ## Committing a small change In this example we are using a simple example of a webservice and change its configuration to use a different message. The easiest -way is to edit your fork of `flux-example` and change the `msg` argument. +way is to edit your fork of `flux-get-started` and change the `PODINFO_UI_COLOR` env var to `blue`. Replace `YOURUSER` in -`https://github.com/YOURUSER/flux-example/blob/master/helloworld-deploy.yaml` +`https://github.com/YOURUSER/flux-get-started/blob/master/workloads/podinfo-dep.yaml` with your Github ID), open the URL in your browser, edit the file, -change the argument value and commit the file. +change the env var value and commit the file. You can check out the Flux logs with: @@ -135,10 +130,11 @@ To access our webservice and check out its welcome message, simply run: ```sh -kubectl port-forward deployment/helloworld 8080:80 & -curl localhost:8080 +kubectl -n demo port-forward deployment/podinfo 9898:9898 & ``` +Open your browser and navigate to `http://localhost:9898`. + ## Conclusion As you can see, the actual steps to set up Flux, get our app diff --git a/site/helm-get-started.md b/site/helm-get-started.md index 1de8616d6..8992ba02c 100644 --- a/site/helm-get-started.md +++ b/site/helm-get-started.md @@ -3,6 +3,14 @@ title: Installing Weave Flux using Helm menu_order: 20 --- +- [Get started with Flux using Helm](#get-started-with-flux-using-helm) + * [Prerequisites](#prerequisites) + * [Install Weave Flux](#install-weave-flux) + * [Giving write access](#giving-write-access) + * [Committing a small change](#committing-a-small-change) + * [Conclusion](#conclusion) +- [Next](#next) + # Get started with Flux using Helm If you are using Helm already, this guide is for you. By the end @@ -19,19 +27,6 @@ You will need to have Kubernetes set up. To get up and running fast, you might want to use `minikube` or `kubeadm`. Any other Kubernetes setup will work as well though. -When using a cluster in the cloud (e.g. GKE), use nodes with at least 2 CPU's. -When using nodes with only 1 CPU (like `n1-standard-1`), an upgrade -may be stuck with not enough CPU resources. This issue usually manifests itself -in the form of pods hanging in the PENDING state, which looks something like: - -```sh -$ kubectl describe pod/helloworld-... | tail -3 -Events: - Type Reason Age From Message - ---- ------ ---- ---- ------- - Warning FailedScheduling 3m (x37 over 13m) default-scheduler 0/2 nodes are available: 2 Insufficient cpu. -``` - Download Helm: - On MacOS: @@ -41,8 +36,7 @@ Download Helm: ``` - On Linux: - - Download the [latest - release](https://github.com/kubernetes/helm/releases/latest), + - Download the [latest release](https://github.com/kubernetes/helm/releases/latest), unpack the tarball and put the binary in your `$PATH`. Now create a service account and a cluster role binding for Tiller: @@ -69,12 +63,18 @@ Add the Flux repository of Weaveworks: helm repo add weaveworks https://weaveworks.github.io/flux ``` +Apply the Helm Release CRD: + +```sh +kubectl apply -f https://raw.githubusercontent.com/weaveworks/flux/master/deploy-helm/flux-helm-release-crd.yaml +``` + In this next step you install Weave Flux using `helm`. Simply - 1. Fork [flux-helm-test](https://github.com/weaveworks/flux-helm-test) - on Github and - 1. Install Weave Flux and its Helm Operator by specifying your fork - URL: + 1. Fork [flux-get-started](https://github.com/weaveworks/flux-get-started) + on Github and replace the `weaveworks` with your GitHub username in + [here](https://github.com/weaveworks/flux-get-started/blob/master/releases/ghost.yaml#L13) + 1. Install Weave Flux and its Helm Operator by specifying your fork URL: *Just make sure you replace `YOURUSER` with your GitHub username in the command below:* @@ -82,10 +82,10 @@ In this next step you install Weave Flux using `helm`. Simply - Using a public git server from `bitbucket.com`, `github.com` or `gitlab.com`: ```sh - helm install --name flux \ + helm upgrade -i flux \ --set helmOperator.create=true \ - --set git.url=ssh://git@github.com/YOURUSER/flux-helm-test \ - --set helmOperator.git.chartsPath=charts \ + --set helmOperator.createCRD=false \ + --set git.url=git@github.com:YOURUSER/flux-get-started \ --namespace flux \ weaveworks/flux ``` @@ -102,7 +102,7 @@ impatient, run the following command and see the pod creation process. ```sh -watch kubectl get pods --all-namespaces +watch kubectl -n flux get pods ``` You will notice that `flux` and `flux-helm-operator` will start @@ -110,17 +110,17 @@ turning up in the `flux` namespace. ## Giving write access -For the real benefits of GitOps, Flux will need acccess to your +For the real benefits of GitOps, Flux will need access to your git repository to update configuration if necessary. To facilitate that you will need to add a deploy key to your fork of the repository. This is pretty straight-forward as Flux generates a SSH key and -logs the public key at startup. Find the SSH public key with: +logs the public key at startup. Find the SSH public key by +installing [fluxctl](./fluxctl.md) and running: ```sh -FLUX_POD=$(kubectl get pods --namespace flux -l "app=flux,release=flux" -o jsonpath="{.items[0].metadata.name}") -kubectl -n flux logs $FLUX_POD | grep identity.pub | cut -d '"' -f2 +fluxctl identity ``` In order to sync your cluster state with git you need to copy the @@ -132,33 +132,36 @@ click on **Add deploy key**, give it a name, check **Allow write access**, paste the Flux public key and click **Add key**. (Or replace `YOURUSER` with your Github ID in this url: -`https://github.com/YOURUSER/flux-helm-test/settings/keys/new` and +`https://github.com/YOURUSER/flux-get-started/settings/keys/new` and paste the key there.) Once Flux has confirmed access to the repository, it will start -deploying the workloads of `flux-helm-test`. After a while you +deploying the workloads of `flux-get-started`. After a while you will be able to see the Helm releases listed like so: ```sh -helm list --namespace test +helm list --namespace demo ``` ## Committing a small change -`flux-helm-test` is a very simple example in which two services -(mongodb and mariadb) are deployed. Here we will simply update the +`flux-get-started` is a simple example in which three services +(mongodb, redis and ghost) are deployed. Here we will simply update the version of mongodb to a newer version to see if Flux will pick this up and update our cluster. -The easiest way is to update your fork of `flux-helm-test` and +The easiest way is to update your fork of `flux-get-started` and change the `image` argument. -Replace `YOURUSER` in `https://github.com/YOURUSER/flux-helm-test/edit/master/releases/mongodb_release.yaml` +Replace `YOURUSER` in `https://github.com/YOURUSER/flux-get-started/edit/master/releases/mongodb.yaml` with your Github ID, open the URL in your browser, edit the file, -change the `image:` line to the following: +change the `tag:` line to the following: ```yaml - image: bitnami/mongodb:3.7.9-r13 + values: + image: + repository: bitnami/mongodb + tag: 4.0.2 ``` Commit the change to your `master` branch. It will now get @@ -171,23 +174,20 @@ kubectl -n flux logs deployment/flux -f ``` The default sync frequency for Flux using the Helm chart is -30 seconds. This can be tweaked easily. By observing the logs +five minutes. This can be tweaked easily. By observing the logs you can see when the change landed in the cluster. -## Confirm the change landed - -To access our webservice and check out its welcome message, simply -run: +Confirm the change landed with: ```sh -kubectl describe -n test deployment.apps/mongodb-database-mongodb | grep Image +kubectl describe -n demo deployment/mongodb | grep Image ``` ## Conclusion As you can see, the actual steps to set up Flux, get our app deployed, give Flux access to it and see modifications land are -very straight-forward and are a quite natural work-flow. +very straight-forward and are a quite natural workflow. # Next diff --git a/site/helm-integration.md b/site/helm-integration.md index b27fd5ab6..b45c3a56a 100644 --- a/site/helm-integration.md +++ b/site/helm-integration.md @@ -3,6 +3,20 @@ title: Using Flux with Helm menu_order: 90 --- +- [Using Flux with Helm](#using-flux-with-helm) + * [The `HelmRelease` custom resource](#the-helmrelease-custom-resource) + + [Using a chart from a Git repo instead of a Helm repo](#using-a-chart-from-a-git-repo-instead-of-a-helm-repo) + + [What the Helm Operator does](#what-the-helm-operator-does) + * [Supplying values to the chart](#supplying-values-to-the-chart) + + [`.spec.values`](#specvalues) + + [`.spec.valueFileSecrets`](#specvaluefilesecrets) + - [Example of `spec.valueFileSecrets`](#example-of-specvaluefilesecrets) + * [Authentication](#authentication) + + [Authentication for Helm repos](#authentication-for-helm-repos) + + [Authentication for Git repos](#authentication-for-git-repos) + * [Upgrading images in a HelmRelease using Flux](#upgrading-images-in-a-helmrelease-using-flux) + + [Using annotations to control updates to HelmRelease resources](#using-annotations-to-control-updates-to-helmrelease-resources) + # Using Flux with Helm You can release charts to your cluster via "GitOps", by combining Flux @@ -65,9 +79,9 @@ with a `chart:` section like this: ```yaml spec: chart: - git: git@github.com:weaveworks/flux + git: git@github.com:weaveworks/flux-get-started ref: master - path: charts/flux + path: charts/ghost ``` In this case, the git repo will be cloned, and the chart will be @@ -286,7 +300,7 @@ values: ### Using annotations to control updates to HelmRelease resources -You can use the [same annotations](./using.md#using-annotations) in +You can use the [same annotations](./fluxctl.md#using-annotations) in the `HelmRelease` as you would for a Deployment or other workload, to control updates and automation. For the purpose of specifying filters, the container name is either `chart-image` (if at the top diff --git a/site/helm-operator.md b/site/helm-operator.md index 7f0148fd2..515e752eb 100644 --- a/site/helm-operator.md +++ b/site/helm-operator.md @@ -1,42 +1,32 @@ # Flux Helm Operator The Helm operator deals with Helm Chart releases. The operator watches for -changes of Custom Resources of kind FluxHelmRelease. It receives Kubernetes +changes of Custom Resources of kind HelmRelease. It receives Kubernetes Events and acts accordingly, installing, upgrading or deleting a Chart release. ## Setup and configuration helm-operator requires setup and offers customization though a multitude of flags. -(TODO: change the flags to reflect reality) - -|flag | default | purpose | -|------------------------|-------------------------------|---------| -|--kubernetes-kubectl | | Optional, explicit path to kubectl tool.| -|--kubeconfig | | Path to a kubeconfig. Only required if out-of-cluster.| -|--master | | The address of the Kubernetes API server. Overrides any value in kubeconfig. Only required if out-of-cluster.| -| | | **Tiller options**| -|--tillerIP | | Tiller IP address. Only required if out-of-cluster.| -|--tillerPort | | Tiller port.| -|--tillerNamespace | | Tiller namespace. If not provided, the default is kube-system.| | + +|flag | default | purpose | +|------------------------------|-------------------------------|---------| +|--kubeconfig | | Path to a kubeconfig. Only required if out-of-cluster. | +|--master | | The address of the Kubernetes API server. Overrides any value in kubeconfig. Only required if out-of-cluster. | +| | | **Tiller options** | +|--tiller-ip | | Tiller IP address. Only required if out-of-cluster. | +|--tiller-port | | Tiller port. | +|--tiller-namespace | | Tiller namespace. If not provided, the default is kube-system. | |--tiller-tls-enable |`false` | Enable TLS communication with Tiller. If provided, requires TLSKey and TLSCert to be provided as well. | |--tiller-tls-verify |`false` | Verify TLS certificate from Tiller. Will enable TLS communication when provided. | -|--tiller-tls-tls-key-path |`/etc/fluxd/helm/tls.key` | Path to private key file used to communicate with the Tiller server. | -|--tiller-tls-tls-cert-path |`/etc/fluxd/helm/tls.crt` | Path to certificate file used to communicate with the Tiller server. | -|--tiller-tls-tls-ca-cert-path | | Path to CA certificate file used to validate the Tiller server. Required if tiller-tls-verify is enabled. | +|--tiller-tls-key-path |`/etc/fluxd/helm/tls.key` | Path to private key file used to communicate with the Tiller server. | +|--tiller-tls-cert-path |`/etc/fluxd/helm/tls.crt` | Path to certificate file used to communicate with the Tiller server. | +|--tiller-tls-ca-cert-path | | Path to CA certificate file used to validate the Tiller server. Required if tiller-tls-verify is enabled. | |--tiller-tls-hostname | | The server name used to verify the hostname on the returned certificates from the Tiller server. | -| | | **Git repo & key etc.**| -|--git-url | | URL of git repo with Helm Charts; e.g., `ssh://git@github.com/weaveworks/flux-example`| -|--git-branch | `master` | Branch of git repo to use for Kubernetes manifests| -|--git-charts-path | `charts` | Path within git repo to locate Kubernetes Charts (relative path)| | | | **repo chart changes** (none of these need overriding, usually) | -|--git-timeout | `20 seconds` | duration after which git operations time out | -|--git-poll-interval | `5 minutes` | period at which to poll git repo for new commits| -|--chartsSyncInterval | 3*time.Minute | Interval at which to check for changed charts.| -|--chartsSyncTimeout | 1*time.Minute | Timeout when checking for changed charts.| -| | | **k8s-secret backed ssh keyring configuration**| -|--k8s-secret-volume-mount-path | `/etc/fluxd/ssh` | Mount location of the k8s secret storing the private SSH key| -|--k8s-secret-data-key | `identity` | Data key holding the private SSH key within the k8s secret| -|--queueWorkerCount | 2 | Number of workers to process queue with Chart release jobs.| +|--charts-sync-interval | `3m` | Interval at which to check for changed charts. | +|--git-timeout | `20s` | Duration after which git operations time out. | +|--log-release-diffs | `false` | Log the diff when a chart release diverges. **Potentially insecure.** | +|--update-chart-deps | `true` | Update chart dependencies before installing or upgrading a release. | ## Installing Weave Flux helm-operator and Helm with TLS enabled diff --git a/site/helm-upgrading-to-beta.md b/site/helm-upgrading-to-beta.md index 64120d722..36d81f991 100644 --- a/site/helm-upgrading-to-beta.md +++ b/site/helm-upgrading-to-beta.md @@ -37,10 +37,18 @@ since the new operator will ignore them. ### Using the flux chart -The chart (from v0.5.0-beta, or from this git repo) provides the +The chart (from v0.5.0, or from this git repo) provides the correct arguments to the operator; to upgrade, do - helm upgrade flux --reuse-values weaveworks/flux --version 0.5.0-beta +```sh +helm repo update + +helm upgrade flux --reuse-values \ +--set image.tag=1.8.1 \ +--set helmOperator.tag=0.5.1 \ +--namespace=flux \ +weaveworks/flux --version 0.5.1 +``` The chart will leave the old custom resource definition and custom resources in place. You will need to replace the individual resources, diff --git a/site/introduction.md b/site/introduction.md index 2ed6f248e..063dd5d1b 100644 --- a/site/introduction.md +++ b/site/introduction.md @@ -52,14 +52,14 @@ version. ## Integrations with other devops tools -One final high level feature is that Flux increases visibility of -your application. It provides an audit history for -your deployments and Slack integration for "ChatOps" style -development. - -Clear visibility of the state of a cluster is key for maintaining -operational systems. Developers can be confident in their changes by -observing a predictable series of deployment events. +One final high-level feature is that Flux increases visibility of your +application. Clear visibility of the state of a cluster is key for +maintaining operational systems. Developers can be confident in their +changes by observing a predictable series of deployment events. + +Flux can send notifications to a service (e.g., [Weave +Cloud](https://cloud.weave.works/)) to provide integrations with Slack +and other such media. ## Next diff --git a/site/standalone-setup.md b/site/standalone-setup.md index 422163a4c..303a0b57f 100644 --- a/site/standalone-setup.md +++ b/site/standalone-setup.md @@ -3,6 +3,17 @@ title: Customising the deployment menu_order: 20 --- +- [Customising the deployment](#customising-the-deployment) +- [Customising the daemon configuration](#customising-the-daemon-configuration) + * [Connect Flux to a repository](#connect-flux-to-a-repository) + * [Memcache](#memcache) + * [Flux deployment](#flux-deployment) + * [Add an SSH deploy key to the repository](#add-an-ssh-deploy-key-to-the-repository) + + [1. Allow Flux to generate a key for you.](#1-allow-flux-to-generate-a-key-for-you) + + [2. Specify a key to use](#2-specify-a-key-to-use) + + [Note for Kubernetes >=1.6 with role-based access control (RBAC)](#note-for-kubernetes-16-with-role-based-access-control-rbac) + * [Using a private git host](#using-a-private-git-host) + # Customising the deployment The deployment installs Flux and its dependencies. First, change to @@ -51,7 +62,7 @@ You have two options: ### 1. Allow Flux to generate a key for you. If you don't specify a key to use, Flux will create one for you. Obtain -the public key through `fluxctl`: +the public key through [fluxctl](./fluxctl.md): ```sh fluxctl identity