Using helmfiles, but unable to pass Environment down to helmfile.yaml

[nikolajbrinch]

When using

{{ readFile "environments.yaml }}}
--- 
helmfiles:
- ./*/helmfile.yaml

Environment.Values are not passed to the globbed helmfiles. I would expect that to happen/work, in order to keep helmfiles DRY.

[Tarick]

The same issue, using helmfile version v0.40.1

{{ readFile "common.yaml" }}
---
{{ readFile "environments.yaml" }}

With debug:

Processing helmfile.yaml
could not deduce environment: block, configuring only .Environment.Name. error: failed to read helmfile.yaml: environment "dev" is not defined
error in first-pass rendering: result of "helmfile.yaml":

Once I put environments block in helmfile.yaml, readFile of environments.yaml succedes as well and I have 2 environments blocks in the output.

[mumoshu]

@nikolajbrinch Hey! Your case isn't supposed to work as of today. That's because each helmfile is meant to be self-contained. Injecting environments like prod/staging/testing from the "parent" helmfile into "child" helmfiles is considered harmful because it works like an implicitly dependency from children to the parent.

I'll shortly reconsider though, about how we could support "inline environment values" and "injecting environment values from the command-line". Once we support either or both of them, I think it will justify passing environment values from the parent to the child.

[mumoshu]

@Tarick Hey! Just to clarify, are you talking about the fact that helmfile doesn't propagate the environment and the environment values from the parent to children? Or you're maybe talking about a possible bug(?) that multiple yaml docs declared within a single helmfile.yaml doesn't get merged?

[nikolajbrinch]

Yes. In my environments.yaml I declare environments and load defaults for environments. This is host-names etc. Data/variables are global for the complete cluster environment. It would be very tedious an error prone to have those replicated across many projects.
This also happens to be certificates etc, that needs updating once in a while. It would be nice to have these set globally, and change them, and run the admin-helmfile orchestra to upgrade all certificates, hostnames etc.

[Tarick]

@Tarick Hey! Just to clarify, are you talking about the fact that helmfile doesn't propagate the environment and the environment values from the parent to children? Or you're maybe talking about a possible bug(?) that multiple yaml docs declared within a single helmfile.yaml doesn't get merged?

The latter case. Basically, I tried to do what https://github.com/roboll/helmfile/blob/master/docs/writing-helmfile.md#layering recommends and this fails.

[nikolajbrinch]

@mumoshu I think one way could be conventions. i.e. environment.yaml declares environment. Just like values.yaml in Helm Charts declares values.
I really like this project and uses it for our infrastructure, but I have a couple of questions:

1. Why the xyz.yaml.gotmpl extension, when Helm uses _xyz.tpl? It seems that it would be easier to adopt if conventions where shared between helm and helmfile.
2. Propagation of values could be handled with global as in Helm.

Environment is handled specially, and I think this should be the case. But a convention would be nice, where I do not need to readFile a helmfile, but a file with a special name, prefix/postfix or something else is automatically merged into .Environment.
One example is Spring Boot that has application.yml as default configuration, and application-dev.yml for the dev environment, configurations are then merged.
This could be a convention in helmfile, where environment.yaml , and environment-dev.yaml are the analogs, and are merged, and available thoughout the chain of helmfiles executed.

[nikolajbrinch]

@Tarick I couldn't get Layering to work either.

[Tarick]

One example is Spring Boot that has application.yml as default configuration, and application-dev.yml
for the dev environment, configurations are then merged.

I use somethink like that in example below. The values files that are supplied to Helm are then merged by Helm.

- name: {{ .Environment.Name }}-api
    labels:
      app_name: "api"
      tier: "apps"
    chart: "somerepo/springboot-app"
    version: 0.1.0
    namespace: {{ .Environment.Values.namespace }}
    values:
      - "environments/default/api.yaml.gotmpl"   # Initial default settings
      - "environments/{{ .Environment.Name }}/api.yaml.gotmpl"   # Overrides

[sruon]

After debugging for a couple hours, I don't see how Layering can work the way it's described in the documentation.

https://github.com/roboll/helmfile/blob/master/tmpl/tmpl.go#L24 parses the file and does not know about the environment yet. If you use a value loaded from an environment, it will just choke on the unknown value and stop loading there i.e.

{{ readFile "environments.yaml"}}
---
releases:
  - name: my-app-{{ .Environment.Values.releaseName }}

I toyed around with splitting the input on --- and parsing each part as a separate fragment / reloading the environment on each loop and that appears to be almost entirely working.
The only issue I'm seeing is the templated values cannot be used as part of a release name, I think the first pass renderer needs to be modified similarily.

index 399109f..00fc2de 100644
--- a/main.go
+++ b/main.go
@@ -750,20 +750,26 @@ func (r *twoPassRenderer) renderEnvironment(content []byte) environment.Environm
 
 func (r *twoPassRenderer) renderTemplate(content []byte) (*bytes.Buffer, error) {
 	// try a first pass render. This will always succeed, but can produce a limited env
-	firstPassEnv := r.renderEnvironment(content)
+	splitContent := bytes.Split(content, []byte("---"))
+	var yamlBuf bytes.Buffer
 
-	secondPassRenderer := tmpl.NewFileRenderer(r.reader, filepath.Dir(r.filename), firstPassEnv, r.namespace)
-	yamlBuf, err := secondPassRenderer.RenderTemplateContentToBuffer(content)
-	if err != nil {
+	firstPassEnv := r.renderEnvironment(content)
+	for _, subContent := range splitContent {
+		secondPassRenderer := tmpl.NewFileRenderer(r.reader, filepath.Dir(r.filename), firstPassEnv, r.namespace)
+		subBuf, err := secondPassRenderer.RenderTemplateContentToBuffer(subContent)
+		if err != nil {
+			if r.logger != nil {
+				r.logger.Debugf("second-pass rendering failed, input of \"%s\":\n%s", r.filename, prependLineNumbers(string(subContent)))
+			}
+			return nil, err
+		}
 		if r.logger != nil {
-			r.logger.Debugf("second-pass rendering failed, input of \"%s\":\n%s", r.filename, prependLineNumbers(string(content)))
+			r.logger.Debugf("second-pass rendering result of \"%s\":\n%s", r.filename, prependLineNumbers(subBuf.String()))
 		}
-		return nil, err
-	}
-	if r.logger != nil {
-		r.logger.Debugf("second-pass rendering result of \"%s\":\n%s", r.filename, prependLineNumbers(yamlBuf.String()))
-	}
-	return yamlBuf, nil
+			yamlBuf.WriteString(subBuf.String())
+			firstPassEnv = r.renderEnvironment(yamlBuf.Bytes())
+		}
+	return &yamlBuf, nil
 }
 
 func (a *app) VisitDesiredStates(fileOrDir string, converge func(*state.HelmState, helmexec.Interface) (bool, []error)) error {```

[sruon]

To amend my previous comment, the loadEnv function always passes an empty environment to the renderer

helmfile/state/create.go

Line 123 in 9808849

r := valuesfile.NewRenderer(readFile, filepath.Dir(envvalFullPath), environment.EmptyEnvironment)

Changing environment.EmptyEnvironment to environment.Environment{Name: name, Values: envVals} gives me the behavior I'm looking for.

Wonder if we could add that behavior as a flag?

[mumoshu]

Hey! Sorry for the delayed response. I'll comment one by one.

@sruon

{{ readFile "environments.yaml"}}
---
releases:
  - name: my-app-{{ .Environment.Values.releaseName }}

This doesn't work because at the time of the first-render(also see #308 for more context) the whole yaml file including before/after the separater --- are rendered altogether. At that time, {{.Environment.Values.releaseName }} isn't populated yet. The second-render works. So you need a special treatment to pass the first-render anyway. That is, use getOrNil like name: my-app-{{ .Environment.Values | getOrNil "releaseName" }}.

[mumoshu]

I use somethink like that in example below. The values files that are supplied to Helm are then merged by Helm.

@Tarick Your example looks awesome! I think you're correctly using the helmfile features as I have expected :)

[mumoshu]

I think one way could be conventions. i.e. environment.yaml declares environment. Just like values.yaml in Helm Charts declares values.

@nikolajbrinch Sounds nice! I do want more conventions so that every helmfile project looks familiar, which is great in regard to maintainability.

Only thing I'm missing is the good convention. Assuming there are a bunch of yaml files in the top-level of every project, giving it a too generic name would express less connection to helmfile. But helmfile.environments.yaml does look too verbose either..

Why the xyz.yaml.gotmpl extension, when Helm uses _xyz.tpl? It seems that it would be easier to adopt if conventions where shared between helm and helmfile.

Just because I thought it is clearer. When I named it, .tpl was thought to be used by not only helm but other template engines as well.

But I'm fine with adding .yaml.tpl as an alias if you like it. But note that helm's .tpl is used by only template helpers. Templated manifest files under chart's template/ directory themselves are suffixed with .yaml even though they are templates.

[mumoshu]

I have a vague idea of allowing helmfile to layer multiple helmfile.yaml files before parsing and calling, like kustomize and kasane do, if it helps:

helmfiles:
- layers:
   - global/environments.yaml
   - helmfiles/myapp-that-depends-on-the-global-env.yaml
- hemlfiles/myanotherapp-that-has-its-own-env.yaml

[joshwand]

I am also having difficulty with this-- I have a situation where I have many production environments (one per customer), and I want to put them in separate helmfiles to keep the file readable, but then I can't use the environment vars in the base file brought in via readfile:

helmfile-dev.yaml

environments: 
  ci-master:
    values: 
    - values-ci-master.yaml 
  dev1:
    values: 
    - values-dev1.yaml

{{ readFile "../Helmfile-app-base.yaml" }}

Helmfile-prod-customer1:

environments: 
  prod-customer1:
    values: 
      - prod-customer1/prod-customer1.yaml
  prod-customer1-sandbox:
    values: 
      - prod-customer1/prod-customer1-sandbox.yaml

{{ readFile ../Helmfile-app-base.yaml }}

but the values from the evnironment aren't rendered in Helmfile-app-base.yaml, which looks like this:

releases:
- name: app-{{ .Environment.Name }}
  namespace: {{ .Environment.Values.namespace }}
  chart: ../charts/myapp
  values: 
    - {{ .Environment.Values }} 

because the context is not passed to readFile ?

[rajiteh]

I was also thrown off by this bug, the readme specifically says this should be possible.

Let's assume that your helmfile.yaml looks like:

{ readFile "commons.yaml" }}
---
{{ readFile "environments.yaml" }}
---
releases:
- name: myapp
  chart: mychart

Whereas commons.yaml contained a monitoring agent:

releases:
- name: metricbaet
  chart: stable/metricbeat

And environments.yaml contained well-known environments:

environments:
  development:
  production:

but when I run helmfile -e test lint I see the following error:

could not deduce `environment:` block, configuring only .Environment.Name. error: failed to read helmfile.yaml: environment "test" is not defined

perhaps the readme should be updated to remove the invalid example.

[royjs]

I also stumbled an issue related to this.

I have a working helmfile that looks like this:

environments:
  dev:
  prod:

templates:
  default: &default
    missingFileHandler: Debug
    values:
      - values/common.yaml.gotmpl 
      {{- range $overrideFolder := .Environment.Values.overrideFolders }}
      - values/{{ $overrideFolder }}/{{`{{  .Release.Name }}`}}.yaml
      {{- end }}

releases:
  ....

I tried to move the environments out of this file and use '{{ readFile "environments.yaml" }}' to include it. When I do that, it stops working because the environment values are not available yet. I also tried delaying the parsing of the templates to the 2nd pass but the issue is that the yaml won't be valid (because of the range loop part) and the parsing fails, it needs to be done on the 1st pass.

I think it would make more sense to handle the environments separately so that they are always available in the state for the first pass. I'm not sure about the exact convention we should use but having a (optionally?) separate file for it makes sense.

[sboschman]

Would be great to be able to combine layering with the directory layout from glob patterns (multiple independent files), helmfile environments and release templates. ❤️

Our almost 300 lines of templated helmfile.yaml, with 24 templated value files and 26 environment specific value files (of which some are templated again) are not very DRY 🤣
(counting 2817 lines of yaml in total 🙄)

Non-working idea:

• helmfile.yaml
• environments.yaml
• helmdefaults.yaml
• repositories.yaml
• templates.yaml
• apps
  • app1
    • helmfile.yaml
  • app2
    • helmfile.yaml

Example environments.yaml:

environments:
  default:
    values:
    - dev.yaml
  previder:
    values:
    - prod.yaml

The helm defaults are specified for a particular environment. Without layering this works. Example helmdefaults.yaml:

helmDefaults:
  kubeContext: "{{ .Environment.Values | getOrNil "kubeContext" }}"
  tillerNamespace: "{{ .Environment.Values | getOrNil "tillerNamespace" }}"

Without layering the release templates are rendered. With layering I get stuck on unknown anchor 'default' referenced. Example templates.yaml:

templates:
  default: &default
    chart: stable/{{`{{ .Release.Name }}`}}
    namespace: {{ .Environment.Values | getOrNil 'namespace' }}

    # this might be oke instead of defining the values on the environment and fix the namespace difference between environments instead
    values:
    - "./config/kubernetes-dashboard/{{ .Environment.Name }}.yaml"

Example apps/app1/helmfile.yaml:

{{ readFile "../../environments.yaml" }}
---
{{ readFile "../../repositories.yaml" }}
---
{{ readFile "../../helmdefaults.yaml" }}
---
{{ readFile "../../templates.yaml" }}
---
    
releases:
- name: consul
  version: ">= 3.5.0"
  <<: *default

[ufou]

I was about to log a new issue with this precise problem, is there any traction on this feature? It seems like an obvious thing to support and it would really help with DRY

[embik]

I'm not sure it's necessary, but here's a use case that misses this feature as far as I can tell. We are managing multiple clusters via helmfile and there are certain "global" values that I would like to pass only once to helmfile. For example, a cluster ID / name that is used for all ingress hostnames.

I tried serveral things. This looks like it does not work because of this issue:

values.yaml:

clusterID: 010101

helmfile.yaml:

environments:
  - default:
    values:
      - values.yaml

helmfiles:
  - helmfiles/prometheus.yaml

helmfiles/prometheus.yaml:

releases:
  - name: prometheus
    namespace: prometheus
    chart: stable/prometheus
    values:
      - values.yaml.gotmpl

helmfiles/values.yaml.gotmpl:

server:
  ingress:
    hosts:
      - prometheus.{{ .Environment.Values.clusterID }}.clusters.lcoal

I also tried including the helmfiles via readFile, but unfortunately their configuration will not get templated and therefore this takes away a considerable amount of features (the hook example from the README won't work with that, for example).

Passing down the configured environment via helmfiles would really be awesome!

[mumoshu]

I started to think we need #96 proposed by @gtaylor for this.

With that we could reuse whatever helmfile.yaml fragment(s) like:

includes:
- common-environments.yaml

And you'll explicitly repeat it in every sub-helmfile. Btw, I think the explicitness is a must-have, because implicitness means that we rely on "globals" #398, and I believe globals should be implemented by environment variables(not values) and the {{ env ... }} template function.

Once this comes reality, I'd deprecate the current layering system with multi-doc YAML as unnecessary.

WDYT?

[mumoshu]

includes seems a bit ambiguous whether it is able to override contents of a helmfile.yaml or becomes the "base" of it. The latter is what I think we need.

So should it better be bases?

bases:
- common-environments.yaml

[royjs]

Yeah for me includes would mean that the content of these files would be included in the helmfile.yaml, that implies that it might overwrite it. We need to use a word that it makes it clear that it's the other way around. I think that bases work

[mumoshu]

@embik Hey, thanks for chiming in!

For global variables, I'd suggest using environment variables with {{ env ... }} template expressions as suggested in #398.

Otherwise you can define environments: section in helmfiles/prometheus.yaml, rather than helmfile.yaml.

[mumoshu]

@royjs Thanks for your confirmation ☺️

[embik]

@mumoshu I'm aware, but defining environments: sections in around 20 files (and counting) gets quite repetitive over time, so I was looking for a way to pass down values. I'll take a look at increased use of environment variables for that, thank you.

Regarding the bases keyword, just another data point: Looks fine to me, doesn't solve my problem the way I'd prefer it but I definitely think it would be useful.

[mumoshu]

@embik Thx!

Would you mind clarifying a bit more about your use-case?

I was thinking bases would also help your use-cases by allowing you to reduce the amount of repetitions, from a few lines of environments: in around 20 files, to one or two lines of bases: ["your-common-environment"].

[mumoshu]

Note that bases implementation is being simplified as described in #388 (comment)

[embik]

@mumoshu This sounds like a great set of changes and improvements. Thank you for investing such much time and energy and keeping everyone updated here!

[mumoshu]

#587 is ready to be reviewed 😄

[mumoshu]

The enahcements will be included since helmfile v0.60.0

[galindro]

@mumoshu The approach that you mentioned here isn't working for me (my helmfile version is v0.98.1). I'm getting this error:

$ helmfile --log-level debug -e test diff
processing file "helmfile.yaml" in directory "."
first-pass rendering starting for "helmfile.yaml.part.0": inherited=&{test map[] map[]}, overrode=<nil>
first-pass uses: &{test map[] map[]}
first-pass rendering output of "helmfile.yaml.part.0":
 0: bases:
 1:   - ./common/environments.yaml
 2:   - ./common/repositories.yaml
 3:   - ./common/defaults.yaml
 4: 
 5: 
 6: 
 7: releases:
 8:   - name: gloo
 9:     <<: *default
10:     chart: gloo/{{ .Release.Name }}
11:     namespace: gloo-system
12:     version: <no value>
13: 

replaced <no value>s to workaround https://github.com/golang/go/issues/24963 to address https://github.com/roboll/helmfile/issues/553:
  strings.Join({
        ... // 10 identical lines
        "    chart: gloo/{{ .Release.Name }}",
        "    namespace: gloo-system",
-       "    version: <no value>",
+       "    version: ",
        "",
  }, "\n")

could not deduce `environment:` block, configuring only .Environment.Name. error: failed to read helmfile.yaml.part.0: reading document at index 1: yaml: unknown anchor 'default' referenced
error in first-pass rendering: result of "helmfile.yaml.part.0":
 0: bases:
 1:   - ./common/environments.yaml
 2:   - ./common/repositories.yaml
 3:   - ./common/defaults.yaml
 4: 
 5: 
 6: 
 7: releases:
 8:   - name: gloo
 9:     <<: *default
10:     chart: gloo/{{ .Release.Name }}
11:     namespace: gloo-system
12:     version: <no value>
13: 

first-pass produced: &{test map[] map[]}
first-pass rendering result of "helmfile.yaml.part.0": {test map[] map[]}
second-pass rendering failed, input of "helmfile.yaml.part.0":
 0: bases:
 1:   - ./common/environments.yaml
 2:   - ./common/repositories.yaml
 3:   - ./common/defaults.yaml
 4: 
 5: {{ readFile "./common/templates.yaml" }}
 6: 
 7: releases:
 8:   - name: gloo
 9:     <<: *default
10:     chart: gloo/{{ `{{ .Release.Name }}` }}
11:     namespace: gloo-system
12:     version: {{ .Values.GlooVersion }}
13: 

err: error during helmfile.yaml.part.0 parsing: template: stringTemplate:13:23: executing "stringTemplate" at <.Values.GlooVersion>: map has no entry for key "GlooVersion"
in ./helmfile.yaml: error during helmfile.yaml.part.0 parsing: template: stringTemplate:13:23: executing "stringTemplate" at <.Values.GlooVersion>: map has no entry for key "GlooVersion"

Directory structure

.
├── common
│   ├── defaults.yaml
│   ├── environments.yaml
│   ├── repositories.yaml
│   └── templates.yaml
├── environments
│   ├── prod.yaml
│   └── test.yaml
├── gloo
│   ├── namespace.yaml
│   └── values.test.yaml
└── helmfile.yaml

helmfile.yaml

bases:
  - ./common/environments.yaml
  - ./common/repositories.yaml
  - ./common/defaults.yaml

{{ readFile "./common/templates.yaml" }}

releases:
  - name: gloo
    <<: *default
    chart: gloo/{{ `{{ .Release.Name }}` }}
    namespace: gloo-system
    version: {{ .Values.GlooVersion }}

To temporally fix it, I just changed my helmfile.yaml to this:

helmfile.yaml

bases:
  - ./common/environments.yaml
  - ./common/repositories.yaml
  - ./common/defaults.yaml

---

templates:
  default: &default
    chart: stable/{{ `{{ .Release.Name }}` }}
    missingFileHandler: Warn
    values:
      - ./{{ `{{ .Release.Name }}` }}/values.{{ `{{ .Environment.Name }}` }}.yaml.gotmpl
    hooks:
      - events: ["prepare"]
        showlogs: true
        command: "kubectl"
        args: ["apply", "-f", "{{ `{{ .Release.Name }}` }}/namespace.yaml"]

releases:
  - name: gloo
    <<: *default
    chart: gloo/{{ `{{ .Release.Name }}` }}
    namespace: gloo-system
    version: {{ .Values.GlooVersion }}

Other files

./common/defaults.yaml

helmDefaults:
  tillerless: true
  kubeContext: eks-services
  wait: true
  timeout: 600
  recreatePods: false
  force: true

./common/environments.yaml

environments:
  prod:
    values:
      - ./environments/prod.yaml
  test:
    values:
      - ./environments/test.yaml

./common/repositories.yaml

repositories:
  - name: stable
    url: https://kubernetes-charts.storage.googleapis.com

./common/templates.yaml

templates:
  default: &default
    chart: stable/{{ `{{ .Release.Name }}` }}
    missingFileHandler: Warn
    values:
      - ./{{ `{{ .Release.Name }}` }}/values.{{ `{{ .Environment.Name }}` }}.yaml.gotmpl
    hooks:
      - events: ["prepare"]
        showlogs: true
        command: "kubectl"
        args: ["apply", "-f", "{{ `{{ .Release.Name }}` }}/namespace.yaml"]

./environments/prod.yaml

GlooVersion: 1.2.10

./environments/test.yaml

GlooVersion: 1.2.10

./gloo/namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: gloo-system

./gloo/values.test.yaml

settings:
  linkerd: false
gatewayProxies:
  gatewayProxy:
    service:
      type: LoadBalancer
      extraAnnotations:
        service.beta.kubernetes.io/aws-load-balancer-internal: "true"

[mumoshu]

@galindro Hey! Thanks for the info.

As you've tested and written, I believe we should always add --- between bases and releases sections of your helmfile.yaml practically. Otherwise we may or may not get strange errors that sometimes makes your env values disappear and sometimes not.

[galindro]

Yeah @mumoshu , but the problem is that I can't split out the templates section from helmfile.yaml into another file, even using the section separator characters ---. Which means that this doesn't works as well. Please, try yourself.

bases:
  - ./common/environments.yaml
  - ./common/repositories.yaml
  - ./common/defaults.yaml
---
{{ readFile "./common/templates.yaml" }}
---
releases:
  - name: gloo
    <<: *default
    chart: gloo/{{ `{{ .Release.Name }}` }}
    namespace: gloo-system
    version: {{ .Values.GlooVersion }}

[mumoshu]

Sorry to reply without testing it as you suggested, but are you saying that it is failing on *default?

If so, yeah it doesn't work. You shouldn't split readFile and releases. Otherwise it fails due to you can't refer to YAML anchors defined in another YAML document.

Your config should look like:

bases:
  - ./common/environments.yaml
  - ./common/repositories.yaml
  - ./common/defaults.yaml
---
{{ readFile "./common/templates.yaml" }}

releases:
  - name: gloo
    <<: *default
    chart: gloo/{{ `{{ .Release.Name }}` }}
    namespace: gloo-system
    version: {{ .Values.GlooVersion }}

[galindro]

@mumoshu I already tested using the mentioned method but it doesn't works also. Take a look:

$ helmfile --log-level debug -e test diff
processing file "helmfile.yaml" in directory "."
first-pass rendering starting for "helmfile.yaml.part.0": inherited=&{test map[] map[]}, overrode=<nil>
first-pass uses: &{test map[] map[]}
first-pass rendering output of "helmfile.yaml.part.0":
 0: bases:
 1:   - ./common/environments.yaml
 2:   - ./common/repositories.yaml
 3:   - ./common/defaults.yaml

error in first-pass rendering: result of "helmfile.yaml.part.0":
 0: bases:
 1:   - ./common/environments.yaml
 2:   - ./common/repositories.yaml
 3:   - ./common/defaults.yaml

first-pass produced: &{test map[] map[]}
first-pass rendering result of "helmfile.yaml.part.0": {test map[] map[]}
second-pass rendering result of "helmfile.yaml.part.0":
 0: bases:
 1:   - ./common/environments.yaml
 2:   - ./common/repositories.yaml
 3:   - ./common/defaults.yaml

first-pass rendering starting for "common/environments.yaml.part.0": inherited=&{test map[] map[]}, overrode=<nil>
first-pass uses: &{test map[] map[]}
first-pass rendering output of "common/environments.yaml.part.0":
 0: environments:
 1:   prod:
 2:     values:
 3:       - ./environments/prod.yaml
 4:   test:
 5:     values:
 6:       - ./environments/test.yaml
 7: 
 8: 

envvals_loader: loaded ./environments/test.yaml:map[GlooVersion:1.2.10]
first-pass produced: &{test map[GlooVersion:1.2.10] map[]}
first-pass rendering result of "common/environments.yaml.part.0": {test map[GlooVersion:1.2.10] map[]}
vals:
map[GlooVersion:1.2.10]
defaultVals:[]
second-pass rendering result of "common/environments.yaml.part.0":
 0: environments:
 1:   prod:
 2:     values:
 3:       - ./environments/prod.yaml
 4:   test:
 5:     values:
 6:       - ./environments/test.yaml
 7: 
 8: 

envvals_loader: loaded ./environments/test.yaml:map[GlooVersion:1.2.10]
merged environment: &{test map[GlooVersion:1.2.10] map[]}
first-pass rendering starting for "common/repositories.yaml.part.0": inherited=&{test map[] map[]}, overrode=<nil>
first-pass uses: &{test map[] map[]}
first-pass rendering output of "common/repositories.yaml.part.0":
 0: repositories:
 1:   - name: stable
 2:     url: https://kubernetes-charts.storage.googleapis.com
 3: 
 4: 

first-pass produced: &{test map[] map[]}
first-pass rendering result of "common/repositories.yaml.part.0": {test map[] map[]}
vals:
map[]
defaultVals:[]
second-pass rendering result of "common/repositories.yaml.part.0":
 0: repositories:
 1:   - name: stable
 2:     url: https://kubernetes-charts.storage.googleapis.com
 3: 
 4: 

merged environment: &{test map[] map[]}
first-pass rendering starting for "common/defaults.yaml.part.0": inherited=&{test map[] map[]}, overrode=<nil>
first-pass uses: &{test map[] map[]}
first-pass rendering output of "common/defaults.yaml.part.0":
 0: helmDefaults:
 1:   tillerless: true
 2:   kubeContext: eks-services
 3:   wait: true
 4:   timeout: 600
 5:   recreatePods: false
 6:   force: true
 7: 
 8: 

first-pass produced: &{test map[] map[]}
first-pass rendering result of "common/defaults.yaml.part.0": {test map[] map[]}
vals:
map[]
defaultVals:[]
second-pass rendering result of "common/defaults.yaml.part.0":
 0: helmDefaults:
 1:   tillerless: true
 2:   kubeContext: eks-services
 3:   wait: true
 4:   timeout: 600
 5:   recreatePods: false
 6:   force: true
 7: 
 8: 

merged environment: &{test map[] map[]}
envvals_loader: loaded ./environments/test.yaml:map[GlooVersion:1.2.10]
merged environment: &{test map[GlooVersion:1.2.10] map[]}
first-pass rendering starting for "helmfile.yaml.part.1": inherited=&{test map[GlooVersion:1.2.10] map[]}, overrode=<nil>
first-pass uses: &{test map[GlooVersion:1.2.10] map[]}
first-pass rendering output of "helmfile.yaml.part.1":
 0: 
 1: 
 2: releases:
 3:   - name: gloo
 4:     <<: *default
 5:     chart: gloo/{{ .Release.Name }}
 6:     namespace: gloo-system
 7:     version: <no value>
 8: 
 9: 

replaced <no value>s to workaround https://github.com/golang/go/issues/24963 to address https://github.com/roboll/helmfile/issues/553:
  strings.Join({
        ... // 5 identical lines
        "    chart: gloo/{{ .Release.Name }}",
        "    namespace: gloo-system",
-       "    version: <no value>",
+       "    version: ",
        "",
        "",
  }, "\n")

could not deduce `environment:` block, configuring only .Environment.Name. error: failed to read helmfile.yaml.part.1: reading document at index 1: yaml: unknown anchor 'default' referenced
error in first-pass rendering: result of "helmfile.yaml.part.1":
 0: 
 1: 
 2: releases:
 3:   - name: gloo
 4:     <<: *default
 5:     chart: gloo/{{ .Release.Name }}
 6:     namespace: gloo-system
 7:     version: <no value>
 8: 
 9: 

first-pass produced: &{test map[GlooVersion:1.2.10] map[]}
first-pass rendering result of "helmfile.yaml.part.1": {test map[GlooVersion:1.2.10] map[]}
second-pass rendering failed, input of "helmfile.yaml.part.1":
 0: {{ readFile "./common/templates.yaml" }}
 1: 
 2: releases:
 3:   - name: gloo
 4:     <<: *default
 5:     chart: gloo/{{ `{{ .Release.Name }}` }}
 6:     namespace: gloo-system
 7:     version: {{ .Values.GlooVersion }}
 8: 
 9: 

err: error during helmfile.yaml.part.1 parsing: template: stringTemplate:8:23: executing "stringTemplate" at <.Values.GlooVersion>: map has no entry for key "GlooVersion"
in ./helmfile.yaml: error during helmfile.yaml.part.1 parsing: template: stringTemplate:8:23: executing "stringTemplate" at <.Values.GlooVersion>: map has no entry for key "GlooVersion"

You can download this archive and test by yourself:
https://gist.github.com/galindro/c9b9bafe18b30f0e54748f38121bf47d#file-helmfile-test-tar-gz

[mumoshu]

@galindro Thanks! Ah that makes sense. Then you have no way to use anchors like *default over readFile. This is a fundamental issue coming from how double-rendering works. See also #932.

In the meantime, could you try using readFile to carry the default?

Like:

releases:
  - name: gloo
    {{ readFile "./common/default.yaml" | nindent 4}}
    chart: gloo/{{ `{{ .Release.Name }}` }}
    namespace: gloo-system
    version: {{ .Values.GlooVersion }}

[galindro]

@mumoshu , unfortunately, it not worked as well. Take a look:

First I changed the templates.yaml to this:

chart: stable/{{ `{{ .Release.Name }}` }}
missingFileHandler: Warn
values:
  - ./{{ `{{ .Release.Name }}` }}/values.{{ `{{ .Environment.Name }}` }}.yaml.gotmpl
hooks:
  - events: ["prepare"]
    showlogs: true
    command: "kubectl"
    args: ["apply", "-f", "{{ `{{ .Release.Name }}` }}/namespace.yaml"]

Then, I changed helmfile.yaml with your proposed change:

bases:
  - ./common/environments.yaml
  - ./common/repositories.yaml
  - ./common/defaults.yaml

releases:
  - name: gloo
    {{ readFile "./common/templates.yaml" | nindent 4 }}
    chart: gloo/{{ `{{ .Release.Name }}` }}
    namespace: gloo-system
    version: {{ .Values.GlooVersion }}

In the end, I executed helmfile, and this was the result:

$ helmfile --log-level debug -e test diff
processing file "helmfile.yaml" in directory "."
first-pass rendering starting for "helmfile.yaml.part.0": inherited=&{test map[] map[]}, overrode=<nil>
first-pass uses: &{test map[] map[]}
first-pass rendering output of "helmfile.yaml.part.0":
 0: bases:
 1:   - ./common/environments.yaml
 2:   - ./common/repositories.yaml
 3:   - ./common/defaults.yaml
 4: 
 5: releases:
 6:   - name: gloo
 7:     
 8:     
 9:     chart: gloo/{{ .Release.Name }}
10:     namespace: gloo-system
11:     version: <no value>
12: 
13: 

replaced <no value>s to workaround https://github.com/golang/go/issues/24963 to address https://github.com/roboll/helmfile/issues/553:
  strings.Join({
        ... // 9 identical lines
        "    chart: gloo/{{ .Release.Name }}",
        "    namespace: gloo-system",
-       "    version: <no value>",
+       "    version: ",
        "",
        "",
  }, "\n")

error in first-pass rendering: result of "helmfile.yaml.part.0":
 0: bases:
 1:   - ./common/environments.yaml
 2:   - ./common/repositories.yaml
 3:   - ./common/defaults.yaml
 4: 
 5: releases:
 6:   - name: gloo
 7:     
 8:     
 9:     chart: gloo/{{ .Release.Name }}
10:     namespace: gloo-system
11:     version: <no value>
12: 
13: 

first-pass produced: &{test map[] map[]}
first-pass rendering result of "helmfile.yaml.part.0": {test map[] map[]}
second-pass rendering failed, input of "helmfile.yaml.part.0":
 0: bases:
 1:   - ./common/environments.yaml
 2:   - ./common/repositories.yaml
 3:   - ./common/defaults.yaml
 4: 
 5: releases:
 6:   - name: gloo
 7:     {{ readFile "./common/templates.yaml" | nindent 4 }}
 8:     chart: gloo/{{ `{{ .Release.Name }}` }}
 9:     namespace: gloo-system
10:     version: {{ .Values.GlooVersion }}
11: 
12: 

err: error during helmfile.yaml.part.0 parsing: template: stringTemplate:11:23: executing "stringTemplate" at <.Values.GlooVersion>: map has no entry for key "GlooVersion"
in ./helmfile.yaml: error during helmfile.yaml.part.0 parsing: template: stringTemplate:11:23: executing "stringTemplate" at <.Values.GlooVersion>: map has no entry for key "GlooVersion"

P.S.: the defaults.yaml carries helmDefaults content, not the defaults that should be used by releases.

[galindro]

@mumoshu should we continue our talk here or here?

[mumoshu]

@galindro I think the last missing piece is that --- between bases and releases, as suggested in #388 (comment)

[vavdoshka]

Does passing the state down to helmfile through the inline yaml object looks like a viable approach?

environments:
  stage:
    values:
    - env/common.yaml
    - env/stage.yaml
  prod:
    values:
    - env/common.yaml
    - env/prod.yaml

helmfiles:
- path: releases/scdf/helmfile.yaml
  values:
    - {{ toYaml .Values | nindent 8 }}

It seems it works, at least on POC surface. It allows having only one point of the environment load and hence keeps code DRYer .
I'm new to helmfile so would be very happy to get feedback to know if and why the technique I shared is a no-go. If that matters our use-case is just to use helmfile template and the rest we delegate to ArgoCD.

[mumoshu]

@vavdoshka It should work! I've also came up with that myself in #762 (comment)

Please feel free to add your voice to #762 if you find that helps your use case.

[mumoshu]

@vavdoshka BTW a general recommendation is that you should put --- between environments and the dynamic part of your helmfie.yaml to avoid nasty things from happening due to the chicken-and-egg problem rendering helmfile. (you need to render helmfile template to load environments, but you need to render the helmfile as yaml before parsing it and loading environments)

environments:
  stage:
    values:
    - env/common.yaml
    - env/stage.yaml
  prod:
    values:
    - env/common.yaml
    - env/prod.yaml

---

helmfiles:
- path: releases/scdf/helmfile.yaml
  values:
    - {{ toYaml .Values | nindent 8 }}

[vavdoshka]

@mumoshu thanks for such a quick feedback!

Cool! I think this approach is clear and explicit enough, it just probably needs to be documented somewhere. Thanks for highlighting the --- issue.

[mumoshu]

@vavdoshka Thanks for confirming! I agree very much. Just not sure where to start. Perhaps adding that to the bottom of https://github.com/roboll/helmfile/blob/master/docs/writing-helmfile.md makes sense?

[vavdoshka]

yes, @mumoshu sounds good to me
I've opened a draft PR on this #1808, please feel free to amend it.

[sshishov]

Hello guys, I have a question.

We are using like this:

bases:
  - flags.yaml (per environment flags)
---
bases:
  - environments.yaml.gotmpl  (per environment configuration, should also use `.Values` defined in flags, NOT WORKING HERE)
  - repositories.yaml
---
bases:
  - helmdefaults.yaml.gotmpl  (because it is using `.Values`, IT IS WORKING HERE)
---
helmfiles:
  - releases/*

But unfortunately when environments are rendered the .Values are empty. Why?

The question is how to make some Values per environment depends on some flag which is also defined per environment. As this flag then used in Values with {{- if ... }} statement to conditionally include some content

[MurzNN]

Is it possible to use same technique to access data from previous values files, described as several files in release, or from secrets?
Something like this:
helmfile:

releases:
  - name: my-chart
    chart: my-chart
    namespace: my-chart
    values:
      - ./config/my-chart/secrets.yaml
    values:
      - ./config/my-chart/values1.yaml.gotmpl
      - ./config/my-chart/values2.yaml.gotmpl

./config/my-chart/secrets.yaml.:

mySecret: superSecret

./config/my-chart/values1.yaml.gotmpl:

Variable1: value1

./config/my-chart/values2.yaml.gotmpl:

Variable2: {{ .Values | get "Variable1" }}
MySecretInValues: {{ .Values | get "mySecret" }}