Variable and environment variable order and use needs clarification

[tommyalatalo]

I've been trying to write some tasks today and found that the way that variables and environment variables have some gaps in the documentation regarding how they behave.

First off, in the example below the EXPORTED variable is an env var set with export EXPORTED=foo:

  var:
    desc: try out some vars
    env:
      MYVAR:
        sh: echo bar
    cmds:
      - echo {{.EXPORTED}}
      - echo $MYVAR
      - echo {{.MYVAR}}

This results in the output:

echo foo
foo
echo $MYVAR
bar
echo <no value>
task: Failed to run task "var": 1:15: > must be followed by a word

Why does it work to do echo {{.EXPORTED}} for an exported env var, but not echo {{.MYVAR}}?
There is no clear explanation in the docs as to when the $ notation vs the {{ }} should be used with regards to environment variables.

Second, it's not really explained in the documentation in which order and how variables are interpolated, I found and issue when trying to do something like the below:

var:
    desc: try out some vars
    cmds:
      - echo {{.ONE}}
      - echo {{.TWO}}
    vars:
      ONE: "one:one:one"
      TWO: '{{ splitList ":" .ONE }}'

resulting output:

template: :1:17: executing "" at <.ONE>: invalid value; expected string

The variable TWO doesn't expand/find ".ONE", but I couldn't find an explanation in the docs as to why. This is problematic as I would like to do several operations throughout the vars to parse a string which I then am going to use in multiple commands for http requests with curl.

However I can't seem to pass the resulting vars forward. I think this could relate to #277 in being able to run multiple commands in the same shell. Maybe a simple rationale here would be to interpolate vars in order from top to bottom as they're defined in the "vars"?

[andreynering]

Hi @tommyalatalo,

This issue is related to #287 and #218. i.e. variables in Task have some bugs and gotchas, and I plan to rewrite the whole thing in v3, specially respecting declaration order as you suggested, and avoiding the need for multiple passes/expansions if possible.

This should avoid a lot of confusion users have often regarding variables.

The reason for why it didn't happen yet is because I haven't find the time to work on this, but that will happen eventually.

[tommyalatalo]

Yeah I see now that the example in #287 is pretty much exactly what I was trying to do, get some string from git and refine in in the second variable. It will be really good to fix this, I worked around it for now by making really long commands where I did the parsing in the sh: module. Not optimal, and really ugly to be honest, but it works.

Do you have any idea of what the time frame would be to get this implemented?

[tommyalatalo]

This seems to be another behavior that's not covered in the docs:
If I add a dynamic variable with the "sh:" interpreter, then I can't add another variable without "sh:" because I get an implicit mapping pair error from yaml.

    vars:
      _ORIGIN: {sh: git config --get remote.origin.url}
      _PACKAGE: "{{ default "INIT_PACKAGE" .PACKAGE }}"

Doesn't matter if I write it in this format:

    vars:
      _ORIGIN: 
          sh: git config --get remote.origin.url
      _PACKAGE: "{{ default "INIT_PACKAGE" .PACKAGE }}"

Is this a known thing, and how do I get around it?

[andreynering]

Do you have any idea of what the time frame would be to get this implemented?

Sorry, but this can take a while. My time is limited and I try my best to keep moving this project forward.

I think I'll give priority to the vars rework, though, since it's the biggest blocker to the v3 release. But your reports are very useful so I can try to fix these issues.

[tommyalatalo]

I ran into another issue today when evaluating a sh variable, wondering if this is somehow related.
The below variable fails when trying to evaluate it.

PROJECT_PATH:
    sh: "cat .git/config | grep $(basename -s .git `git config --get remote.origin.url`)"

The error message doesn't really specify anything: task: Command "cat .git/config | grep $(basename -s .git `git config --get remote.origin.url` )" in taskvars file failed: exit status 1

However just running cat .git/config | grep $(basename -s .git `git config --get remote.origin.url`) in my shell (zsh or bash) works just fine.

[kevinkjt2000]

I bumped into something odd with variables today too:

version: '2'

vars:
  FIRST:
    sh: echo first
  SECOND:
    sh: echo second after {{.FIRST}}

tasks:
  default:
    cmds:

$ task
task: Command "echo second after <no value>" in taskvars file failed: 1:28: > must be followed by a word

Seems that I cannot reference variables from within the same section

[andreynering]

After #311 some of the confusion with variables should have been fixed. Task will now respect the order of declaration of variables.

If possible, give the v3 branch a try and give some feedback before the final release.