"invalid template" error from certain YAML comments

[jrykr]

Description

A previously valid docker compose file started raising an error with docker compose v2. I that noticed a possibly similar issue was raised a few days ago in a public project.

My workaround was to revise the text in my YAML comments in the place where comments described a command.

Note, this is only with Docker Compose v2, I didn't encounter any errors using docker-compose v1.29.

Results of my quick experimentation with YAML comments:

# `echo $(pwd)` <- invalid template error
# `echo foo` <- works
# awk '{print $3}' <- invalid template error
# awk '{print}' <- works
# $1 <- invalid template error
# $FOO <- works

Steps to reproduce the issue:

1. Add the following line to any valid docker compose file: # $1.
2. Run docker compose -f docker-compose.yaml config

Describe the results you received:

$ docker compose up
Invalid template: "version: \"3.9\"\nservices:\n  <SNIPPED> \n"

Describe the results you expected:

The previously valid file continues to be valid.

Additional information you deem important (e.g. issue happens only occasionally):

n/a

Output of docker compose version:

This is the version where I'm encountering the errors.

Docker Compose version 2.0.0-beta.4

And docker-compose --version, just FYI, no errors running the same file with this:

docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020

Output of docker version:

Client:
 Cloud integration: 1.0.17
 Version:           20.10.7
 API version:       1.41
 Go version:        go1.16.4
 Git commit:        f0df350
 Built:             Wed Jun  2 11:56:22 2021
 OS/Arch:           darwin/amd64
 Context:           desktop-linux
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.7
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       b0f5bc3
  Built:            Wed Jun  2 11:54:58 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.6
  GitCommit:        d71fcd7d8303cbf684402823e425e9dd2e99285d
 runc:
  Version:          1.0.0-rc95
  GitCommit:        b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker context show:
You can also run docker context inspect context-name to give us more details but don't forget to remove sensitive content.

desktop-linux

Output of docker info:

Client:
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
  compose: Docker Compose (Docker Inc., 2.0.0-beta.4)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 8
 Server Version: 20.10.7
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d
 runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.25-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 1.941GiB
 Name: docker-desktop
 ID: 2GCX:VMBV:LPFZ:QCQ6:ESVL:M37D:DCZK:LGQM:2X5D:EJOS:HCC3:YQIJ
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details (AWS ECS, Azure ACI, local, etc.):

Only tested local.

[ndeloof]

I can't reproduce this error with 2.0.0-beta.6, but it seems this is caused by variable interpolation to inadequately apply inside comments.

Variable substitution is currently implemented as plain string replacement on compose file content.
We should pre-process this content to drop all comments, so that we only process the actual yaml configuration.

[ndeloof]

internally tracked as https://docker.atlassian.net/browse/IL-395

[markshust]

The mention above with the fix at https://github.com/alan-turing-institute/AutSPACEs/pull/132/files hinted at me that dollar signs $ within comments are causing this "invalid template" error. I had a similar fix in markshust/docker-magento#486 which seems to resolve this issue.

[Seros]

It seems this bug got re-introduced in https://github.com/docker/compose as I use version 2.9.0 and observe the same behavior with dollar signs in a .env file

[glours]

@Seros Can you try this, please?

[Seros]

I tested and both the double sign and the quotes are working. Thank you!

[maharg101]

Seeing the same issue here with Docker Compose version v2.7.0 since a recent update to docker desktop on Mac. Any dollar signs in .env file values trigger Invalid template.

[gquittet]

I agree just adding my variable into quote fixed the issue 🎉

I think it's a regression 🤔

Before

ENV=env_value$

After

ENV='env_value$'

---

#1896 (comment)

[TimTim74]

I am seeing the same issue on Linux with Docker Compose v2.10.2. Any dollar signs in .env file values trigger Invalid template.

[PipeKnight]

Guys, I had the same issue today with such code in .env:
TEST_USER_PASSWORD="lalala$!"

I have fixed it like this and docker-compose started to work:
TEST_USER_PASSWORD="lalala\$!"

So I hope you all can use this solution, just make it an escape character with backslash

P.S Docker Compose version v2.10.2

[lily2908]

It seems this bug got re-introduced in https://github.com/docker/compose as I use version 2.9.0 and observe the same behavior with dollar signs in a .env file

thanks for your answer.

[gquittet]

@ndeloof I think we should reopen this issue or create another one

[digitaltim-de]

Same issue here, if you have installed the CORS extension from Symfony, they have wildcards in env. Delete it, fixing, really.

[cross19xx]

For mine, I wrapped the entire defaulting string in single quotes and it worked

[MHLut]

Can confirm this issue occurs when using a dollar sign in any value in .env

You can fix this by wrapping the value in single quotes. Wrapping the value in double quotes doesn't fix it.

[miketheman]

See docker/compose#9746 for more - appears resolved in version 2.11.2