Formatting Docker Compose files removes the YAML document end

[ferrarimarco]

Hi all!

I noticed that formatting a Docker Compose file strips away the YAML end document marker (...). This doesn't happen if you manually set a Docker Compose file as a YAML file, and format it.

Steps to reproduce:

1. Create a Docker Compose file named compose.yaml (although any other file name recognized by the dockercompose language definition would work).
2. Ensure that VS Code recognizes the file compose.yaml file as a Docker Compose file (look for the Compose language mode in the status bar (bottom right).
3. Paste the following content in the file:

   ---
   services:
     test-service-1:
   ...

4. Save the file.
5. Format the file (if you don't have "Format on Save" enabled).

The resulting, formatted file is:

---
services:
  test-service-1:

While I was expecting:

---
services:
  test-service-1:
...

Is this behavior something I can configure? Or is it a bug?

Thanks!

[bwateratmsft]

I'll take a look but I suspect it's a bug. I'll transfer this to the language service repo.

[ferrarimarco]

Thanks @bwateratmsft !

[bwateratmsft]

@ferrarimarco Do you know if Docker Compose actually supports multiple YAML documents in one file? I've tried it out but it just seems to use only the first document and ignore everything else.

[ferrarimarco]

The compose file spec supports merging multiple compose files. The spec doesn't say anything about multiple YAML documents in the same file. So I'd say that having multiple YAML documents in the same file is not a use case that Docker Compose supports, which is consistent with your test.

I guess this makes this a formatting bug only, rather than one that causes a loss of functionality.

I tried to have a look at where compose-language-server sets YAML formatting options it uses the yaml package to provide this feature, if I got the code correctly.

There's one interesting option to play:directivesEndMarker

From the yaml package docs

Whether the document should always include a directives-end marker --- at its start, even if it includes no directives.

Unfortunately, I found no mentions of the document end marker ... in the yaml package docs or code.

[bwateratmsft]

Yeah, I'd noticed the same about directivesEndMarker. Apparently since the ... isn't strictly necessary it isn't being included at all.

[ferrarimarco]

Yup. Compose file are valid yaml files. Start and end markers can be there even if you have a single yaml document in a yaml file, according to the yaml spec. Given that both markers are optional, the formatter shouldn't mandate either, nor strip any marker if it's there. Maybe this is due more to a limitation of the yaml library that the formatter relies on, rather than an explicit design choice? Also, I was surprised about the name of that option, given that `---` is actually the yaml document start marker, not the end marker.

[bwateratmsft]

The formatter works by parsing the YAML document and then rewriting it using the yaml library. It will only act if there are no YAML syntax errors in the document; if there are syntax errors, it is more likely to harm than help so it's better to do nothing until the user fixes them.

While the --- and ... are valid in a single-document YAML file--why use them? Compose apparently doesn't support multi-document files, thus rendering those directives optional.

[ferrarimarco]

My use case is that I'm also parsing those yamls with other tools that use the document start and end markers to delimit those documents.

[ferrarimarco]

Anyway, I think the formatter shouldn't alter that part of the document, as the "plain" yaml formatter is doing. It shouldn't enforce either choice, given that those are optional parts of the yaml spec. Or, at least, make it configurable.

[bwateratmsft]

That makes sense. Can you open an issue at https://github.com/eemeli/yaml? It makes sense to me that the toString function should emit the ... at the end if it was present in the input, or at least provide an option.

[ferrarimarco]

Done: eemeli/yaml#368

[ferrarimarco]

@bwateratmsft eemeli/yaml#368 was merged.

[ferrarimarco]

@bwateratmsft Hi! It appears that the above PR made through the current eemeli/yaml release, so the necessary API should now be available.

In the version that compose-language-service ships now, the document start marker is not stripped if it's there, nor it's added if it's not there. I'd expect the same for the next version, but I'm not sure if that applies to the doc end marker.

[bwateratmsft]

It looks like yaml@2.0.0 just shipped so we'll switch over to that.

[ferrarimarco]

Thanks! I'll have a look as soon as a new release of the extension is available.

[ferrarimarco]

This is the related PR btw: microsoft/vscode-docker#3485

[bwateratmsft]

Glad to help! If you want to test it sooner, and you have Node.js installed, you can try out just the compose language service with the following steps:

1. Clone this repo
2. npm install from the repo root
3. cd src/test/clientExtension, and npm install in there too
4. Choose the launch config "Client + Server" in VSCode, and hit F5. This will launch an extension host, which activates the language server alone.

I use this approach for testing, since it's a lot quicker than packaging, installing the package in the Docker extension, and building and running that.