docker-driver: loki-relabel-config does not rename container_name to the source label

[rgl]

Describe the bug

loki-relabel-config does not rename container_name to the source label.

NB I'm not sure if this is related to #3812.

To Reproduce

Steps to reproduce the behavior:

1. Start Loki 2.2.1
2. Install the docker-driver 2.2.1 docker plugin
3. Set the following docker 20.10.7 daemon configuration:

   {
     "debug": true,
     "log-driver": "loki",
     "log-opts": {
       "labels": "worker_id",
       "loki-url": "http://10.10.0.2:3100/loki/api/v1/push",
       "loki-external-labels": "job=docker",
       "loki-relabel-config": "- regex: filename|source\n  action: labeldrop\n- regex: container_name\n  replacement: source\n  action: labelmap\n"
     }
   }

4. Query: logcli query --tail '{job="docker"}'

Got:

2021-06-07T08:31:45+01:00 {container_name="hello-docker", host="linuxkit-52540039804b", job="docker", source="stdout", worker_id="123"} hello docker Mon Jun 7 07:31:45 UTC 2021

Expected behavior

I expected the container_name label to be renamed to source.

[kavirajk]

thanks @rgl for reporting. I think its also related to #3811 (comment)

document needs to be bit clear. container_name is added only if no external labels are provided!

[rgl]

I think, this issue is a mixture of two nuanced things (that I'm now able to connect, because of what @cyriltovena described at #3812 (comment)):

1. the loki-relabel-config setting is only executed once per container.
2. the source label is added (by docker) to each container log message. as such, it does not exist at the time loki-relabel-config is executed, so it is impossible for my loki-relabel-config setting to do what I wanted to do.

To actually rename the source label we have to use the loki-pipeline-stages setting instead (which is executed for each container log message).

[rgl]

Thank you for the help!

Its now working with:

{
  "debug": true,
  "log-driver": "loki",
  "log-opts": {
    "labels": "worker_id",
    "loki-url": "http://10.10.0.2:3100/loki/api/v1/push",
    "loki-external-labels": "job=docker,container_name={{.Name}}",
    "loki-relabel-config": "- regex: filename\n  action: labeldrop\n",
    "loki-pipeline-stages": "- labels:\n    source: container_name\n- labeldrop:\n    - container_name\n"
  }
}