[carry 1738] Clear hook environ variables on empty Env #4323

lifubang · 2024-06-19T14:49:06Z

The runtime spec has:

env (array of strings, OPTIONAL) with the same semantics as IEEE Std 1003.1-2008's environ.

And running execle or similar with NULL env results in an empty environent:

$ cat test.c
#include <unistd.h>

int main()
{
  return execle("/usr/bin/env", "env", NULL, NULL);
}
$ cc -o test test.c
$ ./test
…no output…

Go's Cmd.Env, on the other hand, has:

If Env is nil, the new process uses the current process's environment.

This commit works around that by setting ~~a single dummy environment variable~~ []string{} in those cases to avoid leaking the runtime environment into the hooks.

kolyshkin · 2024-06-19T19:15:39Z

tests/integration/hooks.bats

+				"env": ["mm=tt"]
+			}]
+		}'
+		mm=nn runc run "test_hook-$hook"


Not sure what this like is doing here. A debug leftover?

Or maybe you wanted to check that this var is not visible? But it should be a different var name/value.

If you want a different name, what about runc_env or similar?

In my brain, the Prestart, Poststart, CreateContainer and CreateRuntime hooks are running in the host, not in the container? So they use host's env when running the hook.

Only StartContainer hook is running in the container.

kolyshkin · 2024-06-19T19:17:24Z

tests/integration/hooks.bats

+		update_config '.hooks = {
+			"'$hook'": [{
+				"path": "/bin/sh",
+				"args": ["/bin/sh", "-c", "[ \"$mm\"==\"tt\" ] && echo yes, we got tt from the env mm && exit 1 || exit 0"],


You can just have

"args": ["/bin/env"],

which prints all environment, and then check below that $output contains ^mm=tt$.

My bad, I mean "path", not "args".

In the beginning, I wanted to test it like the way you provided, but we should know that if the hook running success, there is no output. We should raise an error to get the output.
https://github.com/opencontainers/runc/blob/main/libcontainer/configs/config.go#L487-L493

kolyshkin · 2024-06-19T19:20:10Z

tests/integration/hooks.bats

+				"path": "/bin/sh",
+				"args": ["/bin/sh", "-c", "[[ \"$mm\" == \"nn\" ]] && echo \"$mm\" && exit 1 || exit 0"]


Same here, you can use

"path": "/bin/env"

and check that there is no output.

#4323 (comment)

kolyshkin · 2024-06-19T19:21:17Z

tests/integration/hooks.bats

+	done
+}
+
+@test "runc run [hook without env]" {


I would add what we check here ("hook without env does not inherit host env"). Maybe add a bug reference as well.

I guess this test fails today (without the PR), but it will be great to confirm it

I guess this test fails today (without the PR), but it will be great to confirm it

Yes, you are right, I have tested before I submit this PR.

rata

Thanks for carrying the fix!

rata · 2024-06-21T10:00:07Z

tests/integration/hooks.bats

+				"env": ["mm=tt"]
+			}]
+		}'
+		mm=nn runc run "test_hook-$hook"


Or maybe you wanted to check that this var is not visible? But it should be a different var name/value.

If you want a different name, what about runc_env or similar?

rata · 2024-06-21T10:01:07Z

tests/integration/hooks.bats

+
+@test "runc run [hook with env]" {
+	update_config '.process.args = ["/bin/true"]'
+	update_config '.process.env = ["mm=nn"]'


What if we use a more readable name?

Suggested change

update_config '.process.env = ["mm=nn"]'

update_config '.process.env = ["container_var=yes"]'

Because we do a test in one loop for hook in prestart createRuntime createContainer startContainer poststart; do, but as mentioned in #4323 (comment), the var may be in host, or may be in container, but we should use one var name, so we should not use a specific var name.

What about self-descriptive names such as TEST_VAR for variable name, and host_value and TEST_VAR=container_value (for values), or some such, to increase the test code readability.

rata · 2024-06-21T10:02:26Z

tests/integration/hooks.bats

+	done
+}
+
+@test "runc run [hook without env]" {


I guess this test fails today (without the PR), but it will be great to confirm it

lifubang · 2024-06-21T10:51:42Z

Even though this is a bug, but it has existed for many years, so if we merge this, it may cause a break change for docker and other tools which use runc for the container runtime. We should check a config.json file generated by containerd to see if there is a env config for hooks.

kolyshkin · 2024-06-26T01:37:09Z

What I see is crun works the same way, ie the current environment is passed on to the hook. Which is obviously wrong, but the fix may bring in some compatibility issues. Not sure how to assess if it's going to be the case...

lifubang · 2024-06-27T09:47:08Z

We should check a config.json file generated by containerd to see if there is a env config for hooks.

I had some time to look some config.json files in my company's cluster:

In k8s cluster: no hooks;
In docker swarm cluster: only one prestart hook, and using a absolute path:

cat config.json | jq .hooks
{
  "prestart": [
    {
      "path": "/proc/1158/exe",
      "args": [
        "libnetwork-setkey",
        "-exec-root=/var/run/docker",
        "2123cdedbd057cd58fba1c9869d410fcd1e6588f705e0eda6deb345d9a9670c0",
        "406aba1a8aaa"
      ]
    }
  ]
}

I also test the binary compiled from this PR with docker, it works fine for a normal use.

but the fix may bring in some compatibility issues.

It seems that there is no compatibility issues for a normal use, but I don't know whether there are issues for some other unknown complex scenarios.

The runtime spec has [1]: * env (array of strings, OPTIONAL) with the same semantics as IEEE Std 1003.1-2008's environ. And running execle or similar with NULL env results in an empty environent: $ cat test.c #include <unistd.h> int main() { return execle("/usr/bin/env", "env", NULL, NULL); } $ cc -o test test.c $ ./test ...no output... Go's Cmd.Env, on the other hand, has [2]: If Env is nil, the new process uses the current process's environment. This commit works around that by setting Env to an empty slice in those cases to avoid leaking the runtime environment into the hooks. [1]: https://github.com/opencontainers/runtime-spec/blob/v1.0.1/config.md#posix-platform-hooks [2]: https://golang.org/pkg/os/exec/#Cmd Signed-off-by: W. Trevor King <wking@tremily.us> (cherry picked from commit c11bd33) Signed-off-by: lfbzhm <lifubang@acmcoder.com>

Signed-off-by: lfbzhm <lifubang@acmcoder.com>

kolyshkin

Left a few comments, mostly about the test.

We also need a changelog entry as this might bring compatibility issues.

@opencontainers/runc-maintainers PTAL

kolyshkin · 2025-01-08T00:49:30Z

libcontainer/configs/config.go

@@ -480,6 +480,9 @@ func (c Command) Run(s *specs.State) error {
 		Stdout: &stdout,
 		Stderr: &stderr,
 	}
+	if cmd.Env == nil {
+		cmd.Env = []string{}


Maybe add a comment like

Suggested change

cmd.Env = []string{}

// If Env is not specified, do not inherit the current environment.

cmd.Env = []string{}

kolyshkin · 2025-01-08T00:55:05Z

tests/integration/hooks.bats

+@test "runc run [hook with env property]" {
+	update_config '.process.args = ["/bin/true"]'
+	update_config '.process.env = ["TEST_VAR=val"]'
+	# All hooks except Poststop.
+	for hook in prestart createRuntime createContainer startContainer poststart; do
+		echo "testing hook $hook"
+		# shellcheck disable=SC2016
+		update_config '.hooks = {
+			"'$hook'": [{
+				"path": "/bin/sh",
+				"args": ["/bin/sh", "-c", "[ \"$TEST_VAR\"==\"val\" ] && echo yes, we got val from the env TEST_VAR && exit 1 || exit 0"],
+				"env": ["TEST_VAR=val"]
+			}]
+		}'
+		TEST_VAR="val" runc run "test_hook-$hook"
+		[ "$status" -ne 0 ]
+		[[ "$output" == *"yes, we got val from the env TEST_VAR"* ]]
+	done
+}


So, this test case checks that hooks.$name.env is set as expected ($TEST_VAR == "val", but the test code also sets the same TEST_ENV=val for

process.env

host env

It's not clear why 1 and 2 are needed, and why they use the same name and value.

To me, if you want to check that host env and/or process env is not leaking into hook's env, you should use different names, and check they are not present.

Or, just don't set either host or process env, and merely check that hook env is exactly the same as specified.

kolyshkin · 2025-01-08T00:55:39Z

tests/integration/hooks.bats

+@test "runc run [hook without env property should not inherit host env]" {
+	update_config '.process.args = ["/bin/true"]'
+	update_config '.process.env = ["TEST_VAR=val"]'
+	# All hooks except Poststop.


Why not test poststop?

kolyshkin · 2025-01-08T00:57:09Z

tests/integration/hooks.bats

+# https://github.com/opencontainers/runtime-spec/blob/v1.0.1/config.md#posix-platform-hooks
+@test "runc run [hook without env property should not inherit host env]" {
+	update_config '.process.args = ["/bin/true"]'
+	update_config '.process.env = ["TEST_VAR=val"]'
+	# All hooks except Poststop.
+	for hook in prestart createRuntime createContainer startContainer poststart; do
+		echo "testing hook $hook"
+		# shellcheck disable=SC2016
+		update_config '.hooks = {
+			"'$hook'": [{
+				"path": "/bin/sh",
+				"args": ["/bin/sh", "-c", "[[ \"$TEST_VAR\" == \"val\" ]] && echo \"$TEST_VAR\" && exit 1 || exit 0"]
+			}]
+		}'
+		TEST_VAR="val" runc run "test_hook-$hook"
+		[ "$status" -eq 0 ]
+	done


Unlike the previous @test, this one actually makes sense (but maybe add a comment that here we set TEST_VAR=val for runc run and for the process spec, and check that neither one leaks to hook's env.

kolyshkin · 2025-01-08T00:59:50Z

tests/integration/hooks.bats

+@test "runc run [hook with env property]" {
+	update_config '.process.args = ["/bin/true"]'
+	update_config '.process.env = ["TEST_VAR=val"]'
+	# All hooks except Poststop.


Why not test poststop?

lifubang mentioned this pull request Jun 19, 2024

libcontainer/configs/config: Clear hook environ variables on empty Env #1738

Closed

lifubang force-pushed the feat-carry-1738-hook-clear-env branch 2 times, most recently from d3908aa to 6cedbc9 Compare June 19, 2024 15:30

kolyshkin reviewed Jun 19, 2024

View reviewed changes

rata reviewed Jun 21, 2024

View reviewed changes

lifubang force-pushed the feat-carry-1738-hook-clear-env branch from 6cedbc9 to 827bbdb Compare June 21, 2024 10:25

This comment was marked as duplicate.

Sign in to view

lifubang mentioned this pull request Jun 27, 2024

libct: speedup process.Env handling #4325

Merged

lifubang mentioned this pull request Jul 16, 2024

fix runc's poststart behaviour doesn't match the runtime-spec #4348

Draft

lifubang force-pushed the feat-carry-1738-hook-clear-env branch 3 times, most recently from 53ccb05 to eb74b56 Compare December 11, 2024 15:33

lifubang force-pushed the feat-carry-1738-hook-clear-env branch from eb74b56 to a1861dc Compare December 11, 2024 15:34

add tests for env param in hooks

c635e4b

Signed-off-by: lfbzhm <lifubang@acmcoder.com>

lifubang force-pushed the feat-carry-1738-hook-clear-env branch from a1861dc to c635e4b Compare December 11, 2024 15:57

kolyshkin requested changes Jan 8, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[carry 1738] Clear hook environ variables on empty Env #4323

[carry 1738] Clear hook environ variables on empty Env #4323

lifubang commented Jun 19, 2024

kolyshkin Jun 19, 2024

rata Jun 21, 2024

lifubang Jun 21, 2024

lifubang Jun 21, 2024

kolyshkin Jun 19, 2024

kolyshkin Jun 19, 2024

lifubang Jun 21, 2024

kolyshkin Jun 19, 2024

lifubang Jun 21, 2024

kolyshkin Jun 19, 2024

rata Jun 21, 2024

lifubang Jun 21, 2024

rata left a comment

rata Jun 21, 2024

rata Jun 21, 2024

lifubang Jun 21, 2024 •

edited

Loading

kolyshkin Dec 10, 2024

rata Jun 21, 2024

lifubang commented Jun 21, 2024

This comment was marked as duplicate.

kolyshkin commented Jun 26, 2024

lifubang commented Jun 27, 2024

kolyshkin left a comment

kolyshkin Jan 8, 2025

kolyshkin Jan 8, 2025

kolyshkin Jan 8, 2025

kolyshkin Jan 8, 2025

kolyshkin Jan 8, 2025

		"path": "/bin/sh",
		"args": ["/bin/sh", "-c", "[[ \"$mm\" == \"nn\" ]] && echo \"$mm\" && exit 1 \|\| exit 0"]

	update_config '.process.env = ["mm=nn"]'
	update_config '.process.env = ["container_var=yes"]'

	cmd.Env = []string{}
	// If Env is not specified, do not inherit the current environment.
	cmd.Env = []string{}

[carry 1738] Clear hook environ variables on empty Env #4323

Are you sure you want to change the base?

[carry 1738] Clear hook environ variables on empty Env #4323

Conversation

lifubang commented Jun 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rata left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lifubang Jun 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lifubang commented Jun 21, 2024

This comment was marked as duplicate.

kolyshkin commented Jun 26, 2024

lifubang commented Jun 27, 2024

kolyshkin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lifubang Jun 21, 2024 •

edited

Loading