--overlay-named exits with error as of linux 5.1.15 (overlayfs)

[lskrejci]

Running firejail --noprofile --overlay-named=test fails:

• On a system with a separate /home partition:

Error mounting overlayfs for mounted home directory: fs.c:1132 fs_overlayfs: Too many levels of symbolic links

• On a system with a single root partition:

Error mounting overlayfs: fs.c:1077 fs_overlayfs: Too many levels of symbolic links

dmesg on both system configurations contains:

overlayfs: overlapping upperdir path

The same command works on linux 5.1.14.
Upstream commit that appears to have introduced the issue: ovl: detect overlapping layers

Environment:

• Distribution: Arch Linux
• firejail: 0.9.60-1
• linux: 5.1.15.arch1-1

Relates to:

• CVE-2021-26910 and `--overlay-tmpfs` #4178

[Hello71]

I think --rbind might be able to work around this.

[Vincent43]

Fix for docker: moby/moby@477bf1e

[smitsohu]

Maybe I don't fully understand the docker fix, but I wonder a bit what's the point of an overlay just to make the file system read-only.

@netblue30 do you have an idea regarding the way forward?

[smitsohu]

but I wonder a bit what's the point of an overlay just to make the file system read-only.

Or maybe it could serve as a simple switch for apps that really don't need to write anything at all.

[Vincent43]

There is some kernel fix available in Linux 5.3.1 and 5.2.17, @lskrejci could you try if you can still reproduce issue no those kernels?

[Vincent43]

Still fails on Linux 5.2.17 so some firejail changes are still needed:

firejail --noprofile --overlay-named=test
Parent pid 9167, child pid 9168

**     Warning: dropping all Linux capabilities     **
Error mounting overlayfs: fs.c:977 fs_overlayfs: Too many levels of symbolic links
Error: proc 9167 cannot sync with peer: unexpected EOF
Peer 9168 unexpectedly exited with status 1

[schendstok]

The same problem happens with the latest Linux kernel update in Debian Buster.

With the previous 4.19.0-5 kernel Firejail works fine with the overlay feature, but when you upgrade to 4.19.0-6 you get the same "fs_overlayfs: Too many levels of symbolic links" error

[Vincent43]

I guess debian backported upstream patches as they were security related.

[Hocuri]

I have the same problem (at least I think that it's the same) and some logs:

$ journalctl -f
...
Okt 20 10:21:37 j2 kernel: overlayfs: filesystem on '/home/hocuri/.firejail/4112/odiff' not supported as upperdir

and:

$ firejail --overlay --noprofile       

Parent pid 4422, child pid 4423

**     Warning: dropping all Linux capabilities     **
Error mounting overlayfs: fs.c:1077 fs_overlayfs: Invalid argument
Error: proc 4422 cannot sync with peer: unexpected EOF
Peer 4423 unexpectedly exited with status 1

Kernel 4.19.79-1-MANJARO x86_64.

[netblue30]

For now, I ended up disabling --overlay feature for kernels 4.19 and newer.

I am getting "fs_overlayfs: Too many levels of symbolic links" on debian stable, kernel 4.19.

[Hello71]

I recall testing and finding that mount --rbind did fix the issue. I can't quite recall what I bind mounted though.

[Hello71]

I think it's a similar trick to making pivot_root works, you do mount --rbind $dir $dir and then that "tricks" the kernel into allowing it.

[netblue30]

thanks, I'll try it out!

[springzfx]

Distribution: Arch Linux
firejail: 0.9.60-1
linux: 5.4.3.arch1-1
still have the same problem.

[rusty-snake]

@springzfx OP use the same firejail version. There is a release coming, see release-0.9.62 branch.

[springzfx]

Distribution: Arch Linux
firejail: 0.9.62
linux: 5.4.3.arch1-1
No luck.

[darkf]

This still happens, any fixes in sight? Any viable workaround?

[jonleivent]

Instead of waiting for kernel fixes related to this, could it be worked around by having firejail use:
https://github.com/containers/fuse-overlayfs

[0x0D15]

@netblue30 Is there any planned fix or known workaround for this issue? It was one of my favorite things about this project.

[Hello71]

took another look at the issue. I think the kernel is valid to reject this mount. if you do firejail --overlay-named=x and then access $HOME/.firejail/x then there would be a cycle. but, that change also added runtime checking, so you can apply a patch like this:

From 507605925f22fa2ec3f3ef0ce9ac747139495f88 Mon Sep 17 00:00:00 2001
From: "Alex Xu (Hello71)" <alex_y_xu@yahoo.ca>
Date: Sun, 29 Mar 2020 17:18:20 -0400
Subject: [PATCH] ovl: add ignore_overlap option

check overlaps at runtime, allows firejail --overlay-named

Signed-off-by: Alex Xu (Hello71) <alex_y_xu@yahoo.ca>
---
 fs/overlayfs/ovl_entry.h |  1 +
 fs/overlayfs/super.c     | 37 +++++++++++++++++++++++++++++++++----
 2 files changed, 34 insertions(+), 4 deletions(-)

diff --git a/fs/overlayfs/ovl_entry.h b/fs/overlayfs/ovl_entry.h
index 89015ea822e7..f831c06f3bd5 100644
--- a/fs/overlayfs/ovl_entry.h
+++ b/fs/overlayfs/ovl_entry.h
@@ -17,6 +17,7 @@ struct ovl_config {
 	bool nfs_export;
 	int xino;
 	bool metacopy;
+	bool ignore_overlap;
 };
 
 struct ovl_sb {
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index ac967f1cb6e5..f670559defdb 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -403,6 +403,8 @@ enum {
 	OPT_XINO_AUTO,
 	OPT_METACOPY_ON,
 	OPT_METACOPY_OFF,
+	OPT_IGNORE_OVERLAP_ON,
+	OPT_IGNORE_OVERLAP_OFF,
 	OPT_ERR,
 };
 
@@ -421,6 +423,8 @@ static const match_table_t ovl_tokens = {
 	{OPT_XINO_AUTO,			"xino=auto"},
 	{OPT_METACOPY_ON,		"metacopy=on"},
 	{OPT_METACOPY_OFF,		"metacopy=off"},
+	{OPT_IGNORE_OVERLAP_ON,		"ignore_overlap=on"},
+	{OPT_IGNORE_OVERLAP_OFF,	"ignore_overlap=off"},
 	{OPT_ERR,			NULL}
 };
 
@@ -559,6 +563,14 @@ static int ovl_parse_opt(char *opt, struct ovl_config *config)
 			config->metacopy = false;
 			break;
 
+		case OPT_IGNORE_OVERLAP_ON:
+			config->ignore_overlap = true;
+			break;
+
+		case OPT_IGNORE_OVERLAP_OFF:
+			config->ignore_overlap = false;
+			break;
+
 		default:
 			pr_err("unrecognized mount option \"%s\" or missing value\n",
 					p);
@@ -1024,6 +1036,19 @@ static int ovl_report_in_use(struct ovl_fs *ofs, const char *name)
 	}
 }
 
+static int ovl_report_trap_inode(struct ovl_fs *ofs, const char *name)
+{
+	if (ofs->config.ignore_overlap) {
+		pr_warn("overlapping %s path, accessing overlapped path will result in ELOOP.\n",
+			name);
+		return 0;
+	} else {
+		pr_err("overlapping %s path, mount with '-o ignore_overlap=on' to override overlapping upperdir protection.\n",
+			name);
+		return -ELOOP;
+	}
+}
+
 static int ovl_get_upper(struct super_block *sb, struct ovl_fs *ofs,
 			 struct path *upperpath)
 {
@@ -1537,18 +1562,22 @@ static int ovl_check_layer(struct super_block *sb, struct ovl_fs *ofs,
 	/* Walk back ancestors to root (inclusive) looking for traps */
 	while (!err && parent != next) {
 		if (ovl_lookup_trap_inode(sb, parent)) {
-			err = -ELOOP;
-			pr_err("overlapping %s path\n", name);
-		} else if (ovl_is_inuse(parent)) {
+			err = ovl_report_trap_inode(ofs, name);
+			if (err)
+				goto out;
+		}
+		if (ovl_is_inuse(parent)) {
 			err = ovl_report_in_use(ofs, name);
+			if (err)
+				goto out;
 		}
 		next = parent;
 		parent = dget_parent(next);
 		dput(next);
 	}
 
+out:
 	dput(parent);
-
 	return err;
 }
 
-- 
2.26.0

then, patch firejail to use mount -o ignore_overlap=on.

[firejailing]

@netblue30 where are you?

this has been broken for over a year now, ridiculous, new profiles are less important than fixing this

[shuhaowu]

Are there no workarounds for this for now?

[reinerh]

Another workaround has been suggested in: https://bugs.debian.org/971578

[garywill]

--overlay used to work for me in May 2019 (on openSUSE 15.0 or 15.1).
But not now. Now I'm on openSUSE 15.2 with kernel 5.3.18

# firejail --name=test --overlay --noprofile --debug
Autoselecting /bin/bash as shell
Command name #/bin/bash#
DISPLAY=:0 parsed as 0
Enabling IPC namespace
Using the local network stack
Parent pid 7586, child pid 7587
The new log directory is /proc/7587/root/var/log
Initializing child process
Host network configured
PID namespace installed
Mounting tmpfs on /run/firejail/mnt directory
Creating empty /run/firejail/mnt/seccomp directory
Creating empty /run/firejail/mnt/seccomp/seccomp.protocol file
Creating empty /run/firejail/mnt/seccomp/seccomp.postexec file
Creating empty /run/firejail/mnt/seccomp/seccomp.postexec32 file
Linux kernel version 5.3
Mounting OverlayFS
Debug: running on kernel version 5.3
Error mounting overlayfs: fs.c:1063 fs_overlayfs: Too many levels of symbolic links
Error: proc 7586 cannot sync with peer: unexpected EOF
Peer 7587 unexpectedly exited with status 1


# firejail --version
firejail version 0.9.64


# uname -a
Linux linux-lwid 5.3.18-lp152.57-default #1 SMP Fri Dec 4 07:27:58 UTC 2020 (7be5551) x86_64 x86_64 x86_64 GNU/Linux

Wish a fix. Thanks all developers 👍 :)

[pushqrdx]

5.8.0-48-generic

same here

[Ristovski]

@netblue30 Any news on this? I too like the suggested alternative of using fuse-overlayfs which is also what podman uses for example.

[rusty-snake]

#6632 (comment):

• The usage in firejail was broken at least two times in the past because of kernel changes (IIRC). And is likely to break again in the future.
  https://www.kernel.org/doc/html/latest/filesystems/overlayfs.html

  Changes to the underlying filesystems while part of a mounted overlay filesystem are not allowed. If the underlying filesystem is changed, the behavior of the overlay is undefined, though it will not result in a crash or deadlock.

  systemd-nspawn has a correct implementation. It cp -a --reflink=auto /*~(proc|sys|dev|...|._my_lower_dir_root) /.my_lower_dir_root first and then uses /.my_lower_dir_root as lower-dir. However this is very slow and inefficient unless you have everything in one Btrfs/XFS.