firejail hangs with net parameter

[fjthrowaway]

Hi,

Bug and expected behavior

some recent arch update must have (partially) broken firejails --net parameter. It used to work fine but now the program just hangs when executed with --net on a bridge device.
For some reason, one of my systemd startup service has no issues starting with --net.
What I have tried so far: downgrade kernel, downgrade networkmanager, downgrade firejail with no success.
I have also encountered #3948 which could be "solved" by downgrading the kernel from 5.10.13 to 5.10.12
I used networkmanager to create the bridge.

No profile and disabling firejail

• What changed calling firejail --noprofile /path/to/program in a terminal?
  nothing changed
• What changed calling the program by path (check which <program> or firejail --list while the sandbox is running)?
  nothing changed

Reproduce
Steps to reproduce the behavior:

1. Run in bash firejail --noprofile --net=br1 --defaultgw=192.168.0.1 --dns=192.168.0.1 --ip=192.168.0.254 --debug sh
2. See error

Autoselecting /bin/bash as shell
Building quoted command line: 'sh' 
Command name #sh#
get interface br1 configuration
MTU of br1 is 1500.
Bridge device br1 at 192.168.0.2/24
Trying 192.168.0.254 ...

3. It just hangs forever.

Environment

• Linux distribution and version (ie output of lsb_release -a, screenfetch or cat /etc/os-release)

cat /etc/os-release
NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
ANSI_COLOR="38;2;23;147;209"
HOME_URL="https://www.archlinux.org/"
DOCUMENTATION_URL="https://wiki.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://bugs.archlinux.org/"
LOGO=archlinux

• Firejail version (output of firejail --version) exclusive or used git commit (git rev-parse HEAD)

firejail version 0.9.64.2

Compile time support:
	- AppArmor support is enabled
	- AppImage support is enabled
	- chroot support is enabled
	- D-BUS proxy support is enabled
	- file and directory whitelisting support is enabled
	- file transfer support is enabled
	- firetunnel support is enabled
	- networking support is enabled
	- overlayfs support is enabled
	- private-home support is enabled
	- private-cache and tmpfs as user enabled
	- SELinux support is disabled
	- user namespace support is enabled
	- X11 sandboxing support is enabled

debug output

Autoselecting /bin/bash as shell
Building quoted command line: 'sh' 
Command name #sh#
get interface br1 configuration
MTU of br1 is 1500.
Bridge device br1 at 192.168.0.2/24
Trying 192.168.0.254 ...

[waterlubber]

I'm having the same issue. I tried with --noprofile and with various network interfaces. It would occasionally start, but didn't work with any actual interface or bridge interface.

[smitsohu]

We got another report about problems with AppArmor, which automagically resolve upon kernel upgrade to 5.10.16.

If you get a chance, could you confirm the issues persist with a newer kernel?

[fjthrowaway]

I just tried 5.10.16 and had mixed results with it. First few attempts didn't work (i.e. stuck) but then it started working.

The problem is that I can't use 5.10.16 due to bug #3948. I can't enter any sandbox with 5.10.16.

[smitsohu]

The problem is that I can't use 5.10.16 due to bug #3948. I can't enter any sandbox with 5.10.16.

Do you see something in the system log?

(only for the record: I cannot reproduce either issue on 5.10.13 kernels of Debian or Fedora, so this looks like something Arch specific)

[fjthrowaway]

(only for the record: I cannot reproduce either issue on 5.10.13 kernels of Debian or Fedora, so this looks like something Arch specific)

Could it be this commit?

the other changes seem unrelated

[matthew-nichols]

I just updated from Pop OS 20.10 to 21.04. 21.04 appears to have this issue while it was working in 20.10.

firejail version 0.9.64.4

Compile time support:
	- AppArmor support is enabled
	- AppImage support is enabled
	- chroot support is enabled
	- D-BUS proxy support is enabled
	- file and directory whitelisting support is enabled
	- file transfer support is enabled
	- firetunnel support is enabled
	- networking support is enabled
	- overlayfs support is disabled
	- private-home support is enabled
	- private-cache and tmpfs as user enabled
	- SELinux support is enabled
	- user namespace support is enabled
	- X11 sandboxing support is enabled
$ lsb_release -a
No LSB modules are available.
Distributor ID:	Pop
Description:	Pop!_OS 21.04
Release:	21.04
Codename:	hirsute
$ uname -a
Linux pop-desktop 5.11.0-7620-generic #21~1624379747~21.04~3abeff8-Ubuntu SMP Wed Jun 23 02:34:03 UTC  x86_64 x86_64 x86_64 GNU/Linux

[glitsj16]

@matthew-nichols Latest Firejail release is 0.9.66. You can install that via https://launchpad.net/~deki/+archive/ubuntu/firejail. Would be appreciated if you could confirm your issue with that version.

[matthew-nichols]

Slightly different result (operation not permitted as normal user, hang as root)

$ firejail --net=br0 --noprofile --debug
Autoselecting /bin/bash as shell
Command name #/bin/bash#
get interface br0 configuration
MTU of br0 is 1500.
Bridge device br0 at 192.168.5.9/24
ARP-scan br0, 192.168.5.9/24
IP address range from 192.168.5.1 to 192.168.5.255
Trying 192.168.5.51 ...
Error socket: arp.c:132 arp_check: Operation not permitted
$ sudo firejail --net=br0 --noprofile --debug
sudo firejail --net=br0 --noprofile --debug
Autoselecting /bin/bash as shell
Command name #/bin/bash#
get interface br0 configuration
MTU of br0 is 1500.
Bridge device br0 at 192.168.5.9/24
ARP-scan br0, 192.168.5.9/24
IP address range from 192.168.5.1 to 192.168.5.255
Trying 192.168.5.51 ...

[minus7]

I observe the same symptoms (firejail hangs on startup at "Trying ..." for a long time and it eventually starting). So far I worked around that by patching out the call to arp_check in net_configure_sandbox_ip.
After some debugging, I think I found the issue: arp_check intends to probe for the configured IP address twice, with a delay of 500ms between ARP requests. But if the interface gets a lot of traffic, the timeout on select doesn't trigger. select may decrease the given timeout value after returning early, but the actual behavior is unspecified by POSIX. And despite the select(2) man page saying that the timeout does get decreased on Linux, that is not the case on my machine.

[minus7]

I'm not sure why it's receiving so many packets, but one reason might be that a recvfrom of fixed length ETH_FRAME_LEN is not correct, assuming the ethernet frames are streamed in on the packet socket and the rest of the frame is not discarded.