Ubuntu 22.04 Exception: Failed to compile BPF module <text>

[megastallman]

Hi guys!
I'm trying to install BCC on my Kubuntu 22.04. Tried both ways, packages and source, having the same issue, please help.
From repo(apt install bpfcc-tools):

# tcptracer-bpfcc 
In file included from <built-in>:2:
In file included from /virtual/include/bcc/bpf.h:12:
In file included from include/linux/types.h:6:
In file included from include/uapi/linux/types.h:14:
In file included from include/uapi/linux/posix_types.h:5:
In file included from include/linux/stddef.h:5:
In file included from include/uapi/linux/stddef.h:2:
In file included from include/linux/compiler_types.h:80:
include/linux/compiler-clang.h:41:9: warning: '__HAVE_BUILTIN_BSWAP32__' macro redefined [-Wmacro-redefined]
#define __HAVE_BUILTIN_BSWAP32__
        ^
<command line>:4:9: note: previous definition is here
#define __HAVE_BUILTIN_BSWAP32__ 1
        ^
In file included from <built-in>:2:
In file included from /virtual/include/bcc/bpf.h:12:
In file included from include/linux/types.h:6:
In file included from include/uapi/linux/types.h:14:
In file included from include/uapi/linux/posix_types.h:5:
In file included from include/linux/stddef.h:5:
In file included from include/uapi/linux/stddef.h:2:
In file included from include/linux/compiler_types.h:80:
include/linux/compiler-clang.h:42:9: warning: '__HAVE_BUILTIN_BSWAP64__' macro redefined [-Wmacro-redefined]
#define __HAVE_BUILTIN_BSWAP64__
        ^
<command line>:5:9: note: previous definition is here
#define __HAVE_BUILTIN_BSWAP64__ 1
        ^
In file included from <built-in>:2:
In file included from /virtual/include/bcc/bpf.h:12:
In file included from include/linux/types.h:6:
In file included from include/uapi/linux/types.h:14:
In file included from include/uapi/linux/posix_types.h:5:
In file included from include/linux/stddef.h:5:
In file included from include/uapi/linux/stddef.h:2:
In file included from include/linux/compiler_types.h:80:
include/linux/compiler-clang.h:43:9: warning: '__HAVE_BUILTIN_BSWAP16__' macro redefined [-Wmacro-redefined]
#define __HAVE_BUILTIN_BSWAP16__
        ^
<command line>:3:9: note: previous definition is here
#define __HAVE_BUILTIN_BSWAP16__ 1
        ^
In file included from /virtual/main.c:17:
In file included from include/net/sock.h:46:
In file included from include/linux/netdevice.h:41:
In file included from include/net/netprio_cgroup.h:11:
In file included from include/linux/cgroup.h:28:
In file included from include/linux/cgroup-defs.h:22:
In file included from include/linux/bpf-cgroup.h:5:
include/linux/bpf.h:211:35: error: invalid application of 'sizeof' to an incomplete type 'struct bpf_timer'
                memset(dst + map->timer_off, 0, sizeof(struct bpf_timer));
                                                ^     ~~~~~~~~~~~~~~~~~~
include/linux/bpf.h:211:49: note: forward declaration of 'struct bpf_timer'
                memset(dst + map->timer_off, 0, sizeof(struct bpf_timer));
                                                              ^
include/linux/bpf.h:225:10: error: invalid application of 'sizeof' to an incomplete type 'struct bpf_timer'
                t_sz = sizeof(struct bpf_timer);
                       ^     ~~~~~~~~~~~~~~~~~~
include/linux/bpf.h:225:24: note: forward declaration of 'struct bpf_timer'
                t_sz = sizeof(struct bpf_timer);
                                     ^
In file included from /virtual/main.c:17:
In file included from include/net/sock.h:59:
include/linux/filter.h:1518:44: error: use of undeclared identifier 'BPF_F_BROADCAST'; did you mean 'IFF_BROADCAST'?
        if (unlikely(!ri->tgt_value) && !(flags & BPF_F_BROADCAST)) {
                                                  ^~~~~~~~~~~~~~~
                                                  IFF_BROADCAST
include/uapi/linux/if.h:86:2: note: 'IFF_BROADCAST' declared here
        IFF_BROADCAST                   = 1<<1,  /* volatile */
        ^
In file included from /virtual/main.c:17:
In file included from include/net/sock.h:59:
include/linux/filter.h:1533:14: error: use of undeclared identifier 'BPF_F_BROADCAST'; did you mean 'IFF_BROADCAST'?
        if (flags & BPF_F_BROADCAST) {
                    ^~~~~~~~~~~~~~~
                    IFF_BROADCAST
include/uapi/linux/if.h:86:2: note: 'IFF_BROADCAST' declared here
        IFF_BROADCAST                   = 1<<1,  /* volatile */
        ^
3 warnings and 4 errors generated.
Traceback (most recent call last):
  File "/usr/sbin/tcptracer-bpfcc", line 611, in <module>
    b = BPF(text=bpf_text)
  File "/usr/lib/python3/dist-packages/bcc/__init__.py", line 364, in __init__
    raise Exception("Failed to compile BPF module %s" % (src_file or "<text>"))
Exception: Failed to compile BPF module <text>

And from source:

apt install python-is-python3
apt install -y bison build-essential cmake flex git libedit-dev   libllvm11 llvm-11-dev libclang-11-dev zlib1g-dev libelf-dev libfl-dev python3-distutils
cd /opt/bcc
git clone https://github.com/iovisor/bcc
mkdir bcc/build; cd bcc/build
cmake -DCMAKE_INSTALL_PREFIX=/opt/bcc ..
make
make install
cmake -DCMAKE_INSTALL_PREFIX=/opt/bcc -DPYTHON_CMD=python3 ..
pushd src/python/
make
make install
popd

Running my build:

# /opt/bcc/bcc/tools/tcptracer.py 
In file included from <built-in>:2:
In file included from /virtual/include/bcc/bpf.h:12:
In file included from include/linux/types.h:6:
In file included from include/uapi/linux/types.h:14:
In file included from include/uapi/linux/posix_types.h:5:
In file included from include/linux/stddef.h:5:
In file included from include/uapi/linux/stddef.h:2:
In file included from include/linux/compiler_types.h:80:
include/linux/compiler-clang.h:41:9: warning: '__HAVE_BUILTIN_BSWAP32__' macro redefined [-Wmacro-redefined]
#define __HAVE_BUILTIN_BSWAP32__
        ^
<command line>:4:9: note: previous definition is here
#define __HAVE_BUILTIN_BSWAP32__ 1
        ^
In file included from <built-in>:2:
In file included from /virtual/include/bcc/bpf.h:12:
In file included from include/linux/types.h:6:
In file included from include/uapi/linux/types.h:14:
In file included from include/uapi/linux/posix_types.h:5:
In file included from include/linux/stddef.h:5:
In file included from include/uapi/linux/stddef.h:2:
In file included from include/linux/compiler_types.h:80:
include/linux/compiler-clang.h:42:9: warning: '__HAVE_BUILTIN_BSWAP64__' macro redefined [-Wmacro-redefined]
#define __HAVE_BUILTIN_BSWAP64__
        ^
<command line>:5:9: note: previous definition is here
#define __HAVE_BUILTIN_BSWAP64__ 1
        ^
In file included from <built-in>:2:
In file included from /virtual/include/bcc/bpf.h:12:
In file included from include/linux/types.h:6:
In file included from include/uapi/linux/types.h:14:
In file included from include/uapi/linux/posix_types.h:5:
In file included from include/linux/stddef.h:5:
In file included from include/uapi/linux/stddef.h:2:
In file included from include/linux/compiler_types.h:80:
include/linux/compiler-clang.h:43:9: warning: '__HAVE_BUILTIN_BSWAP16__' macro redefined [-Wmacro-redefined]
#define __HAVE_BUILTIN_BSWAP16__
        ^
<command line>:3:9: note: previous definition is here
#define __HAVE_BUILTIN_BSWAP16__ 1
        ^
In file included from /virtual/main.c:17:
In file included from include/net/sock.h:46:
In file included from include/linux/netdevice.h:41:
In file included from include/net/netprio_cgroup.h:11:
In file included from include/linux/cgroup.h:28:
In file included from include/linux/cgroup-defs.h:22:
In file included from include/linux/bpf-cgroup.h:5:
include/linux/bpf.h:211:35: error: invalid application of 'sizeof' to an incomplete type 'struct bpf_timer'
                memset(dst + map->timer_off, 0, sizeof(struct bpf_timer));
                                                ^     ~~~~~~~~~~~~~~~~~~
include/linux/bpf.h:211:49: note: forward declaration of 'struct bpf_timer'
                memset(dst + map->timer_off, 0, sizeof(struct bpf_timer));
                                                              ^
include/linux/bpf.h:225:10: error: invalid application of 'sizeof' to an incomplete type 'struct bpf_timer'
                t_sz = sizeof(struct bpf_timer);
                       ^     ~~~~~~~~~~~~~~~~~~
include/linux/bpf.h:225:24: note: forward declaration of 'struct bpf_timer'
                t_sz = sizeof(struct bpf_timer);
                                     ^
In file included from /virtual/main.c:17:
In file included from include/net/sock.h:59:
include/linux/filter.h:1518:44: error: use of undeclared identifier 'BPF_F_BROADCAST'; did you mean 'IFF_BROADCAST'?
        if (unlikely(!ri->tgt_value) && !(flags & BPF_F_BROADCAST)) {
                                                  ^~~~~~~~~~~~~~~
                                                  IFF_BROADCAST
include/uapi/linux/if.h:86:2: note: 'IFF_BROADCAST' declared here
        IFF_BROADCAST                   = 1<<1,  /* volatile */
        ^
In file included from /virtual/main.c:17:
In file included from include/net/sock.h:59:
include/linux/filter.h:1533:14: error: use of undeclared identifier 'BPF_F_BROADCAST'; did you mean 'IFF_BROADCAST'?
        if (flags & BPF_F_BROADCAST) {
                    ^~~~~~~~~~~~~~~
                    IFF_BROADCAST
include/uapi/linux/if.h:86:2: note: 'IFF_BROADCAST' declared here
        IFF_BROADCAST                   = 1<<1,  /* volatile */
        ^
3 warnings and 4 errors generated.
Traceback (most recent call last):
  File "/opt/bcc/bcc/tools/tcptracer.py", line 637, in <module>
    b = BPF(text=bpf_text)
  File "/usr/lib/python3/dist-packages/bcc/__init__.py", line 364, in __init__
    raise Exception("Failed to compile BPF module %s" % (src_file or "<text>"))
Exception: Failed to compile BPF module <text>

So basically both installations break the same way, probably for the same reason.

My software versions:

• linux-headers-5.15.0-30
• Kernel 5.15.0-30-generic
• llvm-11
• gcc version 11.2.0 (Ubuntu 11.2.0-19ubuntu1)
• Python 3.10.4 (main, Apr 2 2022, 09:04:19) [GCC 11.2.0] on linux

Thanks in advance!

[megastallman]

Same issue found on Canonical Launchpad - https://bugs.launchpad.net/ubuntu/+source/bpfcc/+bug/1969626
Confirmed, but not resolved. Probably the new kernel api has diverged too much.

[piax93]

If it is of interest, I worked around this re-defining the couple of things causing trouble (Yelp/pidtree-bcc#82). It's likely not the most solid of approaches, but at least gets things compiling and working for the time being.

[DavadDi]

I compiled version 0.24 using source code on ubuntu22.04 version and it works fine. The 0.18 version installed using apt install bpfcc-tools has the above compile error.

[megastallman]

Thanks guys! Finally it works, I've build from 0.25-release.

[megastallman]

Just a little cheatsheet for ubuntu.

apt purge bpfcc-tools libbpfcc python3-bpfcc
wget https://github.com/iovisor/bcc/releases/download/v0.25.0/bcc-src-with-submodule.tar.gz
tar xf bcc-src-with-submodule.tar.gz
cd bcc/
apt install -y python-is-python3
apt install -y bison build-essential cmake flex git libedit-dev   libllvm11 llvm-11-dev libclang-11-dev zlib1g-dev libelf-dev libfl-dev python3-distutils
apt install -y checkinstall
mkdir build
cd build/
cmake -DCMAKE_INSTALL_PREFIX=/usr -DPYTHON_CMD=python3 ..
make
checkinstall

[xjqr]

cmake error，what should i do？
-- Found LLVM: /usr/lib/llvm-12/include 12.0.1 (Use LLVM_ROOT envronment variable for another version of LLVM)
CMake Error at CMakeLists.txt:139 (message):
Unable to find clang libraries

[libf-de]

cmake error，what should i do？ -- Found LLVM: /usr/lib/llvm-12/include 12.0.1 (Use LLVM_ROOT envronment variable for another version of LLVM) CMake Error at CMakeLists.txt:139 (message): Unable to find clang libraries

sudo apt install libclang-12-dev

[megastallman]

I had that problem too, with different clang, llvm, libclang versions. To fix it i've removed all of them and installed the latest ones.

[Tejas-MD]

cmake error，what should i do？ -- Found LLVM: /usr/lib/llvm-12/include 12.0.1 (Use LLVM_ROOT envronment variable for another version of LLVM) CMake Error at CMakeLists.txt:139 (message): Unable to find clang libraries

sudo apt install libclang-12-dev

For anyone who struggled like me despite running sudo apt install libclang-12-dev , make sure you also do :

export LLVM_ROOT="/usr/lib/llvm-12/include"
cd ~/bcc/build
rm -rf *  # Caution: This will delete all files in the current directory, make sure you're in the build directory!'
sudo -E cmake -DCMAKE_INSTALL_PREFIX=/usr -DPYTHON_CMD=python3 ..

repeat the rm and cmake steps until it works.

Full Updated list of commands:

sudo apt purge bpfcc-tools libbpfcc python3-bpfcc
wget https://github.com/iovisor/bcc/releases/download/v0.25.0/bcc-src-with-submodule.tar.gz
tar xf bcc-src-with-submodule.tar.gz
cd bcc/
sudo apt install -y python-is-python3
sudo apt install -y bison build-essential cmake flex git libedit-dev   libllvm11 llvm-11-dev libclang-11-dev zlib1g-dev libelf-dev libfl-dev python3-distutils
apt install -y checkinstall
mkdir build
cd build/
sudo apt-get install llvm-12 llvm-12-dev clang-12 libclang-12-dev
export LLVM_ROOT="/usr/lib/llvm-12/include"

#clean and rebuild: 
/bcc/build/bcc
cd ~/bcc/build
~/bcc/build

# if it still fails, keep doing rm and rerunning 
sudo rm -rf *  # Caution: This will delete all files in the current directory!
cmake -DCMAKE_INSTALL_PREFIX=/usr -DPYTHON_CMD=python3 ..

make
sudo checkinstall

Also just found out the cause of my error: it was because of running
$ sudo apt install bpftrace