Build from code and run is not working in a VM with kernel version 5.xxx-generic IF hypervisor (host) has a bigger kernel version like 6.xxx-generic

[denerkr]

I'm following the tutorial here https://loxilb-io.github.io/loxilbdocs/run/, but I had no success to install the loxilb returning the error bellow:

[TrekkieCoder]

The issue is fixed. Please try the following -

cd loxilb-ebpf/
git pull origin main
cd ..
git pull
make

Further if you want to run loxilb in the same build system, you would need to install loxilb loader tools as follows -

curl -sfL https://github.com/loxilb-io/tools/raw/main/loader/install.sh | sh -

[TrekkieCoder]

Meanwhile, we also added support for Ubuntu 24.04 build environment as well. Feel free to have a look.

[vincentmli]

Meanwhile, we also added support for Ubuntu 24.04 build environment as well. Feel free to have a look.

@TrekkieCoder
does Ubuntu 24.04 ships libbpf > 1 version? looks loxilb-ebpf still depends on libbpf 0.8, I could build loxilb but it fails to run because my OS has libbpf > 1

[root@bpfire-3 lib]# /tmp/loxilb 
loxilb start
/tmp/loxilb: symbol lookup error: /tmp/loxilb: undefined symbol: bpf_get_link_xdp_id, version LIBBPF_0.0.2

I think bpf_get_link_xdp_id is deprecated in libbpf > 1 if I recall correct.

[TrekkieCoder]

Yes currently loxilb depends on libbpf 0.8. But currently you can -

cd loxilb-ebpf/libbpf/src
sudo make install

It will install libbpf 0.8 but it might overwrite other libbpf versions already installed. We will statically link libbpf dependency in future. bpf_get_link_xdp_id is deprecated even in 0.8 but porting to newer versions of libbpf is a work in progress.

[vincentmli]

Thanks! I can't install libbpf 0.8 because my other bpf program depending on libbpf > 1. look forward to have static link or porting to newer libbpf version

[UltraInstinct14]

#660 enables static linking. Closing this as completed.

[denerkr]

Hi @TrekkieCoder, The first part of the installation passed but still with some erros and It's not working the "sudo cp api/certification/server.* /opt/loxilb/cert/" saying "No such file or directory" and "sudo ./loxilb" part on tutorial for next steps:

What I did:

1. Created a new VM with ubuntu server from scratch and update/upgrade it;
2. Install latest go version on it;
3. Followed again the tutorial here: https://loxilb-io.github.io/loxilbdocs/run/ from the part "Install standard packages";
4. See bellow all the commands I did and the OUTPUT:

OUTPUT_loxilb_instalation.log

[TrekkieCoder]

I think this step got missed -

curl -sfL https://github.com/loxilb-io/tools/raw/main/loader/install.sh | sh -

[denerkr]

@TrekkieCoder, It still didn't solve the problem. Bellow are the steps I did:

1. Executed the command "curl -sfL https://github.com/loxilb-io/tools/raw/main/loader/install.sh | sh -" and it seems that worked, but please check bellow the output because it contain some erros:
   OUTPUT_curl_loxilb-io.log

2. performed the commands:

3. cd loxilb

4. ./loxilb-ebpf/utils/mkllb_bpffs.sh

5. make

6. cd loxilb-ebpf/libbpf/src

7. sudo make install

8. cd -

9. sudo mkdir -p /opt/loxilb/cert/

10. sudo cp api/certification/server.* /opt/loxilb/cert/ (Returning errors)

11. sudo ./loxilb (Returning errors)

See bellow output for the commands from step 2:
OUTPUT_loxilb_instalation_after_curl.log

[TrekkieCoder]

Thanks for the logs. I too spawned a new host (x86-64), installed Ubuntu 22.04.4 Kernel 6.5.0-28. Roughly followed the steps as per this link which is almost same as you did. It ran just fine.

sudo apt-get update && sudo apt-get install git 
wget https://go.dev/dl/go1.22.0.linux-amd64.tar.gz && sudo tar -xzf go1.22.0.linux-amd64.tar.gz --directory /usr/local/
export PATH="${PATH}:/usr/local/go/bin"
sudo apt install -y clang llvm libelf-dev gcc-multilib libpcap-dev vim net-tools linux-tools-$(uname -r) elfutils dwarves git libbsd-dev bridge-utils wget unzip build-essential bison flex iproute2
sudo apt-get install -y curl
curl -sfL https://github.com/loxilb-io/tools/raw/main/loader/install.sh | sh -
git clone --recurse-submodules https://github.com/loxilb-io/loxilb.git
cd loxilb
./loxilb-ebpf/utils/mkllb_bpffs.sh
make
sudo ./loxilb

Attaching logs -
loxilb-new-u22.log

Finally, I ran latest loxilb docker in the same box. That also went fine.

$ sudo docker run -u root --cap-add SYS_ADMIN   --restart unless-stopped --privileged -dit -v /dev/log:/dev/log --name loxilb ghcr.io/loxilb-io/loxilb:latest
$ sudo docker ps
CONTAINER ID   IMAGE                             COMMAND                  CREATED              STATUS              PORTS                            NAMES
faddd7c80632   ghcr.io/loxilb-io/loxilb:latest   "/root/loxilb-io/lox…"   About a minute ago   Up About a minute   3784/tcp, 11111/tcp, 22222/tcp   loxilb

Further, loxilb build/run is validated in ubuntu22.04 among others as well. Please check logs here

Kind of perplexed why it is not working for you.

[denerkr]

Hi @TrekkieCoder, I don't know but I tried here again and the command "sudo ./loxilb" fails. Can you provide more information about the VM setup you did? because I'm using here the VirtualBox (Version 7.0.10 r158379 (Qt5.15.3)) and my VM has 4 processors, 6gb of memory... bellow screenshot has full details about my VM:

I'm supposing it's an inconpatibility with virtualbox, what do you think?

[TrekkieCoder]

I have tried in VirtualBox 6.1.50_Ubuntu r161033 (Qt5.12.8) and resources assigned seem good enough. My hypervisor (base system) is however Ubuntu 20.04.

Can you please double confirm your system's kernel version (uname -a) and clang version (clang -v) ?

[denerkr]

Hi @TrekkieCoder, my hypervisor (base system) is Ubuntu 22.04, and kernel version 6.5.0-28-generic without clang:

My VM on virtual box has kernel version 5.15.0-105-generic and clang version 14.0.0-1ubuntu1.1:

[TrekkieCoder]

I tried the exact same configuration as yours but could not replicate your error:

It would be great if we are able to solve your issue. If it is fine with you, please join loxilb slack channel. We can discuss this further.

[denerkr]

Hi @TrekkieCoder, I find out that my issue is related the kernel version. So, IF the hypervisor (base system) has a kernel version 6.5.0-28-generic and the VM a lower kernel version like it was 5.15.0-105-generic the loxilb won't work.

What I did here to make it working now was that I first tried directly on my computer with kernel version 6.5.0-28-generic and worked/is working perfectly.

After that I created a new VM with ubuntu 24.04 that has the kernel version 6.8.0-31-generic and boom it worked as well:

For me it seems that the bug here is "loxilb do not work in a VM with lower kernel version then the hypervisor".

[TrekkieCoder]

Thanks for digging into this. Will try to check how interaction between hypervisor host and VM is affecting this.

[TrekkieCoder]

It is currently not reproducible anymore. Kindly reopen if problem persists with latest loxilb version.