Skip to content

Commit

Permalink
docs: document kernel version requirements
Browse files Browse the repository at this point in the history 
Also move the BTF FAQ entry just after the kernel version and add
recommended Linux kernel configuration options.

Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Djalal Harouni <tixxdz@gmail.com>
  • Loading branch information
@tixxdz
tixxdz committed Sep 8, 2023
1 parent daaaa34 commit eb0c09c
Show file tree
Hide file tree
Showing 2 changed files with 102 additions and 42 deletions.
141 changes: 100 additions & 41 deletions docs/content/en/docs/faq/_index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,106 @@ icon: "faq"
description: "List of frequently asked questions"
---

### What is the minimum Linux kernel version to run Tetragon?

Tetragon needs Linux kernel version 4.19 or greater.

We currently run tests on stable long-term support kernels 4.19, 5.4, 5.10,
5.15 and bpf-next, see [this test workflow](https://github.com/cilium/tetragon/actions/workflows/vmtests.yml)
for up to date information. Not all Tetragon features work with older kernel
versions. BPF evolves rapidly and we recommend you use the most recent stable
kernel possible to get the most out of Tetragon's features.

Note that Tetragon also needs [BTF support]({{< ref "/docs/faq/_index.md#tetragon-failed-to-start-complaining-about-a-missing-btf-file">}})
which might take some work on older kernels.

<details><summary>See the recommended Linux kernel configuration options</summary>
<p>

```
# CORE BPF
CONFIG_BPF
CONFIG_BPF_JIT
CONFIG_BPF_JIT_DEFAULT_ON
CONFIG_BPF_EVENTS
CONFIG_BPF_SYSCALL
CONFIG_HAVE_BPF_JIT
CONFIG_HAVE_EBPF_JIT
CONFIG_FTRACE_SYSCALLS
# BTF
CONFIG_DEBUG_INFO_BTF
CONFIG_DEBUG_INFO_BTF_MODULES
# Enforcement
CONFIG_BPF_KPROBE_OVERRIDE
# CGROUP and Process tracking
CONFIG_CGROUPS=y Control Group support
CONFIG_MEMCG=y Memory Control group
CONFIG_BLK_CGROUP=y Generic block IO controller
CONFIG_CGROUP_SCHED=y
CONFIG_CGROUP_PIDS=y Process Control group
CONFIG_CGROUP_FREEZER=y Freeze and unfreeze tasks controller
CONFIG_CPUSETS=y Manage CPUSETs
CONFIG_PROC_PID_CPUSET=y
CONFIG_CGROUP_DEVICE=Y Devices Control group
CONFIG_CGROUP_CPUACCT=y CPU accouting controller
CONFIG_CGROUP_PERF=y
CONFIG_CGROUP_BPF=y Attach eBPF programs to a cgroup
CGROUP_FAVOR_DYNMODS=y (optional) >= 6.0
Reduces the latencies of dynamic cgroup modifications at the
cost of making hot path operations such as forks and exits
more expensive.
Platforms with frequent cgroup migrations could enable this
option as a potential alleviation for pod and containers
association issues.
```

</p>
</details>

### Tetragon failed to start complaining about a missing BTF file

You might have encountered the following issues:
```
level=info msg="BTF discovery: default kernel btf file does not exist" btf-file=/sys/kernel/btf/vmlinux
level=info msg="BTF discovery: candidate btf file does not exist" btf-file=/var/lib/tetragon/metadata/vmlinux-5.15.49-linuxkit
level=info msg="BTF discovery: candidate btf file does not exist" btf-file=/var/lib/tetragon/btf
[...]
level=fatal msg="Failed to start tetragon" error="tetragon, aborting kernel autodiscovery failed: Kernel version \"5.15.49-linuxkit\" BTF search failed kernel is not included in supported list. Please check Tetragon requirements documentation, then use --btf option to specify BTF path and/or '--kernel' to specify kernel version"
```

Tetragon needs BTF ([BPF Type Format](https://www.kernel.org/doc/html/latest/bpf/btf.html))
support to load its BPF programs using CO-RE ([Compile Once - Run Everywhere](https://nakryiko.com/posts/bpf-core-reference-guide/)).
In brief, CO-RE is useful to load BPF programs that have been compiled on a
different kernel version than the target kernel. Indeed, kernel structures
change between versions and BPF programs need to access fields in them. So
CO-RE uses the [BTF file of the kernel](https://nakryiko.com/posts/btf-dedup/)
in which you are loading the BPF program to know the differences between the
struct and patch the fields offset in the accessed structures. CO-RE allows
portability of the BPF programs but requires a kernel with BTF enabled.

Most of the [common Linux distributions](https://github.com/libbpf/libbpf#bpf-co-re-compile-once--run-everywhere)
now ship with BTF enabled and do not require any extra work, this is kernel
option `CONFIG_DEBUG_INFO_BTF=y`. To check if BTF is enabled on your Linux
system and see the BTF data file of your kernel, the standard location is
`/sys/kernel/btf/vmlinux`. By default, Tetragon will look for this file (this
is the first line in the log output above).

If your kernel does not support BTF you can:
- Retrieve the BTF file for your kernel version from an external source.
- Build the BTF file from your kernel debug symbols. You will need pahole to
add BTF metadata to the debugging symbols and LLVM minimize the medata size.
- Rebuild your kernel with `CONFIG_DEBUG_INFO_BTF` to `y`.

Tetragon will also look into `/var/lib/tetragon/btf` for the `vmlinux` file
(this is the third line in the log output above). Or you can use the `--btf`
flag to directly indicate Tetragon where to locate the file.

If you encounter this issue while using Docker Desktop on macOS, please refer
to [can I run Tetragon on Mac computers](#can-i-run-tetragon-on-mac-computers).

### Can I install and use Tetragon in standalone mode (outside of k8s)?

Yes! You can run `make` to generate standalone binaries and run them directly.
Expand Down Expand Up @@ -91,44 +191,3 @@ If you encounter the issue under Linux:
persistently change the value of the CC go environment variable you can use
`go env -w "CC=clang"`.

### Tetragon failed to start complaining about a missing BTF file

You might have encountered the following issues:
```
level=info msg="BTF discovery: default kernel btf file does not exist" btf-file=/sys/kernel/btf/vmlinux
level=info msg="BTF discovery: candidate btf file does not exist" btf-file=/var/lib/tetragon/metadata/vmlinux-5.15.49-linuxkit
level=info msg="BTF discovery: candidate btf file does not exist" btf-file=/var/lib/tetragon/btf
[...]
level=fatal msg="Failed to start tetragon" error="tetragon, aborting kernel autodiscovery failed: Kernel version \"5.15.49-linuxkit\" BTF search failed kernel is not included in supported list. Please check Tetragon requirements documentation, then use --btf option to specify BTF path and/or '--kernel' to specify kernel version"
```

Tetragon needs BTF ([BPF Type Format](https://www.kernel.org/doc/html/latest/bpf/btf.html))
support to load its BPF programs using CO-RE ([Compile Once - Run Everywhere](https://nakryiko.com/posts/bpf-core-reference-guide/)).
In brief, CO-RE is useful to load BPF programs that have been compiled on a
different kernel version than the target kernel. Indeed, kernel structures
change between versions and BPF programs need to access fields in them. So
CO-RE uses the [BTF file of the kernel](https://nakryiko.com/posts/btf-dedup/)
in which you are loading the BPF program to know the differences between the
struct and patch the fields offset in the accessed structures. CO-RE allows
portability of the BPF programs but requires a kernel with BTF enabled.

Most of the [common Linux distributions](https://github.com/libbpf/libbpf#bpf-co-re-compile-once--run-everywhere)
now ship with BTF enabled and do not require any extra work, this is kernel
option `CONFIG_DEBUG_INFO_BTF=y`. To check if BTF is enabled on your Linux
system and see the BTF data file of your kernel, the standard location is
`/sys/kernel/btf/vmlinux`. By default, Tetragon will look for this file (this
is the first line in the log output above).

If your kernel does not support BTF you can:
- Retrieve the BTF file for your kernel version from an external source.
- Build the BTF file from your kernel debug symbols. You will need pahole to
add BTF metadata to the debugging symbols and LLVM minimize the medata size.
- Rebuild your kernel with `CONFIG_DEBUG_INFO_BTF` to `y`.

Tetragon will also look into `/var/lib/tetragon/btf` for the `vmlinux` file
(this is the third line in the log output above). Or you can use the `--btf`
flag to directly indicate Tetragon where to locate the file.

If you encounter this issue while using Docker Desktop on macOS, please refer
to [can I run Tetragon on Mac computers](#can-i-run-tetragon-on-mac-computers).

3 changes: 2 additions & 1 deletion docs/content/en/docs/getting-started/try-tetragon-linux.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ time.
This guide has been tested on Ubuntu 22.04 and 22.10 with respectively kernel
`5.15.0` and `5.19.0` on amd64 and arm64 but
[any recent distribution](https://github.com/libbpf/libbpf#bpf-co-re-compile-once--run-everywhere)
shipping with a relatively recent kernel should work.
shipping with a relatively recent kernel should work. See the FAQ for further details on
the [recommended kernel versions]({{< ref "/docs/faq#what-is-the-minimum-linux-kernel-version-to-run-tetragon" >}}).

Note that you cannot run Tetragon using Docker Desktop on macOS because of a
limitation of the Docker Desktop Linux virtual machine. Learn more about this issue
Expand Down

0 comments on commit eb0c09c

Please sign in to comment.