diff --git a/docs/content/en/docs/faq/_index.md b/docs/content/en/docs/faq/_index.md index 324b18ae4b0..8826a8731c9 100644 --- a/docs/content/en/docs/faq/_index.md +++ b/docs/content/en/docs/faq/_index.md @@ -5,6 +5,106 @@ icon: "faq" description: "List of frequently asked questions" --- +### What is the minimum Linux kernel version to run Tetragon? + +Tetragon needs Linux kernel version 4.19 or greater. + +We currently run tests on stable long-term support kernels 4.19, 5.4, 5.10, +5.15 and bpf-next, see [this test workflow](https://github.com/cilium/tetragon/actions/workflows/vmtests.yml) +for up to date information. Not all Tetragon features work with older kernel +versions. BPF evolves rapidly and we recommend you use the most recent stable +kernel possible to get the most out of Tetragon's features. + +Note that Tetragon also needs [BTF support]({{< ref "/docs/faq/_index.md#tetragon-failed-to-start-complaining-about-a-missing-btf-file">}}) +which might take some work on older kernels. + +
See the recommended Linux kernel configuration options +

+ +``` + # CORE BPF + CONFIG_BPF + CONFIG_BPF_JIT + CONFIG_BPF_JIT_DEFAULT_ON + CONFIG_BPF_EVENTS + CONFIG_BPF_SYSCALL + CONFIG_HAVE_BPF_JIT + CONFIG_HAVE_EBPF_JIT + CONFIG_FTRACE_SYSCALLS + + # BTF + CONFIG_DEBUG_INFO_BTF + CONFIG_DEBUG_INFO_BTF_MODULES + + # Enforcement + CONFIG_BPF_KPROBE_OVERRIDE + + # CGROUP and Process tracking + CONFIG_CGROUPS=y Control Group support + CONFIG_MEMCG=y Memory Control group + CONFIG_BLK_CGROUP=y Generic block IO controller + CONFIG_CGROUP_SCHED=y + CONFIG_CGROUP_PIDS=y Process Control group + CONFIG_CGROUP_FREEZER=y Freeze and unfreeze tasks controller + CONFIG_CPUSETS=y Manage CPUSETs + CONFIG_PROC_PID_CPUSET=y + CONFIG_CGROUP_DEVICE=Y Devices Control group + CONFIG_CGROUP_CPUACCT=y CPU accouting controller + CONFIG_CGROUP_PERF=y + CONFIG_CGROUP_BPF=y Attach eBPF programs to a cgroup + CGROUP_FAVOR_DYNMODS=y (optional) >= 6.0 + Reduces the latencies of dynamic cgroup modifications at the + cost of making hot path operations such as forks and exits + more expensive. + Platforms with frequent cgroup migrations could enable this + option as a potential alleviation for pod and containers + association issues. +``` + +

+
+ +### Tetragon failed to start complaining about a missing BTF file + +You might have encountered the following issues: +``` +level=info msg="BTF discovery: default kernel btf file does not exist" btf-file=/sys/kernel/btf/vmlinux +level=info msg="BTF discovery: candidate btf file does not exist" btf-file=/var/lib/tetragon/metadata/vmlinux-5.15.49-linuxkit +level=info msg="BTF discovery: candidate btf file does not exist" btf-file=/var/lib/tetragon/btf +[...] +level=fatal msg="Failed to start tetragon" error="tetragon, aborting kernel autodiscovery failed: Kernel version \"5.15.49-linuxkit\" BTF search failed kernel is not included in supported list. Please check Tetragon requirements documentation, then use --btf option to specify BTF path and/or '--kernel' to specify kernel version" +``` + +Tetragon needs BTF ([BPF Type Format](https://www.kernel.org/doc/html/latest/bpf/btf.html)) +support to load its BPF programs using CO-RE ([Compile Once - Run Everywhere](https://nakryiko.com/posts/bpf-core-reference-guide/)). +In brief, CO-RE is useful to load BPF programs that have been compiled on a +different kernel version than the target kernel. Indeed, kernel structures +change between versions and BPF programs need to access fields in them. So +CO-RE uses the [BTF file of the kernel](https://nakryiko.com/posts/btf-dedup/) +in which you are loading the BPF program to know the differences between the +struct and patch the fields offset in the accessed structures. CO-RE allows +portability of the BPF programs but requires a kernel with BTF enabled. + +Most of the [common Linux distributions](https://github.com/libbpf/libbpf#bpf-co-re-compile-once--run-everywhere) +now ship with BTF enabled and do not require any extra work, this is kernel +option `CONFIG_DEBUG_INFO_BTF=y`. To check if BTF is enabled on your Linux +system and see the BTF data file of your kernel, the standard location is +`/sys/kernel/btf/vmlinux`. By default, Tetragon will look for this file (this +is the first line in the log output above). + +If your kernel does not support BTF you can: +- Retrieve the BTF file for your kernel version from an external source. +- Build the BTF file from your kernel debug symbols. You will need pahole to + add BTF metadata to the debugging symbols and LLVM minimize the medata size. +- Rebuild your kernel with `CONFIG_DEBUG_INFO_BTF` to `y`. + +Tetragon will also look into `/var/lib/tetragon/btf` for the `vmlinux` file +(this is the third line in the log output above). Or you can use the `--btf` +flag to directly indicate Tetragon where to locate the file. + +If you encounter this issue while using Docker Desktop on macOS, please refer +to [can I run Tetragon on Mac computers](#can-i-run-tetragon-on-mac-computers). + ### Can I install and use Tetragon in standalone mode (outside of k8s)? Yes! You can run `make` to generate standalone binaries and run them directly. @@ -91,44 +191,3 @@ If you encounter the issue under Linux: persistently change the value of the CC go environment variable you can use `go env -w "CC=clang"`. -### Tetragon failed to start complaining about a missing BTF file - -You might have encountered the following issues: -``` -level=info msg="BTF discovery: default kernel btf file does not exist" btf-file=/sys/kernel/btf/vmlinux -level=info msg="BTF discovery: candidate btf file does not exist" btf-file=/var/lib/tetragon/metadata/vmlinux-5.15.49-linuxkit -level=info msg="BTF discovery: candidate btf file does not exist" btf-file=/var/lib/tetragon/btf -[...] -level=fatal msg="Failed to start tetragon" error="tetragon, aborting kernel autodiscovery failed: Kernel version \"5.15.49-linuxkit\" BTF search failed kernel is not included in supported list. Please check Tetragon requirements documentation, then use --btf option to specify BTF path and/or '--kernel' to specify kernel version" -``` - -Tetragon needs BTF ([BPF Type Format](https://www.kernel.org/doc/html/latest/bpf/btf.html)) -support to load its BPF programs using CO-RE ([Compile Once - Run Everywhere](https://nakryiko.com/posts/bpf-core-reference-guide/)). -In brief, CO-RE is useful to load BPF programs that have been compiled on a -different kernel version than the target kernel. Indeed, kernel structures -change between versions and BPF programs need to access fields in them. So -CO-RE uses the [BTF file of the kernel](https://nakryiko.com/posts/btf-dedup/) -in which you are loading the BPF program to know the differences between the -struct and patch the fields offset in the accessed structures. CO-RE allows -portability of the BPF programs but requires a kernel with BTF enabled. - -Most of the [common Linux distributions](https://github.com/libbpf/libbpf#bpf-co-re-compile-once--run-everywhere) -now ship with BTF enabled and do not require any extra work, this is kernel -option `CONFIG_DEBUG_INFO_BTF=y`. To check if BTF is enabled on your Linux -system and see the BTF data file of your kernel, the standard location is -`/sys/kernel/btf/vmlinux`. By default, Tetragon will look for this file (this -is the first line in the log output above). - -If your kernel does not support BTF you can: -- Retrieve the BTF file for your kernel version from an external source. -- Build the BTF file from your kernel debug symbols. You will need pahole to - add BTF metadata to the debugging symbols and LLVM minimize the medata size. -- Rebuild your kernel with `CONFIG_DEBUG_INFO_BTF` to `y`. - -Tetragon will also look into `/var/lib/tetragon/btf` for the `vmlinux` file -(this is the third line in the log output above). Or you can use the `--btf` -flag to directly indicate Tetragon where to locate the file. - -If you encounter this issue while using Docker Desktop on macOS, please refer -to [can I run Tetragon on Mac computers](#can-i-run-tetragon-on-mac-computers). - diff --git a/docs/content/en/docs/getting-started/try-tetragon-linux.md b/docs/content/en/docs/getting-started/try-tetragon-linux.md index 0136c6e59fe..fc8603ec779 100644 --- a/docs/content/en/docs/getting-started/try-tetragon-linux.md +++ b/docs/content/en/docs/getting-started/try-tetragon-linux.md @@ -16,7 +16,8 @@ time. This guide has been tested on Ubuntu 22.04 and 22.10 with respectively kernel `5.15.0` and `5.19.0` on amd64 and arm64 but [any recent distribution](https://github.com/libbpf/libbpf#bpf-co-re-compile-once--run-everywhere) -shipping with a relatively recent kernel should work. +shipping with a relatively recent kernel should work. See the FAQ for further details on +the [recommended kernel versions]({{< ref "/docs/faq#what-is-the-minimum-linux-kernel-version-to-run-tetragon" >}}). Note that you cannot run Tetragon using Docker Desktop on macOS because of a limitation of the Docker Desktop Linux virtual machine. Learn more about this issue