50ff9c40c4
* Add BPF 101 README file Added a README document to introduce and guide new users on BPF (Berkeley Packet Filter) code writing. The file includes useful links, described the environment setup and outlined the supported architectures. This will provide the necessary resources and steps on how to use and develop BPF codes. * Improve README with BPF code structure and usage details Extended the README document in the BPF directory to include a comprehensive explanation on the structure of BPF code and its usage aspects. This elaboration covers the distinction between BPF program and user space code, their locations in the repository, certain limitations, their compilation process, as well as compatibility issues concerning the `libbpfgo` library. This addition is expected to provide clearer guidance for developers working with BPF code in this project. * Add BPF helpers documentation link and license details to README Added a link to the BPF helpers documentation in the README for easy reference. Also included detailed information about the license requirement for BPF programs in the kernel. This is aimed to inform developers about the GPL default setting for the license field in ELF files, and the limitations imposed by using the `Dual BSD/GPL` license in Teleport. Instructions for enabling all BPF features have also been provided. * Add logging section to BPF README.md Included a new section in the BPF README.md file detailing how to enable logging within BPF programs. This change provides clear instructions and a useful resource link to developers on how to set the license to 'GPL' for BPF programs, and how to use the 'bpf_printk' helper for logging messages to the kernel log. Adding this documentation promotes a better understanding of BPF logging functions and their license requirements. * Enhance BPF README with BPF-user space communication info Expanded the BPF README.md file to include a section about communication between BPF programs and user space via maps. This addition provides a valuable explanation and example of how both user space and BPF programs can access maps. This highlights the interaction between BPF and user space, educating developers on properly utilizing BPF-helper for map access in BPF programs and 'libbpfgo' library in user space. * Apply suggestions from code review Co-authored-by: Alan Parra <alan.parra@goteleport.com> * Update BPF README with additional details and instructions. This commit expands on the instructions for setting up BPF, specifically clarifying that 'clang' can be installed through the 'apt install clang' command. A section detailing the requirements for libbpf installation has also been added for further clarity. Moreover, the BPF utilization in Teleport has been further explained, highlighting its usage for enhanced session recording and restricted networking. Lastly, the instructions for enabling LSM hooks on Ubuntu systems have been provided. This commit aims to provide comprehensive and updated information for configuring and understanding BPF usage. * Apply suggestions from code review Co-authored-by: Alan Parra <alan.parra@goteleport.com> * Update bpf/README.md Co-authored-by: Anton Miniailo <anton@goteleport.com> --------- Co-authored-by: Alan Parra <alan.parra@goteleport.com> Co-authored-by: Anton Miniailo <anton@goteleport.com> |
||
|---|---|---|
| .. | ||
| arm64 | ||
| enhancedrecording | ||
| restrictedsession | ||
| x86 | ||
| helpers.h | ||
| README.md | ||
| vmlinux.h | ||
BPF 101
If you're reading this, you were probably tasked with writing a BPF code. This document will help you get started.
Before you start here are some useful links:
- https://docs.kernel.org/bpf/ - BPF documentation
- https://github.com/aquasecurity/libbpfgo - Go bindings for libbpf
- https://github.com/iovisor/bcc - BPF Compiler Collection (a lot of examples)
- https://github.com/cilium/cilium - BPF-based networking, security, and observability (more examples)
- https://man7.org/linux/man-pages/man7/bpf-helpers.7.html - BPF helpers documentation
Environment setup:
We support BPFs only on ARM64 and x86_64 architectures.
- VM with Ubuntu 20.04+ (or any other Linux distro with the kernel 5.8+ - Docker doesn't work for many BPF features)
- Install
clang(required for compiling BPF code) -apt install clangshould be enough - Install
libbpf(required for loading BPF code) - get the required version from our Docker file
You can also use our Dockerfile to build the environment:
make -C build.assets release-centos7
BPF code structure
BPF code can be divided into two parts:
- BPF program - the actual BPF code that will be loaded into the kernel
- User space code - the code that will load the BPF program into the kernel
BPF programs are located in bpf directory in our repository. They are C like programs with some limitations.
The most important limitation is that you can't use any system calls in BPF programs. You can use only BPF helpers.
The BPF programs are compiled using clang 10+, GCC is not supported.
User space code is located in lib/bpf directory. It's written in Go and uses the aquasecurity/libbpfgo library to load BPF programs into the kernel.
aquasecurity/libbpfgo is a Go wrapper around the libbpf library. It's a low-level library that allows you to load BPF programs into the kernel.
Note: aquasecurity/libbpfgo is not backwards compatible, you need to use the
exact libbpf version it requires. For that reason aquasecurity/libbpfgo
doesn't use semantic versioning, instead the tags have the format
v0.4.5-libbpf-1.0.1. The first part is the aquasecurity/libbpfgo version,
the second part is the libbpf version. Using the wrong version of libbpf
results in a runtime/compilation error.
libbpf installation
aquasecurity/libbpfgo requires libbpf to be installed on the system. The most up-to-date installation instructions can be found
in our Docker file: 2fd8f75e38/build.assets/Dockerfile-centos7 (L108-L131)
Remember that the library must be installed in /opt directory not in /usr/local or /usr.
BPF license
BPF programs are compiled into ELF files. ELF files have a license field that is used to verify that the BPF program
is allowed to run in the kernel. The license field is set to GPL by default. Teleport uses Dual BSD/GPL which
disables some BPF features (like logging). To enable all BPF features set the license to GPL in the BPF program
and revert back before merging the code.
Logging
BPF programs can log messages to the kernel log. To enable logging, you need to set the license to GPL. Then you
can use bpf_printk helper to log messages. The messages will be logged to the kernel log. You can get the messages
from /sys/kernel/debug/tracing/trace_pipe. Here is the best explanation that I found so far https://nakryiko.com/posts/bpf-tips-printk/.
Communication between BPF programs and user space
BPF programs can communicate with user space using maps. Maps are key-value stores that can be accessed from both
BPF programs and user space. BPF programs can only access maps using BPF helpers. User space can access maps using
aquasecurity/libbpfgo library. Maps are defined in BPF programs and can be referenced by name from user space code.
Example:
- BPF:
2fd8f75e38/bpf/enhancedrecording/command.bpf.c (L18) - User space:
2fd8f75e38/lib/bpf/common_linux.go (L40-L51)
BPF in Teleport
Teleport uses BPF to implement enhanced session recording and restricted networking. Both features work only on Linux with the kernel 5.8+. Enhanced session recording records all:
- exec family system calls
- open family system calls
- network connections
All events are recorded in the audit log. See https://goteleport.com/docs/server-access/guides/bpf-session-recording/.
Restricted networking allows you to restrict network access for users. It's implemented by using LSM hooks and BPF programs ( see https://goteleport.com/docs/server-access/guides/restricted-session/). On ubuntu systems LSM hooks are not enabled in some versions. Here are the instructions on how to enable them https://github.com/gravitational/teleport/issues/8089#issuecomment-924990678.