mirror of
https://github.com/torvalds/linux
synced 2024-10-15 15:59:15 +00:00
a4561f5afe
Real-world BPF applications keep growing in size. Medium-sized production application can easily have 50K+ verified instructions, and its line info section in .BTF.ext has more than 3K entries. When verifier emits log with log_level>=1, it annotates assembly code with matched original C source code. Currently it uses linear search over line info records to find a match. As complexity of BPF applications grows, this O(K * N) approach scales poorly. So, let's instead of linear O(N) search for line info record use faster equivalent O(log(N)) binary search algorithm. It's not a plain binary search, as we don't look for exact match. It's an upper bound search variant, looking for rightmost line info record that starts at or before given insn_off. Some unscientific measurements were done before and after this change. They were done in VM and fluctuate a bit, but overall the speed up is undeniable. BASELINE ======== File Program Duration (us) Insns -------------------------------- ---------------- ------------- ------ katran.bpf.o balancer_ingress 2497130 343552 pyperf600.bpf.linked3.o on_event 12389611 627288 strobelight_pyperf_libbpf.o on_py_event 387399 52445 -------------------------------- ---------------- ------------- ------ BINARY SEARCH ============= File Program Duration (us) Insns -------------------------------- ---------------- ------------- ------ katran.bpf.o balancer_ingress 2339312 343552 pyperf600.bpf.linked3.o on_event 5602203 627288 strobelight_pyperf_libbpf.o on_py_event 294761 52445 -------------------------------- ---------------- ------------- ------ While Katran's speed up is pretty modest (about 105ms, or 6%), for production pyperf BPF program (on_py_event) it's much greater already, going from 387ms down to 295ms (23% improvement). Looking at BPF selftests's biggest pyperf example, we can see even more dramatic improvement, shaving more than 50% of time, going from 12.3s down to 5.6s. Different amount of improvement is the function of overall amount of BPF assembly instructions in .bpf.o files (which contributes to how much line info records there will be and thus, on average, how much time linear search will take), among other things: $ llvm-objdump -d katran.bpf.o | wc -l 3863 $ llvm-objdump -d strobelight_pyperf_libbpf.o | wc -l 6997 $ llvm-objdump -d pyperf600.bpf.linked3.o | wc -l 87854 Granted, this only applies to debugging cases (e.g., using veristat, or failing verification in production), but seems worth doing to improve overall developer experience anyways. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/bpf/20240214002311.2197116-1-andrii@kernel.org |
||
|---|---|---|
| .. | ||
| preload | ||
| arraymap.c | ||
| bloom_filter.c | ||
| bpf_cgrp_storage.c | ||
| bpf_inode_storage.c | ||
| bpf_iter.c | ||
| bpf_local_storage.c | ||
| bpf_lru_list.c | ||
| bpf_lru_list.h | ||
| bpf_lsm.c | ||
| bpf_struct_ops.c | ||
| bpf_task_storage.c | ||
| btf.c | ||
| cgroup.c | ||
| cgroup_iter.c | ||
| core.c | ||
| cpumap.c | ||
| cpumask.c | ||
| devmap.c | ||
| disasm.c | ||
| disasm.h | ||
| dispatcher.c | ||
| hashtab.c | ||
| helpers.c | ||
| inode.c | ||
| Kconfig | ||
| link_iter.c | ||
| local_storage.c | ||
| log.c | ||
| lpm_trie.c | ||
| Makefile | ||
| map_in_map.c | ||
| map_in_map.h | ||
| map_iter.c | ||
| memalloc.c | ||
| mmap_unlock_work.h | ||
| mprog.c | ||
| net_namespace.c | ||
| offload.c | ||
| percpu_freelist.c | ||
| percpu_freelist.h | ||
| prog_iter.c | ||
| queue_stack_maps.c | ||
| reuseport_array.c | ||
| ringbuf.c | ||
| stackmap.c | ||
| syscall.c | ||
| sysfs_btf.c | ||
| task_iter.c | ||
| tcx.c | ||
| tnum.c | ||
| token.c | ||
| trampoline.c | ||
| verifier.c | ||