linux/kernel/bpf
Dave Marchevsky 4ac4546821 bpf: Introduce task_vma open-coded iterator kfuncs
This patch adds kfuncs bpf_iter_task_vma_{new,next,destroy} which allow
creation and manipulation of struct bpf_iter_task_vma in open-coded
iterator style. BPF programs can use these kfuncs directly or through
bpf_for_each macro for natural-looking iteration of all task vmas.

The implementation borrows heavily from bpf_find_vma helper's locking -
differing only in that it holds the mmap_read lock for all iterations
while the helper only executes its provided callback on a maximum of 1
vma. Aside from locking, struct vma_iterator and vma_next do all the
heavy lifting.

A pointer to an inner data struct, struct bpf_iter_task_vma_data, is the
only field in struct bpf_iter_task_vma. This is because the inner data
struct contains a struct vma_iterator (not ptr), whose size is likely to
change under us. If bpf_iter_task_vma_kern contained vma_iterator directly
such a change would require change in opaque bpf_iter_task_vma struct's
size. So better to allocate vma_iterator using BPF allocator, and since
that alloc must already succeed, might as well allocate all iter fields,
thereby freezing struct bpf_iter_task_vma size.

Signed-off-by: Dave Marchevsky <davemarchevsky@fb.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20231013204426.1074286-4-davemarchevsky@fb.com
2023-10-13 15:48:58 -07:00
..
preload bpf: make preloaded map iterators to display map elements count 2023-07-06 12:42:25 -07:00
arraymap.c bpf: return long from bpf_map_ops funcs 2023-03-22 15:11:30 -07:00
bloom_filter.c bpf: Centralize permissions checks for all BPF map types 2023-06-19 14:04:04 +02:00
bpf_cgrp_storage.c bpf: Teach verifier that certain helpers accept NULL pointer. 2023-04-04 16:57:16 -07:00
bpf_inode_storage.c Networking changes for 6.4. 2023-04-26 16:07:23 -07:00
bpf_iter.c bpf: Don't explicitly emit BTF for struct btf_iter_num 2023-10-13 15:48:58 -07:00
bpf_local_storage.c bpf: bpf_sk_storage: Fix the missing uncharge in sk_omem_alloc 2023-09-06 11:08:14 +02:00
bpf_lru_list.c bpf: Address KCSAN report on bpf_lru_list 2023-05-12 12:01:03 -07:00
bpf_lru_list.h bpf: lru: Remove unused declaration bpf_lru_promote() 2023-08-08 17:21:42 -07:00
bpf_lsm.c bpf: Fix the kernel crash caused by bpf_setsockopt(). 2023-01-26 23:26:40 -08:00
bpf_struct_ops.c bpf: Charge modmem for struct_ops trampoline 2023-09-14 15:30:45 -07:00
bpf_struct_ops_types.h
bpf_task_storage.c bpf: Teach verifier that certain helpers accept NULL pointer. 2023-04-04 16:57:16 -07:00
btf.c bpf: Add bpf_sock_addr_set_sun_path() to allow writing unix sockaddr from bpf 2023-10-11 16:29:25 -07:00
cgroup.c bpf: Implement cgroup sockaddr hooks for unix sockets 2023-10-11 17:27:47 -07:00
cgroup_iter.c cgroup: bpf: use cgroup_lock()/cgroup_unlock() wrappers 2023-03-17 12:07:13 -10:00
core.c Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next 2023-09-17 15:12:06 +01:00
cpumap.c bpf, cpumask: Clean up bpf_cpu_map_entry directly in cpu_map_free 2023-08-21 15:21:16 -07:00
cpumask.c bpf: Convert bpf_cpumask to bpf_mem_cache_free_rcu. 2023-07-12 23:45:23 +02:00
devmap.c bpf, devmap: Remove unused dtab field from bpf_dtab_netdev 2023-07-31 18:26:08 -07:00
disasm.c bpf: change bpf_alu_sign_string and bpf_movsx_string to static 2023-08-04 16:15:50 -07:00
disasm.h
dispatcher.c bpf: Synchronize dispatcher update with bpf_dispatcher_xdp_func 2022-12-14 12:02:14 -08:00
hashtab.c bpf: populate the per-cpu insertions/deletions counters for hashmaps 2023-07-06 12:42:25 -07:00
helpers.c bpf: Introduce task_vma open-coded iterator kfuncs 2023-10-13 15:48:58 -07:00
inode.c bpf: convert to ctime accessor functions 2023-07-24 10:30:07 +02:00
Kconfig bpf: Add fd-based tcx multi-prog infra with link support 2023-07-19 10:07:27 -07:00
link_iter.c
local_storage.c cgroup changes for v6.4-rc1 2023-04-29 10:05:22 -07:00
log.c bpf: drop unnecessary user-triggerable WARN_ONCE in verifierl log 2023-05-16 22:34:50 -07:00
lpm_trie.c bpf: Centralize permissions checks for all BPF map types 2023-06-19 14:04:04 +02:00
Makefile bpf: Add fd-based tcx multi-prog infra with link support 2023-07-19 10:07:27 -07:00
map_in_map.c bpf: Fix elem_size not being set for inner maps 2023-06-02 16:22:12 -07:00
map_in_map.h
map_iter.c bpf: allow any program to use the bpf_map_sum_elem_count kfunc 2023-07-19 09:48:53 -07:00
memalloc.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2023-09-21 21:49:45 +02:00
mmap_unlock_work.h
mprog.c bpf: Fix mprog detachment for empty mprog entry 2023-08-04 09:35:39 -07:00
net_namespace.c
offload.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2023-09-21 21:49:45 +02:00
percpu_freelist.c bpf: Initialize same number of free nodes for each pcpu_freelist 2022-11-11 12:05:14 -08:00
percpu_freelist.h
prog_iter.c
queue_stack_maps.c bpf: Avoid deadlock when using queue and stack maps from NMI 2023-09-11 19:04:49 -07:00
reuseport_array.c bpf: Centralize permissions checks for all BPF map types 2023-06-19 14:04:04 +02:00
ringbuf.c bpf: Remove unnecessary ring buffer size check 2023-07-05 14:09:45 +02:00
stackmap.c bpf: Annotate struct bpf_stack_map with __counted_by 2023-10-06 23:44:35 +02:00
syscall.c bpf: Implement cgroup sockaddr hooks for unix sockets 2023-10-11 17:27:47 -07:00
sysfs_btf.c
task_iter.c bpf: Introduce task_vma open-coded iterator kfuncs 2023-10-13 15:48:58 -07:00
tcx.c tcx: Fix splat during dev unregister 2023-07-31 14:44:02 -07:00
tnum.c
trampoline.c bpf, x64: Fix tailcall infinite loop 2023-09-12 13:06:12 -07:00
verifier.c bpf: Implement cgroup sockaddr hooks for unix sockets 2023-10-11 17:27:47 -07:00