system/linux
system/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux synced 2024-10-03 01:43:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
linux/kernel
History
Stanislav Fomichev 20f2505fb4 bpf: Try to avoid kzalloc in cgroup/{s,g}etsockopt 
When we attach a bpf program to cgroup/getsockopt any other getsockopt()
syscall starts incurring kzalloc/kfree cost.

Let add a small buffer on the stack and use it for small (majority)
{s,g}etsockopt values. The buffer is small enough to fit into
the cache line and cover the majority of simple options (most
of them are 4 byte ints).

It seems natural to do the same for setsockopt, but it's a bit more
involved when the BPF program modifies the data (where we have to
kmalloc). The assumption is that for the majority of setsockopt
calls (which are doing pure BPF options or apply policy) this
will bring some benefit as well.

Without this patch (we remove about 1% __kmalloc):
     3.38%     0.07%  tcp_mmap  [kernel.kallsyms]  [k] __cgroup_bpf_run_filter_getsockopt
            |
             --3.30%--__cgroup_bpf_run_filter_getsockopt
                       |
                        --0.81%--__kmalloc

Signed-off-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Link: https://lore.kernel.org/bpf/20210115163501.805133-3-sdf@google.com
2021-01-20 14:23:00 -08:00
..
bpf bpf: Try to avoid kzalloc in cgroup/{s,g}etsockopt 2021-01-20 14:23:00 -08:00
cgroup Merge branch 'for-5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup 2020-12-28 11:16:38 -08:00
configs staging: ION: remove some references to CONFIG_ION 2021-01-06 17:39:38 +01:00
debug smp: Cleanup smp_call_function*() 2020-11-24 16:47:49 +01:00
dma dma-mapping updates for 5.11: 2020-12-22 13:19:43 -08:00
entry The new preemtible kmap_local() implementation: 2020-12-14 18:35:53 -08:00
events perf: Add build id data in mmap2 event 2021-01-14 19:29:58 -08:00
gcov gcov: fix kernel-doc markup issue 2020-12-15 22:46:18 -08:00
irq genirq: Fix export of irq_to_desc() for powerpc KVM 2020-12-25 11:02:39 -08:00
kcsan kcsan: Fix encoding masks and regain address bit 2020-11-06 17:19:26 -08:00
livepatch livepatch: Use the default ftrace_ops instead of REGS when ARGS is available 2020-11-13 12:15:28 -05:00
locking A moderate set of locking updates: 2020-12-14 17:27:47 -08:00
power Power management updates for 5.11-rc1 2020-12-15 16:30:31 -08:00
printk printk changes for 5.11 2020-12-16 10:45:11 -08:00
rcu Merge branch 'rcu/urgent' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu 2021-01-04 10:55:19 -08:00
sched Fix a context switch performance regression. 2020-12-27 09:00:47 -08:00
time Update/fix two CPU sanity checks in the hotplug and the boot code, 2020-12-27 09:03:41 -08:00
trace tracing/kprobes: Do the notrace functions check without kprobes on ftrace 2021-01-11 16:09:53 -05:00
.gitignore
acct.c kernel/acct.c: use #elif instead of #end and #elif 2020-12-15 22:46:15 -08:00
async.c
audit.c audit: replace atomic_add_return() 2020-12-02 22:52:16 -05:00
audit.h
audit_fsnotify.c fsnotify: generalize handle_inode_event() 2020-12-03 14:58:35 +01:00
audit_tree.c fsnotify: generalize handle_inode_event() 2020-12-03 14:58:35 +01:00
audit_watch.c fsnotify: generalize handle_inode_event() 2020-12-03 14:58:35 +01:00
auditfilter.c
auditsc.c audit/stable-5.11 PR 20201214 2020-12-16 10:54:03 -08:00
backtracetest.c
bounds.c
capability.c
compat.c
configs.c
context_tracking.c
cpu.c Scheduler updates: 2020-12-14 18:29:11 -08:00
cpu_pm.c
crash_core.c kdump: append uts_namespace.name offset to VMCOREINFO 2020-12-15 22:46:18 -08:00
crash_dump.c
cred.c
delayacct.c
dma.c
exec_domain.c
exit.c kernel/io_uring: cancel io_uring before task works 2020-12-30 19:36:54 -07:00
extable.c
fail_function.c fault-injection: handle EI_ETYPE_TRUE 2020-12-15 22:46:19 -08:00
fork.c kasan: rename (un)poison_shadow to (un)poison_range 2020-12-22 12:55:06 -08:00
freezer.c
futex.c Merge branch 'locking/rwsem' 2020-12-09 17:08:45 +01:00
gen_kheaders.sh
groups.c
hung_task.c
iomem.c
irq_work.c irq_work: Optimize irq_work_single() 2020-11-24 16:47:49 +01:00
jump_label.c jump_label: Fix usage in module __init 2020-12-18 16:53:12 +01:00
kallsyms.c
kcmp.c Merge branch 'exec-update-lock-for-v5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2020-12-15 19:36:48 -08:00
Kconfig.freezer
Kconfig.hz
Kconfig.locks
Kconfig.preempt
kcov.c kernel: make kcov_common_handle consider the current context 2020-11-02 18:00:20 -08:00
kexec.c
kexec_core.c crypto: sha - split sha.h into sha1.h and sha2.h 2020-11-20 14:45:33 +11:00
kexec_elf.c
kexec_file.c crypto: sha - split sha.h into sha1.h and sha2.h 2020-11-20 14:45:33 +11:00
kexec_internal.h
kheaders.c
kmod.c
kprobes.c Merge branch 'linus' into perf/kprobes 2020-11-07 13:18:49 +01:00
ksysfs.c
kthread.c Merge branch 'akpm' (patches from Andrew) 2020-12-15 12:53:37 -08:00
latencytop.c
Makefile kcov: don't instrument with UBSAN 2020-12-15 22:46:19 -08:00
module-internal.h
module.c Modules updates for v5.11 2020-12-17 13:01:31 -08:00
module_signature.c
module_signing.c
notifier.c
nsproxy.c fixes-v5.11 2020-12-14 16:40:27 -08:00
padata.c
panic.c panic: don't dump stack twice on warn 2020-11-14 11:26:04 -08:00
params.c Modules updates for v5.11 2020-12-17 13:01:31 -08:00
pid.c Merge branch 'exec-update-lock-for-v5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2020-12-15 19:36:48 -08:00
pid_namespace.c fixes-v5.11 2020-12-14 16:40:27 -08:00
profile.c
ptrace.c Merge branch 'akpm' (patches from Andrew) 2020-12-15 12:53:37 -08:00
range.c
reboot.c reboot: hide from sysfs not applicable settings 2020-12-15 22:46:19 -08:00
regset.c
relay.c relay: allow the use of const callback structs 2020-12-15 22:46:18 -08:00
resource.c kernel/resource.c: fix kernel-doc markups 2020-12-15 22:46:18 -08:00
resource_kunit.c resource: provide meaningful MODULE_LICENSE() in test suite 2020-11-25 18:52:35 +01:00
rseq.c
scftorture.c scftorture: Add full-test stutter capability 2020-11-06 17:13:56 -08:00
scs.c scs: switch to vmapped shadow stacks 2020-12-01 10:30:28 +00:00
seccomp.c seccomp updates for v5.11-rc1 2020-12-16 11:30:10 -08:00
signal.c task_work: unconditionally run task_work from get_signal() 2021-01-08 09:14:21 -07:00
smp.c Merge branch 'linus' into sched/core, to resolve semantic conflict 2020-11-27 11:10:50 +01:00
smpboot.c
smpboot.h
softirq.c Misc fixes/updates: 2020-12-27 09:06:10 -08:00
stackleak.c
stacktrace.c
static_call.c
stop_machine.c Merge branch 'linus' into sched/core, to resolve semantic conflict 2020-11-27 11:10:50 +01:00
sys.c kernel: Implement selective syscall userspace redirection 2020-12-02 15:07:56 +01:00
sys_ni.c epoll: wire up syscall epoll_pwait2 2020-12-19 11:18:38 -08:00
sysctl-test.c
sysctl.c rcu: Panic after fixed number of stalls 2020-11-19 19:37:16 -08:00
task_work.c task_work: remove legacy TWA_SIGNAL path 2020-12-12 09:17:38 -07:00
taskstats.c treewide: rename nla_strlcpy to nla_strscpy. 2020-11-16 08:08:54 -08:00
test_kprobes.c
torture.c rcutorture: Make stutter_wait() caller restore priority 2020-11-06 17:13:54 -08:00
tracepoint.c tracepoints: Migrate to use SYSCALL_WORK flag 2020-11-16 21:53:15 +01:00
tsacct.c
ucount.c
uid16.c
uid16.h
umh.c
up.c
user-return-notifier.c
user.c
user_namespace.c fixes-v5.11 2020-12-14 16:40:27 -08:00
usermode_driver.c
utsname.c
utsname_sysctl.c
watch_queue.c
watchdog.c kernel/watchdog: fix watchdog_allowed_mask not used warning 2020-11-14 11:26:03 -08:00
watchdog_hld.c
workqueue.c Merge branch 'for-5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq 2020-12-28 11:23:02 -08:00
workqueue_internal.h