linux/arch
Joerg Roedel 763802b53a x86/mm: split vmalloc_sync_all()
Commit 3f8fd02b1b ("mm/vmalloc: Sync unmappings in
__purge_vmap_area_lazy()") introduced a call to vmalloc_sync_all() in
the vunmap() code-path.  While this change was necessary to maintain
correctness on x86-32-pae kernels, it also adds additional cycles for
architectures that don't need it.

Specifically on x86-64 with CONFIG_VMAP_STACK=y some people reported
severe performance regressions in micro-benchmarks because it now also
calls the x86-64 implementation of vmalloc_sync_all() on vunmap().  But
the vmalloc_sync_all() implementation on x86-64 is only needed for newly
created mappings.

To avoid the unnecessary work on x86-64 and to gain the performance
back, split up vmalloc_sync_all() into two functions:

	* vmalloc_sync_mappings(), and
	* vmalloc_sync_unmappings()

Most call-sites to vmalloc_sync_all() only care about new mappings being
synchronized.  The only exception is the new call-site added in the
above mentioned commit.

Shile Zhang directed us to a report of an 80% regression in reaim
throughput.

Fixes: 3f8fd02b1b ("mm/vmalloc: Sync unmappings in __purge_vmap_area_lazy()")
Reported-by: kernel test robot <oliver.sang@intel.com>
Reported-by: Shile Zhang <shile.zhang@linux.alibaba.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Tested-by: Borislav Petkov <bp@suse.de>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>	[GHES]
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: <stable@vger.kernel.org>
Link: http://lkml.kernel.org/r/20191009124418.8286-1-joro@8bytes.org
Link: https://lists.01.org/hyperkitty/list/lkp@lists.01.org/thread/4D3JPPHBNOSPFK2KEPC6KGKS6J25AIDB/
Link: http://lkml.kernel.org/r/20191113095530.228959-1-shile.zhang@linux.alibaba.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-03-21 18:56:06 -07:00
..
alpha Kbuild updates for v5.6 (2nd) 2020-02-09 16:05:50 -08:00
arc ARC: define __ALIGN_STR and __ALIGN symbols for ARC 2020-03-11 10:07:15 -07:00
arm ARM fixes for 5.6: 2020-03-16 15:39:52 -07:00
arm64 arm64 fixes for -rc7 2020-03-20 09:28:25 -07:00
c6x Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2020-01-28 10:07:09 -08:00
csky csky: Replace <linux/clk-provider.h> by <linux/of_clk.h> 2020-02-23 12:48:55 +08:00
h8300 Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2020-01-28 10:07:09 -08:00
hexagon Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2020-01-28 10:07:09 -08:00
ia64 proc: convert everything to "struct proc_ops" 2020-02-04 03:05:26 +00:00
m68k Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/gerg/m68knommu 2020-02-06 08:13:23 +00:00
microblaze Microblaze patches for 5.6-rc1 2020-02-04 11:58:07 +00:00
mips MIPS: DTS: CI20: fix interrupt for pcf8563 RTC 2020-03-07 09:54:24 +01:00
nds32 TTY/Serial driver updates for 5.6-rc1 2020-01-29 10:13:27 -08:00
nios2 TTY/Serial driver updates for 5.6-rc1 2020-01-29 10:13:27 -08:00
openrisc TTY/Serial driver updates for 5.6-rc1 2020-01-29 10:13:27 -08:00
parisc Merge branch 'parisc-5.6-1' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/parisc-linux 2020-02-05 07:29:40 +00:00
powerpc KVM: PPC: Fix kernel crash with PR KVM 2020-03-20 13:39:10 +11:00
riscv riscv: fix seccomp reject syscall code path 2020-03-05 13:58:15 -08:00
s390 Bugfixes, x86+s390. 2020-03-14 15:45:26 -07:00
sh proc: convert everything to "struct proc_ops" 2020-02-04 03:05:26 +00:00
sparc Kbuild updates for v5.6 (2nd) 2020-02-09 16:05:50 -08:00
um proc: convert everything to "struct proc_ops" 2020-02-04 03:05:26 +00:00
unicore32 MPX requires recompiling applications, which requires compiler support. 2020-01-30 16:11:50 -08:00
x86 x86/mm: split vmalloc_sync_all() 2020-03-21 18:56:06 -07:00
xtensa Xtensa updates for v5.6: 2020-02-07 12:22:29 -08:00
.gitignore
Kconfig arch/Kconfig: update HAVE_RELIABLE_STACKTRACE description 2020-03-06 07:06:09 -06:00