linux/fs/proc
Minchan Kim b3a81d0841 mm: fix KSM data corruption
Nadav reported KSM can corrupt the user data by the TLB batching
race[1].  That means data user written can be lost.

Quote from Nadav Amit:
 "For this race we need 4 CPUs:

  CPU0: Caches a writable and dirty PTE entry, and uses the stale value
  for write later.

  CPU1: Runs madvise_free on the range that includes the PTE. It would
  clear the dirty-bit. It batches TLB flushes.

  CPU2: Writes 4 to /proc/PID/clear_refs , clearing the PTEs soft-dirty.
  We care about the fact that it clears the PTE write-bit, and of
  course, batches TLB flushes.

  CPU3: Runs KSM. Our purpose is to pass the following test in
  write_protect_page():

	if (pte_write(*pvmw.pte) || pte_dirty(*pvmw.pte) ||
	    (pte_protnone(*pvmw.pte) && pte_savedwrite(*pvmw.pte)))

  Since it will avoid TLB flush. And we want to do it while the PTE is
  stale. Later, and before replacing the page, we would be able to
  change the page.

  Note that all the operations the CPU1-3 perform canhappen in parallel
  since they only acquire mmap_sem for read.

  We start with two identical pages. Everything below regards the same
  page/PTE.

  CPU0        CPU1        CPU2        CPU3
  ----        ----        ----        ----
  Write the same
  value on page

  [cache PTE as
   dirty in TLB]

              MADV_FREE
              pte_mkclean()

                          4 > clear_refs
                          pte_wrprotect()

                                      write_protect_page()
                                      [ success, no flush ]

                                      pages_indentical()
                                      [ ok ]

  Write to page
  different value

  [Ok, using stale
   PTE]

                                      replace_page()

  Later, CPU1, CPU2 and CPU3 would flush the TLB, but that is too late.
  CPU0 already wrote on the page, but KSM ignored this write, and it got
  lost"

In above scenario, MADV_FREE is fixed by changing TLB batching API
including [set|clear]_tlb_flush_pending.  Remained thing is soft-dirty
part.

This patch changes soft-dirty uses TLB batching API instead of
flush_tlb_mm and KSM checks pending TLB flush by using
mm_tlb_flush_pending so that it will flush TLB to avoid data lost if
there are other parallel threads pending TLB flush.

[1] http://lkml.kernel.org/r/BD3A0EBE-ECF4-41D4-87FA-C755EA9AB6BD@gmail.com

Link: http://lkml.kernel.org/r/20170802000818.4760-8-namit@vmware.com
Signed-off-by: Minchan Kim <minchan@kernel.org>
Signed-off-by: Nadav Amit <namit@vmware.com>
Reported-by: Nadav Amit <namit@vmware.com>
Tested-by: Nadav Amit <namit@vmware.com>
Reviewed-by: Andrea Arcangeli <aarcange@redhat.com>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Hugh Dickins <hughd@google.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jeff Dike <jdike@addtoit.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Nadav Amit <nadav.amit@gmail.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2017-08-10 15:54:07 -07:00
..
array.c sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h> 2017-03-02 08:42:39 +01:00
base.c fault-inject: add /proc/<pid>/fail-nth 2017-07-14 15:05:13 -07:00
cmdline.c fs/proc: don't use module_init for non-modular core code 2014-01-23 16:37:02 -08:00
consoles.c fs/proc: don't use module_init for non-modular core code 2014-01-23 16:37:02 -08:00
cpuinfo.c fs/proc: don't use module_init for non-modular core code 2014-01-23 16:37:02 -08:00
devices.c fs/proc: don't use module_init for non-modular core code 2014-01-23 16:37:02 -08:00
fd.c sched/headers: Prepare for new header dependencies before moving code to <linux/sched/signal.h> 2017-03-02 08:42:29 +01:00
fd.h proc: unsigned file descriptors 2016-09-27 18:47:38 -04:00
generic.c fs/proc/generic.c: switch to ida_simple_get/remove 2017-07-10 16:32:34 -07:00
inode.c fs/proc/inode.c: remove cast from memory allocation 2017-05-08 17:15:10 -07:00
internal.h Now that IPC and other changes have landed, enable manual markings for 2017-07-19 08:55:18 -07:00
interrupts.c fs/proc: don't use module_init for non-modular core code 2014-01-23 16:37:02 -08:00
Kconfig fs, proc: add help for CONFIG_PROC_CHILDREN 2015-07-17 16:39:52 -07:00
kcore.c fs/proc: kcore: use kcore_list type to check for vmalloc/module address 2017-06-20 12:42:57 +01:00
kmsg.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
loadavg.c sched/headers: Prepare for new header dependencies before moving code to <linux/sched/stat.h> 2017-03-02 08:42:34 +01:00
Makefile fs/proc: Add compiler check for -Wno-override-init to support gcc < 4.2 2016-08-03 12:45:23 -04:00
meminfo.c mm: fix global NR_SLAB_.*CLAIMABLE counter reads 2017-08-10 15:54:06 -07:00
namespaces.c pidns: expose task pid_ns_for_children to userspace 2017-05-08 17:15:12 -07:00
nommu.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
page.c mm: fix KPF_SWAPCACHE in /proc/kpageflags 2017-02-07 12:08:32 -08:00
proc_net.c Merge branch 'rebased-statx' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-03-03 11:38:56 -08:00
proc_sysctl.c Merge branch 'akpm' (patches from Andrew) 2017-07-13 12:38:49 -07:00
proc_tty.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
root.c Merge branch 'rebased-statx' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-03-03 11:38:56 -08:00
self.c vfs: remove ".readlink = generic_readlink" assignments 2016-12-09 16:45:04 +01:00
softirqs.c fs/proc: don't use module_init for non-modular core code 2014-01-23 16:37:02 -08:00
stat.c sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h> 2017-03-02 08:42:39 +01:00
task_mmu.c mm: fix KSM data corruption 2017-08-10 15:54:07 -07:00
task_nommu.c sched/headers: Prepare for new header dependencies before moving code to <linux/sched/mm.h> 2017-03-02 08:42:28 +01:00
thread_self.c vfs: remove ".readlink = generic_readlink" assignments 2016-12-09 16:45:04 +01:00
uptime.c sched/cputime: Convert kcpustat to nsecs 2017-02-01 09:13:47 +01:00
version.c fs/proc: don't use module_init for non-modular core code 2014-01-23 16:37:02 -08:00
vmcore.c userfaultfd: non-cooperative: add event for memory unmaps 2017-02-24 17:46:55 -08:00