linux/lib
Eric Dumazet d7ce369243 dump_stack: avoid potential deadlocks
Some servers experienced fatal deadlocks because of a combination of
bugs, leading to multiple cpus calling dump_stack().

The checksumming bug was fixed in commit 34ae6a1aa0 ("ipv6: update
skb->csum when CE mark is propagated").

The second problem is a faulty locking in dump_stack()

CPU1 runs in process context and calls dump_stack(), grabs dump_lock.

   CPU2 receives a TCP packet under softirq, grabs socket spinlock, and
   call dump_stack() from netdev_rx_csum_fault().

   dump_stack() spins on atomic_cmpxchg(&dump_lock, -1, 2), since
   dump_lock is owned by CPU1

While dumping its stack, CPU1 is interrupted by a softirq, and happens
to process a packet for the TCP socket locked by CPU2.

CPU1 spins forever in spin_lock() : deadlock

Stack trace on CPU1 looked like :

    NMI backtrace for cpu 1
    RIP: _raw_spin_lock+0x25/0x30
    ...
    Call Trace:
      <IRQ>
      tcp_v6_rcv+0x243/0x620
      ip6_input_finish+0x11f/0x330
      ip6_input+0x38/0x40
      ip6_rcv_finish+0x3c/0x90
      ipv6_rcv+0x2a9/0x500
      process_backlog+0x461/0xaa0
      net_rx_action+0x147/0x430
      __do_softirq+0x167/0x2d0
      call_softirq+0x1c/0x30
      do_softirq+0x3f/0x80
      irq_exit+0x6e/0xc0
      smp_call_function_single_interrupt+0x35/0x40
      call_function_single_interrupt+0x6a/0x70
      <EOI>
      printk+0x4d/0x4f
      printk_address+0x31/0x33
      print_trace_address+0x33/0x3c
      print_context_stack+0x7f/0x119
      dump_trace+0x26b/0x28e
      show_trace_log_lvl+0x4f/0x5c
      show_stack_log_lvl+0x104/0x113
      show_stack+0x42/0x44
      dump_stack+0x46/0x58
      netdev_rx_csum_fault+0x38/0x3c
      __skb_checksum_complete_head+0x6e/0x80
      __skb_checksum_complete+0x11/0x20
      tcp_rcv_established+0x2bd5/0x2fd0
      tcp_v6_do_rcv+0x13c/0x620
      sk_backlog_rcv+0x15/0x30
      release_sock+0xd2/0x150
      tcp_recvmsg+0x1c1/0xfc0
      inet_recvmsg+0x7d/0x90
      sock_recvmsg+0xaf/0xe0
      ___sys_recvmsg+0x111/0x3b0
      SyS_recvmsg+0x5c/0xb0
      system_call_fastpath+0x16/0x1b

Fixes: b58d977432 ("dump_stack: serialize the output from dump_stack()")
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Alex Thorlton <athorlton@sgi.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-02-05 18:10:40 -08:00
..
842
fonts
lz4
lzo
mpi
raid6
reed_solomon
xz
zlib_deflate
zlib_inflate
.gitignore
argv_split.c
asn1_decoder.c
assoc_array.c
atomic64.c
atomic64_test.c
audit.c
bcd.c
bch.c
bitmap.c
bitrev.c
bsearch.c
btree.c
bug.c
build_OID_registry
bust_spinlocks.c
check_signature.c
checksum.c
clz_ctz.c
clz_tab.c
cmdline.c
compat_audit.c
cordic.c
cpu-notifier-error-inject.c
cpu_rmap.c
cpumask.c
crc-ccitt.c
crc-itu-t.c
crc-t10dif.c
crc7.c
crc8.c
crc16.c
crc32.c
crc32defs.h
ctype.c
debug_info.c
debug_locks.c
debugobjects.c debugobjects: Allow bigger number of early boot objects 2016-01-27 15:40:59 +01:00
dec_and_lock.c
decompress.c
decompress_bunzip2.c
decompress_inflate.c
decompress_unlz4.c
decompress_unlzma.c
decompress_unlzo.c
decompress_unxz.c
devres.c
digsig.c
div64.c
dma-debug.c
dump_stack.c dump_stack: avoid potential deadlocks 2016-02-05 18:10:40 -08:00
dynamic_debug.c
dynamic_queue_limits.c
earlycpio.c
extable.c
fault-inject.c
fdt.c
fdt_empty_tree.c
fdt_ro.c
fdt_rw.c
fdt_strerror.c
fdt_sw.c
fdt_wip.c
find_bit.c
flex_array.c
flex_proportions.c
gcd.c
gen_crc32table.c
genalloc.c
glob.c
halfmd4.c
hexdump.c
hweight.c
idr.c
inflate.c
int_sqrt.c
interval_tree.c
interval_tree_test.c
iomap.c
iomap_copy.c
iommu-common.c
iommu-helper.c
ioremap.c
iov_iter.c
irq_poll.c
irq_regs.c
is_single_threaded.c
jedec_ddr_data.c
kasprintf.c
Kconfig Initial roundup of 4.5 merge window patches 2016-01-23 18:45:06 -08:00
Kconfig.debug
Kconfig.kasan
Kconfig.kgdb
Kconfig.kmemcheck
Kconfig.ubsan
kfifo.c
klist.c
kobject.c
kobject_uevent.c
kstrtox.c
kstrtox.h
lcm.c
libcrc32c.c
list_debug.c
list_sort.c
llist.c
locking-selftest-hardirq.h
locking-selftest-mutex.h
locking-selftest-rlock-hardirq.h
locking-selftest-rlock-softirq.h
locking-selftest-rlock.h
locking-selftest-rsem.h
locking-selftest-softirq.h
locking-selftest-spin-hardirq.h
locking-selftest-spin-softirq.h
locking-selftest-spin.h
locking-selftest-wlock-hardirq.h
locking-selftest-wlock-softirq.h
locking-selftest-wlock.h
locking-selftest-wsem.h
locking-selftest.c
lockref.c
lru_cache.c
Makefile Initial roundup of 4.5 merge window patches 2016-01-23 18:45:06 -08:00
md5.c
memory-notifier-error-inject.c
memweight.c
net_utils.c
netdev-notifier-error-inject.c
nlattr.c
nmi_backtrace.c
notifier-error-inject.c
notifier-error-inject.h
of-reconfig-notifier-error-inject.c
oid_registry.c
once.c
parser.c
pci_iomap.c
percpu-refcount.c
percpu_counter.c
percpu_ida.c
percpu_test.c
plist.c
pm-notifier-error-inject.c
proportions.c
radix-tree.c radix-tree: fix race in gang lookup 2016-02-03 08:28:43 -08:00
random32.c
ratelimit.c
rational.c
rbtree.c
rbtree_test.c
reciprocal_div.c
rhashtable.c
scatterlist.c
seq_buf.c
sg_split.c
sha1.c
show_mem.c
smp_processor_id.c
sort.c
stmp_device.c
string.c
string_helpers.c
strncpy_from_user.c
strnlen_user.c
swiotlb.c
syscall.c
test-kstrtox.c
test-string_helpers.c lib/test-string_helpers.c: fix and improve string_get_size() tests 2016-02-03 08:28:43 -08:00
test_bpf.c
test_firmware.c
test_hexdump.c
test_kasan.c
test_module.c
test_printf.c
test_rhashtable.c
test_static_key_base.c
test_static_keys.c
test_user_copy.c
textsearch.c
timerqueue.c
ts_bm.c
ts_fsm.c
ts_kmp.c
ubsan.c
ubsan.h
ucs2_string.c
usercopy.c
uuid.c
vsprintf.c