santosh.shilimkar@oracle.com 7b4b000951 RDS: fix rds-ping deadlock over TCP transport ... Sowmini found hang with rds-ping while testing RDS over TCP. Its a corner case and doesn't happen always. The issue is not reproducible with IB transport. Its clear from below dump why we see it with RDS TCP. [<ffffffff8153b7e5>] do_tcp_setsockopt+0xb5/0x740 [<ffffffff8153bec4>] tcp_setsockopt+0x24/0x30 [<ffffffff814d57d4>] sock_common_setsockopt+0x14/0x20 [<ffffffffa096071d>] rds_tcp_xmit_prepare+0x5d/0x70 [rds_tcp] [<ffffffffa093b5f7>] rds_send_xmit+0xd7/0x740 [rds] [<ffffffffa093bda2>] rds_send_pong+0x142/0x180 [rds] [<ffffffffa0939d34>] rds_recv_incoming+0x274/0x330 [rds] [<ffffffff810815ae>] ? ttwu_queue+0x11e/0x130 [<ffffffff814dcacd>] ? skb_copy_bits+0x6d/0x2c0 [<ffffffffa0960350>] rds_tcp_data_recv+0x2f0/0x3d0 [rds_tcp] [<ffffffff8153d836>] tcp_read_sock+0x96/0x1c0 [<ffffffffa0960060>] ? rds_tcp_recv_init+0x40/0x40 [rds_tcp] [<ffffffff814d6a90>] ? sock_def_write_space+0xa0/0xa0 [<ffffffffa09604d1>] rds_tcp_data_ready+0xa1/0xf0 [rds_tcp] [<ffffffff81545249>] tcp_data_queue+0x379/0x5b0 [<ffffffffa0960cdb>] ? rds_tcp_write_space+0xbb/0x110 [rds_tcp] [<ffffffff81547fd2>] tcp_rcv_established+0x2e2/0x6e0 [<ffffffff81552602>] tcp_v4_do_rcv+0x122/0x220 [<ffffffff81553627>] tcp_v4_rcv+0x867/0x880 [<ffffffff8152e0b3>] ip_local_deliver_finish+0xa3/0x220 This happens because rds_send_xmit() chain wants to take sock_lock which is already taken by tcp_v4_rcv() on its way to rds_tcp_data_ready(). Commit db6526dcb5 ("RDS: use rds_send_xmit() state instead of RDS_LL_SEND_FULL") which was trying to opportunistically finish the send request in same thread context. But because of above recursive lock hang with RDS TCP, the send work from rds_send_pong() needs to deferred to worker to avoid lock up. Given RDS ping is more of connectivity test than performance critical path, its should be ok even for transport like IB. Reported-by: Sowmini Varadhan <sowmini.varadhan@oracle.com> Acked-by: Sowmini Varadhan <sowmini.varadhan@oracle.com> Signed-off-by: Santosh Shilimkar <ssantosh@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilimkar@oracle.com> Acked-by: Sowmini Varadhan <sowmini.varadhan@oracle.com> Signed-off-by: David S. Miller <davem@davemloft.net>		2015-10-18 22:45:55 -07:00
..
af_rds.c	RDS: Use per-bucket rw lock for bind hash-table	2015-09-30 12:43:25 -04:00
bind.c	RDS: Invoke ->laddr_check() in rds_bind() for explicitly bound transports.	2015-10-13 04:22:40 -07:00
cong.c
connection.c	RDS: Use a single TCP socket for both send and receive.	2015-10-05 03:34:51 -07:00
ib.c	RDS: IB: split mr pool to improve 8K messages performance	2015-10-05 11:19:02 -07:00
ib.h	RDS: IB: split mr pool to improve 8K messages performance	2015-10-05 11:19:02 -07:00
ib_cm.c	RDS: IB: handle rds_ibdev release case instead of crashing the kernel	2015-10-05 11:19:01 -07:00
ib_rdma.c	RDS: IB: split mr pool to improve 8K messages performance	2015-10-05 11:19:02 -07:00
ib_recv.c	RDS: IB: ack more receive completions to improve performance	2015-10-05 11:19:01 -07:00
ib_ring.c
ib_send.c	RDS: IB: split send completion handling and do batch ack	2015-10-05 11:19:01 -07:00
ib_stats.c	RDS: IB: split mr pool to improve 8K messages performance	2015-10-05 11:19:02 -07:00
ib_sysctl.c
info.c
info.h
iw.c
iw.h
iw_cm.c
iw_rdma.c
iw_recv.c
iw_ring.c
iw_send.c
iw_stats.c
iw_sysctl.c
Kconfig
loop.c
loop.h
Makefile
message.c
page.c
rdma.c
rdma_transport.c
rdma_transport.h
rds.h	Merge branch 'net/rds/4.3-v3' of git://git.kernel.org/pub/scm/linux/kernel/git/ssantosh/linux	2015-10-08 04:38:37 -07:00
recv.c
send.c	RDS: fix rds-ping deadlock over TCP transport	2015-10-18 22:45:55 -07:00
stats.c
sysctl.c
tcp.c	RDS-TCP: Do not bloat sndbuf/rcvbuf in rds_tcp_tune	2015-10-05 03:34:53 -07:00
tcp.h
tcp_connect.c
tcp_listen.c	RDS-TCP: Reset tcp callbacks if re-using an outgoing socket in rds_tcp_accept_one()	2015-10-13 04:22:41 -07:00
tcp_recv.c
tcp_send.c	RDS-TCP: Set up MSG_MORE and MSG_SENDPAGE_NOTLAST as appropriate in rds_tcp_xmit	2015-10-05 03:34:53 -07:00
tcp_stats.c
threads.c	RDS: use rds_send_xmit() state instead of RDS_LL_SEND_FULL	2015-10-05 11:19:01 -07:00
transport.c