system/linux
system/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux synced 2024-09-22 12:26:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
linux/net/ipv4
History
NeilBrown 8f0db01800 rhashtable: use bit_spin_locks to protect hash bucket. 
This patch changes rhashtables to use a bit_spin_lock on BIT(1) of the
bucket pointer to lock the hash chain for that bucket.

The benefits of a bit spin_lock are:
 - no need to allocate a separate array of locks.
 - no need to have a configuration option to guide the
   choice of the size of this array
 - locking cost is often a single test-and-set in a cache line
   that will have to be loaded anyway.  When inserting at, or removing
   from, the head of the chain, the unlock is free - writing the new
   address in the bucket head implicitly clears the lock bit.
   For __rhashtable_insert_fast() we ensure this always happens
   when adding a new key.
 - even when lockings costs 2 updates (lock and unlock), they are
   in a cacheline that needs to be read anyway.

The cost of using a bit spin_lock is a little bit of code complexity,
which I think is quite manageable.

Bit spin_locks are sometimes inappropriate because they are not fair -
if multiple CPUs repeatedly contend of the same lock, one CPU can
easily be starved.  This is not a credible situation with rhashtable.
Multiple CPUs may want to repeatedly add or remove objects, but they
will typically do so at different buckets, so they will attempt to
acquire different locks.

As we have more bit-locks than we previously had spinlocks (by at
least a factor of two) we can expect slightly less contention to
go with the slightly better cache behavior and reduced memory
consumption.

To enhance type checking, a new struct is introduced to represent the
  pointer plus lock-bit
that is stored in the bucket-table.  This is "struct rhash_lock_head"
and is empty.  A pointer to this needs to be cast to either an
unsigned lock, or a "struct rhash_head *" to be useful.
Variables of this type are most often called "bkt".

Previously "pprev" would sometimes point to a bucket, and sometimes a
->next pointer in an rhash_head.  As these are now different types,
pprev is NULL when it would have pointed to the bucket. In that case,
'blk' is used, together with correct locking protocol.

Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-07 19:12:12 -07:00
..
bpfilter net: bpfilter: disallow to remove bpfilter module while being used 2019-01-11 18:05:41 -08:00
netfilter netfilter: nf_tables: merge ipv4 and ipv6 nat chain types 2019-03-01 14:36:59 +01:00
af_inet.c net: use rcu_dereference_protected to fetch sk_dst_cache in sk_destruct 2019-04-01 18:10:51 -07:00
ah4.c net-ipv4: remove 2 always zero parameters from ipv4_redirect() 2018-09-26 20:30:55 -07:00
arp.c net: Evict neighbor entries on carrier down 2018-10-12 09:47:39 -07:00
cipso_ipv4.c netlabel: fix out-of-bounds memory accesses 2019-02-27 21:45:24 -08:00
datagram.c ipv4: Allow sending multicast packets on specific i/f using VRF socket 2018-10-02 22:28:17 -07:00
devinet.c net: ignore sysctl_devconf_inherit_init_net without SYSCTL 2019-03-04 13:14:34 -08:00
esp4.c esp: Skip TX bytes accounting when sending from a request socket 2019-01-28 11:20:58 +01:00
esp4_offload.c net: use skb_sec_path helper in more places 2018-12-19 11:21:37 -08:00
fib_frontend.c ipv4: Add fib_nh_common to fib_result 2019-04-03 21:50:20 -07:00
fib_lookup.h ipv4: Add fib_nh_common to fib_result 2019-04-03 21:50:20 -07:00
fib_notifier.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
fib_rules.c ipv4: fib_rules: Fix possible infinite loop in fib_empty_table 2018-12-30 12:57:04 -08:00
fib_semantics.c ipv6: Flip to fib_nexthop_info 2019-04-03 21:50:20 -07:00
fib_trie.c ipv4: Add fib_nh_common to fib_result 2019-04-03 21:50:20 -07:00
fou.c fou: Support binding FoU socket 2019-03-27 13:30:07 -07:00
gre_demux.c net: ip_gre: use erspan key field for tunnel lookup 2019-01-22 11:52:17 -08:00
gre_offload.c Merge ra.kernel.org:/pub/scm/linux/kernel/git/davem/net 2018-07-03 10:29:26 +09:00
icmp.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2019-03-02 12:54:35 -08:00
igmp.c net: remove unneeded switch fall-through 2019-02-21 13:48:00 -08:00
inet_connection_sock.c inet: minor optimization for backlog setting in listen(2) 2018-11-07 22:31:07 -08:00
inet_diag.c inet_diag: fix reporting cgroup classid and fallback to priority 2019-02-12 13:35:57 -05:00
inet_fragment.c net: remove unused struct inet_frag_queue.fragments field 2019-02-26 08:27:05 -08:00
inet_hashtables.c net: dccp: fix kernel crash on module load 2018-12-24 15:27:56 -08:00
inet_timewait_sock.c soreuseport: initialise timewait reuseport field 2018-04-07 22:32:32 -04:00
inetpeer.c net: ipv4: use a dedicated counter for icmp_v4 redirect packets 2019-02-08 21:50:15 -08:00
ip_forward.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2018-12-20 11:53:36 -08:00
ip_fragment.c net: remove unused struct inet_frag_queue.fragments field 2019-02-26 08:27:05 -08:00
ip_gre.c route: Add multipath_hash in flowi_common to make user-define hash 2019-02-27 12:50:17 -08:00
ip_input.c vrf: check accept_source_route on the original netdevice 2019-04-01 10:44:58 -07:00
ip_options.c vrf: check accept_source_route on the original netdevice 2019-04-01 10:44:58 -07:00
ip_output.c net: use kfree_skb_list() from ip_do_fragment() 2019-04-04 10:48:26 -07:00
ip_sockglue.c ip: on queued skb use skb_header_pointer instead of pskb_may_pull 2019-01-10 09:27:20 -05:00
ip_tunnel.c iptunnel: NULL pointer deref for ip_md_tunnel_xmit 2019-03-06 10:43:06 -08:00
ip_tunnel_core.c ip_tunnel: Add dst_cache support in lwtunnel_state of ip tunnel 2019-02-24 22:13:49 -08:00
ip_vti.c vti4: Fix a ipip packet processing bug in 'IPCOMP' virtual tunnel 2019-01-09 14:00:37 +01:00
ipcomp.c net-ipv4: remove 2 always zero parameters from ipv4_redirect() 2018-09-26 20:30:55 -07:00
ipconfig.c ipconfig: add carrier_timeout kernel parameter 2019-02-01 15:24:13 -08:00
ipip.c ip_tunnel: Add tnl_update_pmtu in ip_md_tunnel_xmit 2019-01-26 09:43:03 -08:00
ipmr.c rhashtable: use bit_spin_locks to protect hash bucket. 2019-04-07 19:12:12 -07:00
ipmr_base.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2018-10-19 11:03:06 -07:00
Kconfig net: remove blank lines at end of file 2018-07-24 14:10:43 -07:00
Makefile bpf, sockmap: convert to generic sk_msg interface 2018-10-15 12:23:19 -07:00
metrics.c net: Add extack argument to ip_fib_metrics_init 2018-11-06 15:00:45 -08:00
netfilter.c netfilter: ipv4: remove useless export_symbol 2019-01-28 11:32:58 +01:00
netlink.c ipv4: Add ICMPv6 support when parse route ipproto 2019-03-01 16:41:27 -08:00
ping.c ipv4: Allow sending multicast packets on specific i/f using VRF socket 2018-10-02 22:28:17 -07:00
proc.c tcp: implement coalescing on backlog queue 2018-11-30 13:26:54 -08:00
protocol.c fou, fou6: ICMP error handlers for FoU and GUE 2018-11-08 17:13:08 -08:00
raw.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2018-12-20 11:53:36 -08:00
raw_diag.c net: ipv6: add second dif to raw socket lookups 2017-08-07 11:39:22 -07:00
route.c ipv4: Add fib_nh_common to fib_result 2019-04-03 21:50:20 -07:00
syncookies.c tcp: free request sock directly upon TFO or syncookies error 2019-03-19 14:13:01 -07:00
sysctl_net_ipv4.c ipv4: Allow amount of dirty memory from fib resizing to be controllable 2019-03-21 13:29:53 -07:00
tcp.c tcp: cleanup sk_tx_skb_cache before reuse 2019-03-29 13:16:44 -07:00
tcp_bbr.c tcp_bbr: adapt cwnd based on ack aggregation estimation 2019-01-24 22:27:27 -08:00
tcp_bic.c tcp: consolidate congestion control undo functions 2017-08-06 21:25:10 -07:00
tcp_bpf.c bpf: sk_msg, sock{map|hash} redirect through ULP 2018-12-20 23:47:09 +01:00
tcp_cdg.c tcp: cdg: use tcp high resolution clock cache 2018-10-15 22:56:42 -07:00
tcp_cong.c tcp: Namespace-ify sysctl_tcp_default_congestion_control 2017-11-15 14:09:52 +09:00
tcp_cubic.c tcp: consolidate congestion control undo functions 2017-08-06 21:25:10 -07:00
tcp_dctcp.c tcp: Ensure DCTCP reacts to losses 2019-04-04 10:51:16 -07:00
tcp_dctcp.h tcp: refactor DCTCP ECN ACK handling 2018-10-10 22:26:00 -07:00
tcp_diag.c net: sock: replace sk_state_load with inet_sk_state_load and remove sk_state_store 2017-12-20 14:00:25 -05:00
tcp_fastopen.c tcp: pause Fast Open globally after third consecutive timeout 2017-12-13 15:51:12 -05:00
tcp_highspeed.c tcp: consolidate congestion control undo functions 2017-08-06 21:25:10 -07:00
tcp_htcp.c tcp: fix cwnd undo in Reno and HTCP congestion controls 2017-08-06 21:25:10 -07:00
tcp_hybla.c
tcp_illinois.c net/tcp/illinois: replace broken algorithm reference link 2018-02-28 12:03:47 -05:00
tcp_input.c tcp: Accept ECT on SYN in the presence of RFC8311 2019-04-04 17:43:48 -07:00
tcp_ipv4.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2019-04-05 14:14:19 -07:00
tcp_lp.c tcp: switch TCP TS option (RFC 7323) to 1ms clock 2017-05-17 16:06:01 -04:00
tcp_metrics.c genetlink: make policy common to family 2019-03-22 10:38:23 -04:00
tcp_minisocks.c tcp: use tcp_md5_needed for timewait sockets 2019-02-26 13:16:03 -08:00
tcp_nv.c tcp_nv: fix potential integer overflow in tcpnv_acked 2018-01-31 10:26:30 -05:00
tcp_offload.c net: use indirect call wrappers at GRO transport layer 2018-12-15 13:23:02 -08:00
tcp_output.c tcp: remove redundant check on tskb 2019-04-06 18:18:14 -07:00
tcp_rate.c tcp: introduce tcp_skb_timestamp_us() helper 2018-09-21 19:37:59 -07:00
tcp_recovery.c tcp: introduce tcp_skb_timestamp_us() helper 2018-09-21 19:37:59 -07:00
tcp_scalable.c tcp: consolidate congestion control undo functions 2017-08-06 21:25:10 -07:00
tcp_timer.c tcp: Refactor pingpong code 2019-01-27 13:29:43 -08:00
tcp_ulp.c tcp, ulp: remove socket lock assertion on ULP cleanup 2018-10-16 12:38:41 -07:00
tcp_vegas.c tcp: fix under-evaluated ssthresh in TCP Vegas 2017-09-29 06:07:00 +01:00
tcp_vegas.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
tcp_veno.c tcp: consolidate congestion control undo functions 2017-08-06 21:25:10 -07:00
tcp_westwood.c tcp: Revert "tcp: remove CA_ACK_SLOWPATH" 2017-08-30 11:20:08 -07:00
tcp_yeah.c tcp: consolidate congestion control undo functions 2017-08-06 21:25:10 -07:00
tunnel4.c net: Convert protocol error handlers from void to int 2018-11-08 17:13:08 -08:00
udp.c udp: fix possible user after free in error handler 2019-02-22 16:05:11 -08:00
udp_diag.c net: diag: document swapped src/dst in udp_dump_one. 2018-10-28 19:27:21 -07:00
udp_impl.h udp: add missing rehash callback to udplite 2019-01-17 15:01:08 -08:00
udp_offload.c udp: use indirect call wrappers for GRO socket lookup 2018-12-15 13:23:02 -08:00
udp_tunnel.c net/ipv4/udp_tunnel: prefer SO_BINDTOIFINDEX over SO_BINDTODEVICE 2019-01-17 14:55:52 -08:00
udplite.c udp: add missing rehash callback to udplite 2019-01-17 15:01:08 -08:00
xfrm4_input.c xfrm: reset transport header back to network header after all input transforms ahave been applied 2018-09-04 10:26:30 +02:00
xfrm4_mode_beet.c networking: make skb_pull & friends return void pointers 2017-06-16 11:48:39 -04:00
xfrm4_mode_transport.c xfrm: reset transport header back to network header after all input transforms ahave been applied 2018-09-04 10:26:30 +02:00
xfrm4_mode_tunnel.c xfrm: Verify MAC header exists before overwriting eth_hdr(skb)->h_proto 2018-03-07 10:54:29 +01:00
xfrm4_output.c net: xfrm: use skb_gso_validate_network_len() to check gso sizes 2018-03-04 17:49:17 -05:00
xfrm4_policy.c net: Drop pernet_operations::async 2018-03-27 13:18:09 -04:00
xfrm4_protocol.c net: Convert protocol error handlers from void to int 2018-11-08 17:13:08 -08:00
xfrm4_state.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
xfrm4_tunnel.c