linux/net/sched
Eric Dumazet 4d202a0d31 net_sched: generalize bulk dequeue
When qdisc bulk dequeue was added in linux-3.18 (commit
5772e9a346 "qdisc: bulk dequeue support for qdiscs
with TCQ_F_ONETXQUEUE"), it was constrained to some
specific qdiscs.

With some extra care, we can extend this to all qdiscs,
so that typical traffic shaping solutions can benefit from
small batches (8 packets in this patch).

For example, HTB is often used on some multi queue device.
And bonding/team are multi queue devices...

Idea is to bulk-dequeue packets mapping to the same transmit queue.

This brings between 35 and 80 % performance increase in HTB setup
under pressure on a bonding setup :

1) NUMA node contention :   610,000 pps -> 1,110,000 pps
2) No node contention   : 1,380,000 pps -> 1,930,000 pps

Now we should work to add batches on the enqueue() side ;)

Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: John Fastabend <john.r.fastabend@intel.com>
Cc: Jesper Dangaard Brouer <brouer@redhat.com>
Cc: Hannes Frederic Sowa <hannes@stressinduktion.org>
Cc: Florian Westphal <fw@strlen.de>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-25 12:19:35 -04:00
..
act_api.c net_sched: make tcf_hash_check() boolean 2016-06-15 12:43:35 -07:00
act_bpf.c net sched: indentation and other OCD stylistic fixes 2016-06-07 15:53:54 -07:00
act_connmark.c net sched actions: aggregate dumping of actions timeinfo 2016-06-07 15:53:43 -07:00
act_csum.c net sched actions: aggregate dumping of actions timeinfo 2016-06-07 15:53:43 -07:00
act_gact.c net sched: indentation and other OCD stylistic fixes 2016-06-07 15:53:54 -07:00
act_ife.c net_sched: make tcf_hash_check() boolean 2016-06-15 12:43:35 -07:00
act_ipt.c net_sched: make tcf_hash_check() boolean 2016-06-15 12:43:35 -07:00
act_meta_mark.c Support to encoding decoding skb mark on IFE action 2016-03-01 17:15:23 -05:00
act_meta_skbprio.c Support to encoding decoding skb prio on IFE action 2016-03-01 17:15:23 -05:00
act_mirred.c net_sched: make tcf_hash_check() boolean 2016-06-15 12:43:35 -07:00
act_nat.c net sched actions: aggregate dumping of actions timeinfo 2016-06-07 15:53:43 -07:00
act_pedit.c net sched actions: aggregate dumping of actions timeinfo 2016-06-07 15:53:43 -07:00
act_police.c act_police: rename tcf_act_police_locate() to tcf_act_police_init() 2016-06-15 00:05:57 -07:00
act_simple.c net_sched: make tcf_hash_check() boolean 2016-06-15 12:43:35 -07:00
act_skbedit.c net_sched: make tcf_hash_check() boolean 2016-06-15 12:43:35 -07:00
act_vlan.c net_sched: make tcf_hash_check() boolean 2016-06-15 12:43:35 -07:00
cls_api.c net, cls: also reject deleting all filters when TCA_KIND present 2016-06-16 22:50:16 -07:00
cls_basic.c
cls_bpf.c bpf: wire in data and data_end for cls_act_bpf 2016-05-06 16:01:54 -04:00
cls_cgroup.c cls_cgroup: factor out classid retrieval 2015-07-20 12:41:30 -07:00
cls_flow.c sched: cls_flow: use skb_to_full_sk() helper 2015-11-08 20:56:39 -05:00
cls_flower.c net/sched: flower: Return error when hw can't offload and skip_sw is set 2016-06-14 22:37:26 -07:00
cls_fw.c net: revert "net_sched: move tp->root allocation into fw_init()" 2015-09-24 14:33:30 -07:00
cls_route.c
cls_rsvp.c
cls_rsvp.h net_sched: convert rsvp to call tcf_exts_destroy from rcu callback 2015-08-26 11:01:45 -07:00
cls_rsvp6.c
cls_tcindex.c net_sched: convert tcindex to call tcf_exts_destroy from rcu callback 2015-08-26 11:01:44 -07:00
cls_u32.c net: cls_u32: be more strict about skip-sw flag for knodes 2016-06-08 21:43:14 -07:00
em_canid.c
em_cmp.c
em_ipset.c netfilter: x_tables: Pass struct net in xt_action_param 2015-09-18 21:58:14 +02:00
em_meta.c qdisc: constify meta_type_ops structures 2016-04-14 00:35:30 -04:00
em_nbyte.c
em_text.c
em_u32.c
ematch.c
Kconfig Support to encoding decoding skb prio on IFE action 2016-03-01 17:15:23 -05:00
Makefile Support to encoding decoding skb prio on IFE action 2016-03-01 17:15:23 -05:00
sch_api.c sched: remove NET_XMIT_POLICED 2016-06-12 22:02:11 -04:00
sch_atm.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_blackhole.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_cbq.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_choke.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_codel.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_drr.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_dsmark.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_fifo.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_fq.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_fq_codel.c net_sched: fq_codel: cache skb->truesize into skb->cb 2016-06-25 12:19:35 -04:00
sch_generic.c net_sched: generalize bulk dequeue 2016-06-25 12:19:35 -04:00
sch_gred.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_hfsc.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_hhf.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_htb.c net_sched: sch_htb: export class backlog in dumps 2016-06-25 12:19:35 -04:00
sch_ingress.c net: sched: fix tc_should_offload for specific clsact classes 2016-06-07 16:59:53 -07:00
sch_mq.c net: sched: do not acquire qdisc spinlock in qdisc/class stats dump 2016-06-07 16:37:14 -07:00
sch_mqprio.c net: sched: do not acquire qdisc spinlock in qdisc/class stats dump 2016-06-07 16:37:14 -07:00
sch_multiq.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_netem.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_pie.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_plug.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_prio.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_qfq.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_red.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_sfb.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_sfq.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_tbf.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00
sch_teql.c net_sched: drop packets after root qdisc lock is released 2016-06-25 12:19:35 -04:00