linux/net/sched
Daniel Borkmann d2788d3488 net: sched: further simplify handle_ing
Ingress qdisc has no other purpose than calling into tc_classify()
that executes attached classifier(s) and action(s).

It has a 1:1 relationship to dev->ingress_queue. After having commit
087c1a601a ("net: sched: run ingress qdisc without locks") removed
the central ingress lock, one major contention point is gone.

The extra indirection layers however, are not necessary for calling
into ingress qdisc. pktgen calling locally into netif_receive_skb()
with a dummy u32, single CPU result on a Supermicro X10SLM-F, Xeon
E3-1240: before ~21,1 Mpps, after patch ~22,9 Mpps.

We can redirect the private classifier list to the netdev directly,
without changing any classifier API bits (!) and execute on that from
handle_ing() side. The __QDISC_STATE_DEACTIVATE test can be removed,
ingress qdisc doesn't have a queue and thus dev_deactivate_queue()
is also not applicable, ingress_cl_list provides similar behaviour.
In other words, ingress qdisc acts like TCQ_F_BUILTIN qdisc.

One next possible step is the removal of the dev's ingress (dummy)
netdev_queue, and to only have the list member in the netdevice
itself.

Note, the filter chain is RCU protected and individual filter elements
are being kfree'd by sched subsystem after RCU grace period. RCU read
lock is being held by __netif_receive_skb_core().

Joint work with Alexei Starovoitov.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Alexei Starovoitov <ast@plumgrid.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2015-05-11 11:10:35 -04:00
..
act_api.c net: sched: remove TC_MUNGED bits 2015-05-02 22:25:17 -04:00
act_bpf.c bpf: fix bpf helpers to use skb->mac_header relative offsets 2015-04-16 14:08:49 -04:00
act_connmark.c net: sched: act_connmark: don't zap skb->nfct 2015-04-29 14:56:40 -04:00
act_csum.c net: sched: fix skb->protocol use in case of accelerated vlan path 2015-01-13 17:51:08 -05:00
act_gact.c sched: fix act file names in header comment 2014-11-06 15:04:41 -05:00
act_ipt.c sched: fix act file names in header comment 2014-11-06 15:04:41 -05:00
act_mirred.c act_mirred: Fix bogus header when redirecting from VLAN 2015-04-17 13:29:28 -04:00
act_nat.c net_sched: act: move tcf_hashinfo_init() into tcf_register_action() 2014-02-12 19:23:32 -05:00
act_pedit.c net: sched: remove TC_MUNGED bits 2015-05-02 22:25:17 -04:00
act_police.c sched: fix act file names in header comment 2014-11-06 15:04:41 -05:00
act_simple.c sched: fix act file names in header comment 2014-11-06 15:04:41 -05:00
act_skbedit.c net_sched: act: move tcf_hashinfo_init() into tcf_register_action() 2014-02-12 19:23:32 -05:00
act_vlan.c sched: introduce vlan action 2014-11-21 14:20:18 -05:00
cls_api.c net_sched: destroy proto tp when all filters are gone 2015-03-09 15:35:55 -04:00
cls_basic.c net_sched: destroy proto tp when all filters are gone 2015-03-09 15:35:55 -04:00
cls_bpf.c bpf: fix bpf helpers to use skb->mac_header relative offsets 2015-04-16 14:08:49 -04:00
cls_cgroup.c net_sched: destroy proto tp when all filters are gone 2015-03-09 15:35:55 -04:00
cls_flow.c net_sched: destroy proto tp when all filters are gone 2015-03-09 15:35:55 -04:00
cls_fw.c net_sched: destroy proto tp when all filters are gone 2015-03-09 15:35:55 -04:00
cls_route.c net_sched: destroy proto tp when all filters are gone 2015-03-09 15:35:55 -04:00
cls_rsvp.c
cls_rsvp.h net_sched: destroy proto tp when all filters are gone 2015-03-09 15:35:55 -04:00
cls_rsvp6.c
cls_tcindex.c net_sched: destroy proto tp when all filters are gone 2015-03-09 15:35:55 -04:00
cls_u32.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2015-03-20 18:51:09 -04:00
em_canid.c net: sched: remove tcf_proto from ematch calls 2014-10-06 18:02:32 -04:00
em_cmp.c
em_ipset.c net: sched: fix skb->protocol use in case of accelerated vlan path 2015-01-13 17:51:08 -05:00
em_meta.c net: rename vlan_tx_* helpers since "tx" is misleading there 2015-01-13 17:51:08 -05:00
em_nbyte.c net: sched: remove tcf_proto from ematch calls 2014-10-06 18:02:32 -04:00
em_text.c net: Remove state argument from skb_find_text() 2015-02-22 15:59:54 -05:00
em_u32.c
ematch.c ematch: Fix auto-loading of ematch modules. 2015-02-20 15:30:56 -05:00
Kconfig Merge branch 'kconfig' of git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild 2015-02-19 10:36:45 -08:00
Makefile net: sched: Introduce connmark action 2015-01-19 16:02:06 -05:00
sch_api.c net_sched: destroy proto tp when all filters are gone 2015-03-09 15:35:55 -04:00
sch_atm.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_blackhole.c
sch_cbq.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_choke.c sch_choke: Use flow_keys_digest 2015-05-04 00:09:09 -04:00
sch_codel.c codel: add ce_threshold attribute 2015-05-10 19:50:20 -04:00
sch_drr.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_dsmark.c net: sched: fix skb->protocol use in case of accelerated vlan path 2015-01-13 17:51:08 -05:00
sch_fifo.c net: sched: implement qstat helper routines 2014-09-30 01:02:26 -04:00
sch_fq.c pkt_sched: fq: correct spelling of locally 2015-04-01 22:52:29 -04:00
sch_fq_codel.c codel: add ce_threshold attribute 2015-05-10 19:50:20 -04:00
sch_generic.c net_sched: restore qdisc quota fairness limits after bulk dequeue 2014-10-09 19:12:26 -04:00
sch_gred.c net: sched: implement qstat helper routines 2014-09-30 01:02:26 -04:00
sch_hfsc.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_hhf.c sched: Call skb_get_hash_perturb in sch_hhf 2015-05-04 00:09:08 -04:00
sch_htb.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_ingress.c net: sched: further simplify handle_ing 2015-05-11 11:10:35 -04:00
sch_mq.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_mqprio.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_multiq.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_netem.c netem: Fixes byte backlog accounting for the first of two chained netem instances 2015-04-07 18:34:24 -04:00
sch_pie.c sch_pie: schedule the timer after all init succeed 2014-10-29 14:28:01 -04:00
sch_plug.c
sch_prio.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_qfq.c net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
sch_red.c net: sched: implement qstat helper routines 2014-09-30 01:02:26 -04:00
sch_sfb.c sched: Call skb_get_hash_perturb in sch_sfb 2015-05-04 00:09:08 -04:00
sch_sfq.c sched: Call skb_get_hash_perturb in sch_sfq 2015-05-04 00:09:08 -04:00
sch_tbf.c net: sched: avoid costly atomic operation in fq_dequeue() 2014-10-06 00:55:10 -04:00
sch_teql.c net: sched: fix skb->protocol use in case of accelerated vlan path 2015-01-13 17:51:08 -05:00