linux/io_uring
Pavel Begunkov 846072f16e io_uring: mimimise io_cqring_wait_schedule
io_cqring_wait_schedule() is called after we started waiting on the cq
wq and set the state to TASK_INTERRUPTIBLE, for that reason we have to
constantly worry whether we has returned the state back to running or
not. Leave only quick checks in io_cqring_wait_schedule() and move the
rest including running task work to the callers. Note, we run tw in the
loop after the sched checks because of the fast path in the beginning of
the function.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/2814fabe75e2e019e7ca43ea07daa94564349805.1672916894.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2023-01-29 15:17:40 -07:00
..
advise.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
advise.h
alloc_cache.h
cancel.c io_uring/cancel: re-grab ctx mutex after finishing wait 2022-12-21 13:31:40 -07:00
cancel.h
epoll.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
epoll.h
fdinfo.c io_uring/fdinfo: include locked hash table in fdinfo output 2023-01-10 10:24:52 -07:00
fdinfo.h
filetable.c io_uring/filetable: fix file reference underflow 2022-11-25 06:54:46 -07:00
filetable.h io_uring: kill hot path fixed file bitmap debug checks 2022-10-16 17:07:53 -06:00
fs.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
fs.h
io-wq.c io_uring/io-wq: only free worker if it was allocated for creation 2023-01-08 10:39:17 -07:00
io-wq.h
io_uring.c io_uring: mimimise io_cqring_wait_schedule 2023-01-29 15:17:40 -07:00
io_uring.h io_uring: move defer tw task checks 2023-01-29 15:17:39 -07:00
kbuf.c io_uring: don't use complete_post in kbuf 2022-11-25 06:11:15 -07:00
kbuf.h io_uring: allow buffer recycling in READV 2022-09-21 10:30:43 -06:00
Makefile
msg_ring.c io_uring/msg_ring: fix remote queue to disabled ring 2023-01-20 09:49:34 -07:00
msg_ring.h io_uring: get rid of double locking 2022-12-07 06:47:13 -07:00
net.c io_uring/net: cache provided buffer group value for multishot receives 2023-01-23 07:08:08 -07:00
net.h io_uring/net: zerocopy sendmsg 2022-09-21 13:15:02 -06:00
nop.c
nop.h
notif.c io_uring/net: move mm accounting to a slower path 2022-11-21 07:38:31 -07:00
notif.h io_uring: move zc reporting from the hot path 2022-11-21 07:38:31 -07:00
opdef.c io_uring: get rid of double locking 2022-12-07 06:47:13 -07:00
opdef.h io_uring: dont remove file from msg_ring reqs 2022-12-07 06:47:13 -07:00
openclose.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
openclose.h
poll.c io_uring/poll: don't reissue in case of poll race on multishot request 2023-01-20 15:11:54 -07:00
poll.h
refs.h
rsrc.c io_uring: use tw for putting rsrc 2022-12-07 06:47:13 -07:00
rsrc.h io_uring: use tw for putting rsrc 2022-12-07 06:47:13 -07:00
rw.c io_uring: lock overflowing for IOPOLL 2023-01-13 07:32:46 -07:00
rw.h io_uring/rw: don't lose partial IO result on fail 2022-09-21 13:15:02 -06:00
slist.h
splice.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
splice.h
sqpoll.c
sqpoll.h
statx.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
statx.h
sync.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
sync.h
tctx.c io_uring: remove io_register_submitter 2022-10-07 12:25:30 -06:00
tctx.h io_uring: simplify __io_uring_add_tctx_node 2022-10-07 12:25:30 -06:00
timeout.c io_uring: ease timeout flush locking requirements 2022-12-14 08:53:35 -07:00
timeout.h io_uring: remove unused return from io_disarm_next 2022-09-21 13:15:01 -06:00
uring_cmd.c io_uring: iopoll protect complete_post 2022-11-23 10:45:31 -07:00
uring_cmd.h
xattr.c acl: conver higher-level helpers to rely on mnt_idmap 2022-10-31 17:48:12 +01:00
xattr.h