system/linux
system/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux synced 2024-10-07 20:05:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

riscv: vector: make Vector always available for softirq context

Browse source 
The goal of this patch is to provide full support of Vector in kernel
softirq context. So that some of the crypto alogrithms won't need scalar
fallbacks.

By disabling bottom halves in active kernel-mode Vector, softirq will
not be able to nest on top of any kernel-mode Vector. So, softirq
context is able to use Vector whenever it runs.

After this patch, Vector context cannot start with irqs disabled.
Otherwise local_bh_enable() may run in a wrong context.

Disabling bh is not enough for RT-kernel to prevent preeemption. So
we must disable preemption, which also implies disabling bh on RT.

Related-to: commit 696207d425 ("arm64/sve: Make kernel FPU protection RT friendly")
Related-to: commit 66c3ec5a71 ("arm64: neon: Forbid when irqs are disabled")
Signed-off-by: Andy Chiu <andy.chiu@sifive.com>
Reviewed-by: Eric Biggers <ebiggers@google.com>
Tested-by: Björn Töpel <bjorn@rivosinc.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Link: https://lore.kernel.org/r/20240115055929.4736-3-andy.chiu@sifive.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
This commit is contained in:
Andy Chiu 2024-01-15 05:59:21 +00:00 committed by Palmer Dabbelt
parent ecd2ada8a5
commit 956895b9d8
No known key found for this signature in database
GPG key ID: 2E1319F35FBB1889
3 changed files with 19 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
arch/riscv/include/asm/processor.h
View file

@ -78,7 +78,8 @@ struct pt_regs;
* following meaning:
*
* - bit 0: indicates whether the in-kernel Vector context is active. The
* activation of this state disables the preemption.
* activation of this state disables the preemption. On a non-RT kernel, it
* also disable bh.
*/
#define RISCV_KERNEL_MODE_V 0x1

6
arch/riscv/include/asm/simd.h
View file

@ -28,8 +28,12 @@ static __must_check inline bool may_use_simd(void)
/*
* RISCV_KERNEL_MODE_V is only set while preemption is disabled,
* and is clear whenever preemption is enabled.
*
* Kernel-mode Vector temporarily disables bh. So we must not return
* true on irq_disabled(). Otherwise we would fail the lockdep check
* calling local_bh_enable()
*/
return !in_hardirq() && !in_nmi() && !(riscv_v_flags() & RISCV_KERNEL_MODE_V);
return !in_hardirq() && !in_nmi() && !irqs_disabled() && !(riscv_v_flags() & RISCV_KERNEL_MODE_V);
}
#else /* ! CONFIG_RISCV_ISA_V */

14
arch/riscv/kernel/kernel_mode_vector.c
View file

@ -46,7 +46,14 @@ static inline void riscv_v_stop(u32 flags)
*/
void get_cpu_vector_context(void)
{
preempt_disable();
/*
* disable softirqs so it is impossible for softirqs to nest
* get_cpu_vector_context() when kernel is actively using Vector.
*/
if (!IS_ENABLED(CONFIG_PREEMPT_RT))
local_bh_disable();
else
preempt_disable();
riscv_v_start(RISCV_KERNEL_MODE_V);
}
@ -62,7 +69,10 @@ void put_cpu_vector_context(void)
{
riscv_v_stop(RISCV_KERNEL_MODE_V);
preempt_enable();
if (!IS_ENABLED(CONFIG_PREEMPT_RT))
local_bh_enable();
else
preempt_enable();
}
/*