system/linux
system/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux synced 2024-10-16 16:29:20 +00:00
Code Issues Packages Projects Releases Wiki Activity
linux/arch/arm64/crypto
History
Ard Biesheuvel 5318d3db46 crypto: arm64/aes-ctr - improve tail handling 
Counter mode is a stream cipher chaining mode that is typically used
with inputs that are of arbitrarily length, and so a tail block which
is smaller than a full AES block is rule rather than exception.

The current ctr(aes) implementation for arm64 always makes a separate
call into the assembler routine to process this tail block, which is
suboptimal, given that it requires reloading of the AES round keys,
and prevents us from handling this tail block using the 5-way stride
that we use for better performance on deep pipelines.

So let's update the assembler routine so it can handle any input size,
and uses NEON permutation instructions and overlapping loads and stores
to handle the tail block. This results in a ~16% speedup for 1420 byte
blocks on cores with deep pipelines such as ThunderX2.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-01-03 08:41:37 +11:00
..
.gitignore SPDX patches for 5.7-rc1. 2020-04-03 13:12:26 -07:00
aes-ce-ccm-core.S crypto: arm64 - Use modern annotations for assembly functions 2019-12-20 14:58:35 +08:00
aes-ce-ccm-glue.c crypto: remove CRYPTO_TFM_RES_BAD_KEY_LEN 2020-01-09 11:30:53 +08:00
aes-ce-core.S crypto: arm64 - Use modern annotations for assembly functions 2019-12-20 14:58:35 +08:00
aes-ce-glue.c crypto: remove CRYPTO_TFM_RES_BAD_KEY_LEN 2020-01-09 11:30:53 +08:00
aes-ce-setkey.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
aes-ce.S arm64: crypto: Modernize names for AES function macros 2020-03-09 17:35:04 +00:00
aes-cipher-core.S crypto: arm64 - Use modern annotations for assembly functions 2019-12-20 14:58:35 +08:00
aes-cipher-glue.c crypto: arm64/aes-ce-cipher - use AES library as fallback 2019-07-26 14:58:09 +10:00
aes-glue.c crypto: arm64/aes-ctr - improve tail handling 2021-01-03 08:41:37 +11:00
aes-modes.S crypto: arm64/aes-ctr - improve tail handling 2021-01-03 08:41:37 +11:00
aes-neon.S arm64: crypto: Modernize names for AES function macros 2020-03-09 17:35:04 +00:00
aes-neonbs-core.S crypto: arm64: Use x16 with indirect branch to bti_c 2020-10-06 18:14:47 +01:00
aes-neonbs-glue.c crypto: arm/neon - memzero_explicit aes-cbc key 2020-03-20 14:36:51 +11:00
chacha-neon-core.S crypto: arm64/chacha - simplify tail block handling 2020-11-13 20:38:55 +11:00
chacha-neon-glue.c crypto: arch/lib - limit simd usage to 4k chunks 2020-04-30 15:16:59 +10:00
crct10dif-ce-core.S crypto: arm64 - Consistently enable extension 2020-04-24 17:42:16 +10:00
crct10dif-ce-glue.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
ghash-ce-core.S crypto: arm64/gcm - move authentication tag check to SIMD domain 2020-11-20 14:45:32 +11:00
ghash-ce-glue.c crypto: arm64/gcm - move authentication tag check to SIMD domain 2020-11-20 14:45:32 +11:00
Kconfig crypto: arm64/poly1305 - incorporate OpenSSL/CRYPTOGAMS NEON implementation 2019-11-17 09:02:41 +08:00
Makefile crypto: arm64/poly1305 - incorporate OpenSSL/CRYPTOGAMS NEON implementation 2019-11-17 09:02:41 +08:00
nh-neon-core.S crypto: arm64 - Use modern annotations for assembly functions 2019-12-20 14:58:35 +08:00
nhpoly1305-neon-glue.c crypto: arch/nhpoly1305 - process in explicit 4k chunks 2020-04-30 15:16:59 +10:00
poly1305-armv8.pl crypto: arm64/poly1305-neon - reorder PAC authentication with SP update 2020-11-06 14:29:11 +11:00
poly1305-core.S_shipped crypto: arm64/poly1305-neon - reorder PAC authentication with SP update 2020-11-06 14:29:11 +11:00
poly1305-glue.c crypto: hash - Use memzero_explicit() for clearing state 2020-10-30 17:35:03 +11:00
sha1-ce-core.S crypto: arm64 - Use modern annotations for assembly functions 2019-12-20 14:58:35 +08:00
sha1-ce-glue.c crypto: sha - split sha.h into sha1.h and sha2.h 2020-11-20 14:45:33 +11:00
sha2-ce-core.S crypto: arm64 - Use modern annotations for assembly functions 2019-12-20 14:58:35 +08:00
sha2-ce-glue.c crypto: sha - split sha.h into sha1.h and sha2.h 2020-11-20 14:45:33 +11:00
sha3-ce-core.S crypto: arm64 - Use modern annotations for assembly functions 2019-12-20 14:58:35 +08:00
sha3-ce-glue.c crypto: hash - Use memzero_explicit() for clearing state 2020-10-30 17:35:03 +11:00
sha256-core.S_shipped crypto: clarify licensing of OpenSSL asm code 2018-05-31 00:13:44 +08:00
sha256-glue.c crypto: sha - split sha.h into sha1.h and sha2.h 2020-11-20 14:45:33 +11:00
sha512-armv8.pl crypto: clarify licensing of OpenSSL asm code 2018-05-31 00:13:44 +08:00
sha512-ce-core.S crypto: arm64 - Use modern annotations for assembly functions 2019-12-20 14:58:35 +08:00
sha512-ce-glue.c crypto: sha - split sha.h into sha1.h and sha2.h 2020-11-20 14:45:33 +11:00
sha512-core.S_shipped crypto: clarify licensing of OpenSSL asm code 2018-05-31 00:13:44 +08:00
sha512-glue.c crypto: sha - split sha.h into sha1.h and sha2.h 2020-11-20 14:45:33 +11:00
sm3-ce-core.S crypto: arm64 - Use modern annotations for assembly functions 2019-12-20 14:58:35 +08:00
sm3-ce-glue.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
sm4-ce-core.S crypto: arm64 - Use modern annotations for assembly functions 2019-12-20 14:58:35 +08:00
sm4-ce-glue.c crypto: arm64 - convert to use crypto_simd_usable() 2019-03-22 20:57:27 +08:00