system/linux
system/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux synced 2024-10-16 00:10:42 +00:00
Code Issues Packages Projects Releases Wiki Activity
linux/crypto
History
Ondrej Mosnacek 78105c7e76 crypto: xts - Drop use of auxiliary buffer 
Since commit acb9b159c7 ("crypto: gf128mul - define gf128mul_x_* in
gf128mul.h"), the gf128mul_x_*() functions are very fast and therefore
caching the computed XTS tweaks has only negligible advantage over
computing them twice.

In fact, since the current caching implementation limits the size of
the calls to the child ecb(...) algorithm to PAGE_SIZE (usually 4096 B),
it is often actually slower than the simple recomputing implementation.

This patch simplifies the XTS template to recompute the XTS tweaks from
scratch in the second pass and thus also removes the need to allocate a
dynamic buffer using kmalloc().

As discussed at [1], the use of kmalloc causes deadlocks with dm-crypt.

PERFORMANCE RESULTS
I measured time to encrypt/decrypt a memory buffer of varying sizes with
xts(ecb-aes-aesni) using a tool I wrote ([2]) and the results suggest
that after this patch the performance is either better or comparable for
both small and large buffers. Note that there is a lot of noise in the
measurements, but the overall difference is easy to see.

Old code:
       ALGORITHM KEY (b)        DATA (B)   TIME ENC (ns)   TIME DEC (ns)
        xts(aes)     256              64             331             328
        xts(aes)     384              64             332             333
        xts(aes)     512              64             338             348
        xts(aes)     256             512             889             920
        xts(aes)     384             512            1019             993
        xts(aes)     512             512            1032             990
        xts(aes)     256            4096            2152            2292
        xts(aes)     384            4096            2453            2597
        xts(aes)     512            4096            3041            2641
        xts(aes)     256           16384            9443            8027
        xts(aes)     384           16384            8536            8925
        xts(aes)     512           16384            9232            9417
        xts(aes)     256           32768           16383           14897
        xts(aes)     384           32768           17527           16102
        xts(aes)     512           32768           18483           17322

New code:
       ALGORITHM KEY (b)        DATA (B)   TIME ENC (ns)   TIME DEC (ns)
        xts(aes)     256              64             328             324
        xts(aes)     384              64             324             319
        xts(aes)     512              64             320             322
        xts(aes)     256             512             476             473
        xts(aes)     384             512             509             492
        xts(aes)     512             512             531             514
        xts(aes)     256            4096            2132            1829
        xts(aes)     384            4096            2357            2055
        xts(aes)     512            4096            2178            2027
        xts(aes)     256           16384            6920            6983
        xts(aes)     384           16384            8597            7505
        xts(aes)     512           16384            7841            8164
        xts(aes)     256           32768           13468           12307
        xts(aes)     384           32768           14808           13402
        xts(aes)     512           32768           15753           14636

[1] https://lkml.org/lkml/2018/8/23/1315
[2] https://gitlab.com/omos/linux-crypto-bench

Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2018-09-21 13:24:50 +08:00
..
asymmetric_keys Replace magic for trusting the secondary keyring with #define 2018-08-16 09:57:20 -07:00
async_tx async_pq: Remove VLA usage 2018-06-18 20:17:38 +05:30
842.c crypto: acomp - add support for 842 via scomp 2016-10-25 11:08:33 +08:00
ablkcipher.c crypto: ablkcipher - fix crash flushing dcache in error path 2018-08-03 18:06:04 +08:00
acompress.c crypto: acomp - allow registration of multiple acomps 2017-04-21 20:30:50 +08:00
aead.c crypto: aead - prevent using AEADs without setting key 2018-01-12 23:03:39 +11:00
aegis.h crypto: aegis - Add generic AEGIS AEAD implementations 2018-05-19 00:13:58 +08:00
aegis128.c crypto: aead - remove useless setting of type flags 2018-07-09 00:30:26 +08:00
aegis128l.c crypto: aead - remove useless setting of type flags 2018-07-09 00:30:26 +08:00
aegis256.c crypto: aead - remove useless setting of type flags 2018-07-09 00:30:26 +08:00
aes_generic.c crypto: aes-generic - drop alignment requirement 2017-02-11 17:50:43 +08:00
aes_ti.c crypto: aes_ti - fix comment for MixColumns step 2017-06-19 14:11:53 +08:00
af_alg.c net: simplify sock_poll_wait 2018-07-30 09:10:25 -07:00
ahash.c crypto: hash - Remove VLA usage 2018-09-04 11:35:03 +08:00
akcipher.c crypto: Replaced gcc specific attributes with macros from compiler.h 2017-01-13 00:24:39 +08:00
algapi.c crypto: api - Introduce notifier for new crypto algorithms 2018-09-04 11:37:04 +08:00
algboss.c crypto: api - Introduce notifier for new crypto algorithms 2018-09-04 11:37:04 +08:00
algif_aead.c Revert changes to convert to ->poll_mask() and aio IOCB_CMD_POLL 2018-06-28 10:40:47 -07:00
algif_hash.c crypto: hash - Remove VLA usage 2018-09-04 11:35:03 +08:00
algif_rng.c net: remove sock_no_poll 2018-05-26 09:16:44 +02:00
algif_skcipher.c Revert changes to convert to ->poll_mask() and aio IOCB_CMD_POLL 2018-06-28 10:40:47 -07:00
ansi_cprng.c
anubis.c
api.c evm: Don't deadlock if a crypto algorithm is unavailable 2018-07-18 07:27:22 -04:00
arc4.c
authenc.c crypto: authenc - don't leak pointers to authenc keys 2018-04-21 00:58:30 +08:00
authencesn.c crypto: authencesn - don't leak pointers to authenc keys 2018-04-21 00:58:30 +08:00
blkcipher.c crypto: blkcipher - fix crash flushing dcache in error path 2018-08-03 18:06:04 +08:00
blowfish_common.c
blowfish_generic.c
camellia_generic.c crypto: replace FSF address with web source in license notices 2017-11-29 17:33:25 +11:00
cast5_generic.c crypto: replace FSF address with web source in license notices 2017-11-29 17:33:25 +11:00
cast6_generic.c crypto: replace FSF address with web source in license notices 2017-11-29 17:33:25 +11:00
cast_common.c
cbc.c crypto: cbc - Propagate NEED_FALLBACK bit 2017-03-09 18:34:39 +08:00
ccm.c crypto: ccm - Remove VLA usage 2018-09-04 11:35:03 +08:00
cfb.c crypto: remove several VLAs 2018-04-21 00:58:34 +08:00
chacha20_generic.c crypto: chacha20 - Fix keystream alignment for chacha20_block() 2017-11-29 17:33:33 +11:00
chacha20poly1305.c crypto: chacha20poly1305 - validate the digest size 2017-12-22 19:02:33 +11:00
cipher.c crypto: remove several VLAs 2018-04-21 00:58:34 +08:00
cmac.c crypto: algapi - make crypto_xor() and crypto_inc() alignment agnostic 2017-02-11 17:52:28 +08:00
compress.c crypto: api - Remove no-op exit_ops code 2016-10-21 11:03:42 +08:00
crc32_generic.c crypto: crc32-generic - remove __crc32_le() 2018-05-27 00:12:09 +08:00
crc32c_generic.c crypto: crc32c-generic - remove cra_alignmask 2018-05-27 00:12:08 +08:00
crct10dif_common.c
crct10dif_generic.c crypto: squash lines for simple wrapper functions 2016-09-13 20:27:26 +08:00
cryptd.c crypto: hash - annotate algorithms taking optional key 2018-01-12 23:03:35 +11:00
crypto_engine.c crypto: engine - Permit to enqueue all async requests 2018-02-15 23:26:50 +08:00
crypto_null.c crypto: shash - remove useless setting of type flags 2018-07-09 00:30:24 +08:00
crypto_user.c crypto: user - Replace GFP_ATOMIC with GFP_KERNEL in crypto_report 2018-02-15 23:26:47 +08:00
crypto_wq.c
ctr.c crypto: remove several VLAs 2018-04-21 00:58:34 +08:00
cts.c crypto: remove several VLAs 2018-04-21 00:58:34 +08:00
deflate.c crypto: scomp - add support for deflate rfc1950 (zlib) 2017-04-24 18:11:08 +08:00
des_generic.c
dh.c crypto: dh - fix memory leak 2018-07-20 13:51:21 +08:00
dh_helper.c crypto: dh - make crypto_dh_encode_key() make robust 2018-08-03 18:06:06 +08:00
drbg.c crypto: drbg - in-place cipher operation for CTR 2018-08-03 18:05:48 +08:00
ecb.c
ecc.c crypto: ecdh - add public key verification test 2018-07-09 00:26:19 +08:00
ecc.h crypto: ecc - Actually remove stack VLA usage 2018-04-21 00:58:29 +08:00
ecc_curve_defs.h crypto: ecdh - fix typo of P-192 b value 2018-07-20 13:51:22 +08:00
ecdh.c crypto: ecc - Actually remove stack VLA usage 2018-04-21 00:58:29 +08:00
ecdh_helper.c crypto: ecdh - return unsigned value for crypto_ecdh_key_len() 2017-10-12 22:55:00 +08:00
echainiv.c crypto: echainiv - Remove unused alg/spawn variable 2017-12-22 19:52:45 +11:00
fcrypt.c
fips.c
gcm.c crypto: null - Get rid of crypto_{get,put}_default_null_skcipher2() 2017-12-22 19:29:08 +11:00
gf128mul.c crypto: gf128mul - remove incorrect comment 2017-12-22 19:52:40 +11:00
ghash-generic.c crypto: shash - remove useless setting of type flags 2018-07-09 00:30:24 +08:00
hash_info.c
hmac.c crypto: hmac - require that the underlying hash algorithm is unkeyed 2017-11-29 13:39:15 +11:00
internal.h crypto: api - Introduce notifier for new crypto algorithms 2018-09-04 11:37:04 +08:00
jitterentropy-kcapi.c crypto: jitterentropy - drop duplicate header module.h 2016-11-17 23:34:52 +08:00
jitterentropy.c
Kconfig crypto: x86 - remove SHA multibuffer routines and mcryptd 2018-09-04 11:37:04 +08:00
keywrap.c crypto: keywrap - Add missing ULL suffixes for 64-bit constants 2017-11-29 17:33:26 +11:00
khazad.c
kpp.c crypto: Replaced gcc specific attributes with macros from compiler.h 2017-01-13 00:24:39 +08:00
lrw.c crypto: scatterwalk - remove 'chain' argument from scatterwalk_crypto_chain() 2018-08-03 18:06:03 +08:00
lz4.c crypto: lz4 - fixed decompress function to return error code 2017-04-10 19:17:27 +08:00
lz4hc.c crypto: lz4 - fixed decompress function to return error code 2017-04-10 19:17:27 +08:00
lzo.c treewide: use kv[mz]alloc* rather than opencoded variants 2017-05-08 17:15:13 -07:00
Makefile crypto: x86 - remove SHA multibuffer routines and mcryptd 2018-09-04 11:37:04 +08:00
md4.c crypto: shash - remove useless setting of type flags 2018-07-09 00:30:24 +08:00
md5.c crypto: shash - remove useless setting of type flags 2018-07-09 00:30:24 +08:00
memneq.c
michael_mic.c
morus640.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux 2018-08-03 17:55:12 +08:00
morus1280.c crypto: aead - remove useless setting of type flags 2018-07-09 00:30:26 +08:00
pcbc.c crypto: remove several VLAs 2018-04-21 00:58:34 +08:00
pcrypt.c crypto: pcrypt - fix freeing pcrypt instances 2017-12-22 19:02:47 +11:00
poly1305_generic.c crypto: shash - remove useless setting of type flags 2018-07-09 00:30:24 +08:00
proc.c proc: introduce proc_create_seq{,_data} 2018-05-16 07:23:35 +02:00
ripemd.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
rmd128.c crypto: shash - remove useless setting of type flags 2018-07-09 00:30:24 +08:00
rmd160.c crypto: shash - remove useless setting of type flags 2018-07-09 00:30:24 +08:00
rmd256.c crypto: rmd256 - use swap macro in rmd256_transform 2018-07-27 19:28:36 +08:00
rmd320.c crypto: rmd320 - use swap macro in rmd320_transform 2018-07-27 19:28:36 +08:00
rng.c crypto: rng - ensure that the RNG is ready before using 2017-07-28 17:56:00 +08:00
rsa-pkcs1pad.c crypto: rsa-pkcs1pad - Replace GFP_ATOMIC with GFP_KERNEL in pkcs1pad_encrypt_sign_complete 2018-02-15 23:26:47 +08:00
rsa.c crypto: rsa - Remove unneeded error assignment 2018-04-21 00:58:37 +08:00
rsa_helper.c kbuild: rename *-asn1.[ch] to *.asn1.[ch] 2018-04-07 19:04:02 +09:00
rsaprivkey.asn1 crypto: rsa - Store rest of the private key components 2016-07-05 23:05:26 +08:00
rsapubkey.asn1
salsa20_generic.c crypto: salsa20 - Revert "crypto: salsa20 - export generic helpers" 2018-05-31 00:13:57 +08:00
scatterwalk.c crypto: scatterwalk - remove 'chain' argument from scatterwalk_crypto_chain() 2018-08-03 18:06:03 +08:00
scompress.c crypto: scompress - use sgl_alloc() and sgl_free() 2018-01-06 09:18:00 -07:00
seed.c
seqiv.c crypto: seqiv - Remove unused alg/spawn variable 2017-12-22 19:52:45 +11:00
serpent_generic.c crypto: serpent - improve __serpent_setkey with UBSAN 2017-08-09 20:17:54 +08:00
sha1_generic.c crypto: shash - remove useless setting of type flags 2018-07-09 00:30:24 +08:00
sha3_generic.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux 2018-08-03 17:55:12 +08:00
sha256_generic.c crypto: shash - remove useless setting of type flags 2018-07-09 00:30:24 +08:00
sha512_generic.c crypto: shash - remove useless setting of type flags 2018-07-09 00:30:24 +08:00
shash.c crypto: shash - Remove VLA usage in unaligned hashing 2018-09-04 11:37:03 +08:00
simd.c crypto: simd - allow registering multiple algorithms at once 2018-03-03 00:03:17 +08:00
skcipher.c crypto: skcipher - fix crash flushing dcache in error path 2018-08-03 18:06:04 +08:00
sm3_generic.c crypto: shash - remove useless setting of type flags 2018-07-09 00:30:24 +08:00
sm4_generic.c crypto: sm4 - export encrypt/decrypt routines to other drivers 2018-05-05 14:52:51 +08:00
tcrypt.c crypto: tcrypt - reschedule during speed tests 2018-08-03 18:06:02 +08:00
tcrypt.h
tea.c
testmgr.c crypto: speck - remove Speck 2018-09-04 11:35:03 +08:00
testmgr.h crypto: speck - remove Speck 2018-09-04 11:35:03 +08:00
tgr192.c crypto: shash - remove useless setting of type flags 2018-07-09 00:30:24 +08:00
twofish_common.c crypto: replace FSF address with web source in license notices 2017-11-29 17:33:25 +11:00
twofish_generic.c crypto: replace FSF address with web source in license notices 2017-11-29 17:33:25 +11:00
vmac.c crypto: vmac - remove insecure version with hardcoded nonce 2018-07-01 21:00:44 +08:00
wp512.c crypto: shash - remove useless setting of type flags 2018-07-09 00:30:24 +08:00
xcbc.c crypto: xcbc - Remove VLA usage 2018-09-04 11:35:03 +08:00
xor.c kmemcheck: stop using GFP_NOTRACK and SLAB_NOTRACK 2017-11-15 18:21:04 -08:00
xts.c crypto: xts - Drop use of auxiliary buffer 2018-09-21 13:24:50 +08:00
zstd.c crypto: zstd - Add zstd support 2018-04-21 00:58:30 +08:00