The AVX2 variant of ChaCha20 is used only for messages with >= 512 bytes
length. With the existing test vectors, the implementation could not be
tested. Due that lack of such a long official test vector, this one is
self-generated using chacha20-generic.
Signed-off-by: Martin Willi <martin@strongswan.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Extends the x86_64 ChaCha20 implementation by a function processing eight
ChaCha20 blocks in parallel using AVX2.
For large messages, throughput increases by ~55-70% compared to four block
SSSE3:
testing speed of chacha20 (chacha20-simd) encryption
test 0 (256 bit key, 16 byte blocks): 42249230 operations in 10 seconds (675987680 bytes)
test 1 (256 bit key, 64 byte blocks): 46441641 operations in 10 seconds (2972265024 bytes)
test 2 (256 bit key, 256 byte blocks): 33028112 operations in 10 seconds (8455196672 bytes)
test 3 (256 bit key, 1024 byte blocks): 11568759 operations in 10 seconds (11846409216 bytes)
test 4 (256 bit key, 8192 byte blocks): 1448761 operations in 10 seconds (11868250112 bytes)
testing speed of chacha20 (chacha20-simd) encryption
test 0 (256 bit key, 16 byte blocks): 41999675 operations in 10 seconds (671994800 bytes)
test 1 (256 bit key, 64 byte blocks): 45805908 operations in 10 seconds (2931578112 bytes)
test 2 (256 bit key, 256 byte blocks): 32814947 operations in 10 seconds (8400626432 bytes)
test 3 (256 bit key, 1024 byte blocks): 19777167 operations in 10 seconds (20251819008 bytes)
test 4 (256 bit key, 8192 byte blocks): 2279321 operations in 10 seconds (18672197632 bytes)
Benchmark results from a Core i5-4670T.
Signed-off-by: Martin Willi <martin@strongswan.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Implements an x86_64 assembler driver for the ChaCha20 stream cipher. This
single block variant works on a single state matrix using SSE instructions.
It requires SSSE3 due the use of pshufb for efficient 8/16-bit rotate
operations.
For large messages, throughput increases by ~65% compared to
chacha20-generic:
testing speed of chacha20 (chacha20-generic) encryption
test 0 (256 bit key, 16 byte blocks): 45089207 operations in 10 seconds (721427312 bytes)
test 1 (256 bit key, 64 byte blocks): 43839521 operations in 10 seconds (2805729344 bytes)
test 2 (256 bit key, 256 byte blocks): 12702056 operations in 10 seconds (3251726336 bytes)
test 3 (256 bit key, 1024 byte blocks): 3371173 operations in 10 seconds (3452081152 bytes)
test 4 (256 bit key, 8192 byte blocks): 422468 operations in 10 seconds (3460857856 bytes)
testing speed of chacha20 (chacha20-simd) encryption
test 0 (256 bit key, 16 byte blocks): 43141886 operations in 10 seconds (690270176 bytes)
test 1 (256 bit key, 64 byte blocks): 46845874 operations in 10 seconds (2998135936 bytes)
test 2 (256 bit key, 256 byte blocks): 18458512 operations in 10 seconds (4725379072 bytes)
test 3 (256 bit key, 1024 byte blocks): 5360533 operations in 10 seconds (5489185792 bytes)
test 4 (256 bit key, 8192 byte blocks): 692846 operations in 10 seconds (5675794432 bytes)
Benchmark results from a Core i5-4670T.
Signed-off-by: Martin Willi <martin@strongswan.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
As architecture specific drivers need a software fallback, export a
ChaCha20 en-/decryption function together with some helpers in a header
file.
Signed-off-by: Martin Willi <martin@strongswan.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Adds individual ChaCha20 and Poly1305 and a combined rfc7539esp AEAD speed
test using mode numbers 214, 321 and 213. For Poly1305 we add a specific
speed template, as it expects the key prepended to the input data.
Signed-off-by: Martin Willi <martin@strongswan.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch converts rfc7539 and rfc7539esp to the new AEAD interface.
The test vectors for rfc7539esp have also been updated to include
the IV.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Tested-by: Martin Willi <martin@strongswan.org>
Introduce constrains for RSA keys lengths.
Only key lengths of 512, 1024, 1536, 2048, 3072, and 4096 bits
will be supported.
Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch disables the rfc4309 test while the conversion to the
new seqiv calling convention takes place. It also replaces the
rfc4309 test vectors with ones that will work with the new IV
convention.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch converts rfc4106 to the new calling convention where
the IV is now part of the AD and needs to be skipped. This patch
also makes use of the new type-safe way of freeing instances.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch allows the AEAD speed tests to cope with the new seqiv
calling convention as well as the old one.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch disables the rfc4106 test while the conversion to the
new seqiv calling convention takes place. It also converts the
rfc4106 test vectors to the new format.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch replaces the seqniv generator with seqiv when the
underlying algorithm understands the new calling convention.
This not only makes more sense as now seqiv is solely responsible
for IV generation rather than also determining how the IV is going
to be used, it also allows for optimisations in the underlying
implementation. For example, the space for the IV could be used
to add padding for authentication.
This patch also removes the unnecessary copying of IV to dst
during seqiv decryption as the IV is part of the AD and not cipher
text.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch fixes a bug where we were incorrectly including the
IV in the AD during encryption. The IV must remain in the plain
text for it to be encrypted.
During decryption there is no need to copy the IV to dst because
it's now part of the AD.
This patch removes an unncessary check on authsize which would be
performed by the underlying decrypt call.
Finally this patch makes use of the type-safe init/exit functions.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch allows the CRYPTO_ALG_AEAD_NEW flag to be propagated.
It also restores the ASYNC bit that went missing during the AEAD
conversion.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds a type-safe function for freeing AEAD instances
to struct aead_instance. This replaces the existing free function
in struct crypto_template which does not know the type of the
instance that it's freeing.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Currently the task of freeing an instance is given to the crypto
template. However, it has no type information on the instance so
we have to resort to checking type information at runtime.
This patch introduces a free function to crypto_type that will be
used to free an instance. This can then be used to free an instance
in a type-safe manner.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The AEAD speed tests doesn't do a wait_for_completition,
if the return value is EINPROGRESS or EBUSY.
Fixing it here.
Also add a test case for gcm(aes).
Signed-off-by: Lokesh Vutla <lokeshvutla@ti.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
All tests for cbc(aes) use only blocks of data with a multiple of 4.
This test adds a test with some odd SG size.
Signed-off-by: LABBE Corentin <clabbe.montjoie@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The AEAD version of cryptd uses the same context for its own state
as well as that of the child. In doing so it did not maintain the
proper ordering, thus resulting in potential state corruption where
the child will overwrite the state stored by cryptd.
This patch fixes and also sets the request size properly.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAABAgAGBQJVkO2uAAoJEOvOhAQsB9HWOT0P/jvFrpj2iuWqdMcvBuMdlx6K
/9qiMsOStbxmbmjj3RsbFCkeHJMUBCI0ZVrIosdEyeZWx33fHkZJisvN1i1PMA63
qppcZtkGxSmCOi6+e9k+qZNBvKBWy2oCXyLx7zaUKkWuR7wGe9488+9dqd9x+/gu
i/HTvf8Omrukwko2a0omWUSmUimhveb/hQ7Vxc/M9lbxFeO3jMabV5ZzgfnoTTrh
Rd2zo4kGbhj8nxonCUHgamnk6hoTE3KYhBPvqohzTDSTALmZVxhCwaQzQWzq+kdl
oSLa+tik508/csh98tN9dlMkJReHlDNdJIbfazQ1vHI00T5PsQELexFT02iKBhS7
5mdHSPp5P1TgGB6Fk3lN+hVO6Ja0S/vmJUi72M2y5DPd55lnvOaCVuqzWIJTYOoN
tqllBH4WIz27hsJWiMLgkJQaDxfBFYx104tOq56s5SeOthBluSE2xoNDrzZGyvRh
NeVza4ccgxIj0p2xfgeErx89r4GSCNk/LakpcMJReaT7ri23mTCDZJNLMcVW7BYm
2MW6M3LF748eN3P2YyNnU+TeQNpIho4whuwfOV+uR4tpdd5MtMaObWimwxBN7URM
LeW3gIwsZFHxYU9NLeZoQZVi6gDmaVe7ma82AbHXaV/mQVYsSP9M6gCO+FASCTGt
Rz6Nyl4/Ns8rdXUoOud8
=qotW
-----END PGP SIGNATURE-----
Merge tag 'module-implicit-v4.1-rc8' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux
Pull implicit module.h fixes from Paul Gortmaker:
"Fix up implicit <module.h> users that will break later.
The files changed here are simply modular source files that are
implicitly relying on <module.h> being present. We fix them up now,
so that we can decouple some of the module related init code from the
core init code in the future.
The addition of the module.h include to several files here is also a
no-op from a code generation point of view, else there would already
be compile issues with these files today.
There may be lots more implicit includes of <module.h> in tree, but
these are the ones that extensive build test coverage has shown that
must be fixed in order to avoid build breakage fallout for the pending
module.h <---> init.h code relocation we desire to complete"
* tag 'module-implicit-v4.1-rc8' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux:
frv: add module.h to mb93090-mb00/flash.c to avoid compile fail
drivers/cpufreq: include <module.h> for modular exynos-cpufreq.c code
drivers/staging: include <module.h> for modular android tegra_ion code
crypto/asymmetric_keys: pkcs7_key_type needs module.h
sh: mach-highlander/psw.c is tristate and should use module.h
drivers/regulator: include <module.h> for modular max77802 code
drivers/pcmcia: include <module.h> for modular xxs1500_ss code
drivers/hsi: include <module.h> for modular omap_ssi code
drivers/gpu: include <module.h> for modular rockchip code
drivers/gpio: include <module.h> for modular crystalcove code
drivers/clk: include <module.h> for clk-max77xxx modular code
Pull security subsystem updates from James Morris:
"The main change in this kernel is Casey's generalized LSM stacking
work, which removes the hard-coding of Capabilities and Yama stacking,
allowing multiple arbitrary "small" LSMs to be stacked with a default
monolithic module (e.g. SELinux, Smack, AppArmor).
See
https://lwn.net/Articles/636056/
This will allow smaller, simpler LSMs to be incorporated into the
mainline kernel and arbitrarily stacked by users. Also, this is a
useful cleanup of the LSM code in its own right"
* 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security: (38 commits)
tpm, tpm_crb: fix le64_to_cpu conversions in crb_acpi_add()
vTPM: set virtual device before passing to ibmvtpm_reset_crq
tpm_ibmvtpm: remove unneccessary message level.
ima: update builtin policies
ima: extend "mask" policy matching support
ima: add support for new "euid" policy condition
ima: fix ima_show_template_data_ascii()
Smack: freeing an error pointer in smk_write_revoke_subj()
selinux: fix setting of security labels on NFS
selinux: Remove unused permission definitions
selinux: enable genfscon labeling for sysfs and pstore files
selinux: enable per-file labeling for debugfs files.
selinux: update netlink socket classes
signals: don't abuse __flush_signals() in selinux_bprm_committed_creds()
selinux: Print 'sclass' as string when unrecognized netlink message occurs
Smack: allow multiple labels in onlycap
Smack: fix seq operations in smackfs
ima: pass iint to ima_add_violation()
ima: wrap event related data to the new ima_event_data structure
integrity: add validity checks for 'path' parameter
...
There are two generated files: crypto/rsakey-asn1.c and crypto/raskey-asn1.h,
after the cfc2bb32b3 commit. Let's add
.gitignore to ignore *-asn1.[ch] files.
Signed-off-by: Alexander Kuleshov <kuleshovmail@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
arm64:allmodconfig fails to build as follows.
In file included from include/acpi/platform/aclinux.h:74:0,
from include/acpi/platform/acenv.h:173,
from include/acpi/acpi.h:56,
from include/linux/acpi.h:37,
from ./arch/arm64/include/asm/dma-mapping.h:21,
from include/linux/dma-mapping.h:86,
from include/linux/skbuff.h:34,
from include/crypto/algapi.h:18,
from crypto/asymmetric_keys/rsa.c:16:
include/linux/ctype.h:15:12: error: expected ‘;’, ‘,’ or ‘)’
before numeric constant
#define _X 0x40 /* hex digit */
^
crypto/asymmetric_keys/rsa.c:123:47: note: in expansion of macro ‘_X’
static int RSA_I2OSP(MPI x, size_t xLen, u8 **_X)
^
crypto/asymmetric_keys/rsa.c: In function ‘RSA_verify_signature’:
crypto/asymmetric_keys/rsa.c:256:2: error:
implicit declaration of function ‘RSA_I2OSP’
The problem is caused by an unrelated include file change, resulting in
the inclusion of ctype.h on arm64. This in turn causes the local variable
_X to conflict with macro _X used in ctype.h.
Fixes: b6197b93fa ("arm64 : Introduce support for ACPI _CCA object")
Cc: Suthikulpanit, Suravee <Suravee.Suthikulpanit@amd.com>
Signed-off-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Don't print info about missing test for the internal
helper __driver-gcm-aes-aesni
changes in v2:
- marked test as fips allowed
Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The kzfree() function tests whether its argument is NULL and then
returns immediately. Thus the test around the call is not needed.
This issue was detected by using the Coccinelle software.
Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The core of the Jitter RNG is intended to be compiled with -O0. To
ensure that the Jitter RNG can be compiled on all architectures,
separate out the RNG core into a stand-alone C file that can be compiled
with -O0 which does not depend on any kernel include file.
As no kernel includes can be used in the C file implementing the core
RNG, any dependencies on kernel code must be extracted.
A second file provides the link to the kernel and the kernel crypto API
that can be compiled with the regular compile options of the kernel.
Signed-off-by: Stephan Mueller <smueller@chronox.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Pull networking updates from David Miller:
1) Add TX fast path in mac80211, from Johannes Berg.
2) Add TSO/GRO support to ibmveth, from Thomas Falcon
3) Move away from cached routes in ipv6, just like ipv4, from Martin
KaFai Lau.
4) Lots of new rhashtable tests, from Thomas Graf.
5) Run ingress qdisc lockless, from Alexei Starovoitov.
6) Allow servers to fetch TCP packet headers for SYN packets of new
connections, for fingerprinting. From Eric Dumazet.
7) Add mode parameter to pktgen, for testing receive. From Alexei
Starovoitov.
8) Cache access optimizations via simplifications of build_skb(), from
Alexander Duyck.
9) Move page frag allocator under mm/, also from Alexander.
10) Add xmit_more support to hv_netvsc, from KY Srinivasan.
11) Add a counter guard in case we try to perform endless reclassify
loops in the packet scheduler.
12) Extern flow dissector to be programmable and use it in new "Flower"
classifier. From Jiri Pirko.
13) AF_PACKET fanout rollover fixes, performance improvements, and new
statistics. From Willem de Bruijn.
14) Add netdev driver for GENEVE tunnels, from John W Linville.
15) Add ingress netfilter hooks and filtering, from Pablo Neira Ayuso.
16) Fix handling of epoll edge triggers in TCP, from Eric Dumazet.
17) Add an ECN retry fallback for the initial TCP handshake, from Daniel
Borkmann.
18) Add tail call support to BPF, from Alexei Starovoitov.
19) Add several pktgen helper scripts, from Jesper Dangaard Brouer.
20) Add zerocopy support to AF_UNIX, from Hannes Frederic Sowa.
21) Favor even port numbers for allocation to connect() requests, and
odd port numbers for bind(0), in an effort to help avoid
ip_local_port_range exhaustion. From Eric Dumazet.
22) Add Cavium ThunderX driver, from Sunil Goutham.
23) Allow bpf programs to access skb_iif and dev->ifindex SKB metadata,
from Alexei Starovoitov.
24) Add support for T6 chips in cxgb4vf driver, from Hariprasad Shenai.
25) Double TCP Small Queues default to 256K to accomodate situations
like the XEN driver and wireless aggregation. From Wei Liu.
26) Add more entropy inputs to flow dissector, from Tom Herbert.
27) Add CDG congestion control algorithm to TCP, from Kenneth Klette
Jonassen.
28) Convert ipset over to RCU locking, from Jozsef Kadlecsik.
29) Track and act upon link status of ipv4 route nexthops, from Andy
Gospodarek.
* git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next: (1670 commits)
bridge: vlan: flush the dynamically learned entries on port vlan delete
bridge: multicast: add a comment to br_port_state_selection about blocking state
net: inet_diag: export IPV6_V6ONLY sockopt
stmmac: troubleshoot unexpected bits in des0 & des1
net: ipv4 sysctl option to ignore routes when nexthop link is down
net: track link-status of ipv4 nexthops
net: switchdev: ignore unsupported bridge flags
net: Cavium: Fix MAC address setting in shutdown state
drivers: net: xgene: fix for ACPI support without ACPI
ip: report the original address of ICMP messages
net/mlx5e: Prefetch skb data on RX
net/mlx5e: Pop cq outside mlx5e_get_cqe
net/mlx5e: Remove mlx5e_cq.sqrq back-pointer
net/mlx5e: Remove extra spaces
net/mlx5e: Avoid TX CQE generation if more xmit packets expected
net/mlx5e: Avoid redundant dev_kfree_skb() upon NOP completion
net/mlx5e: Remove re-assignment of wq type in mlx5e_enable_rq()
net/mlx5e: Use skb_shinfo(skb)->gso_segs rather than counting them
net/mlx5e: Static mapping of netdev priv resources to/from netdev TX queues
net/mlx4_en: Use HW counters for rx/tx bytes/packets in PF device
...
As the AEAD conversion is still ongoing, we do not yet wish to
export legacy AEAD implementations to user-space, as their calling
convention will change.
This patch actually disables all AEAD algorithms because some of
them (e.g., cryptd) will need to be modified to propagate this flag.
Subsequent patches will reenable them on an individual basis.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The bit CRYPTO_ALG_INTERNAL was added to stop af_alg from accessing
internal algorithms. However, af_alg itself was never modified to
actually stop that bit from being used by the user. Therefore the
user could always override it by specifying the relevant bit in the
type and/or mask.
This patch silently discards the bit in both type and mask.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch changes the RNG allocation so that we only hold a
reference to the RNG during initialisation.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When seqiv is used in compatibility mode, this patch allows it
to function even when an RNG Is not available. It also changes
the RNG allocation for the new explicit seqiv interface so that
we only hold a reference to the RNG during initialisation.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The RNG may not be available during early boot, e.g., the relevant
modules may not be included in the initramfs. As the RNG Is only
needed for IPsec, we should not let this prevent use of ciphers
without IV generators, e.g., for disk encryption.
This patch postpones the RNG allocation to the init function so
that one failure during early boot does not make the RNG unavailable
for all subsequent users of the same cipher.
More importantly, it lets the cipher live even if RNG allocation
fails. Of course we no longer offer IV generation and which will
fail with an error if invoked. But all other cipher capabilities
will function as usual.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The RNG may not be available during early boot, e.g., the relevant
modules may not be included in the initramfs. As the RNG Is only
needed for IPsec, we should not let this prevent use of ciphers
without IV generators, e.g., for disk encryption.
This patch postpones the RNG allocation to the init function so
that one failure during early boot does not make the RNG unavailable
for all subsequent users of the same cipher.
More importantly, it lets the cipher live even if RNG allocation
fails. Of course we no longer offer IV generation and which will
fail with an error if invoked. But all other cipher capabilities
will function as usual.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds a new crypto_user command that allows the admin to
delete the crypto system RNG. Note that this can only be done if
the RNG is currently not in use. The next time it is used a new
system RNG will be allocated.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Currently we free the default RNG when its use count hits zero.
This was OK when the IV generators would latch onto the RNG at
instance creation time and keep it until the instance is torn
down.
Now that IV generators only keep the RNG reference during init
time this scheme causes the default RNG to come and go at a high
frequencey. This is highly undesirable as we want to keep a single
RNG in use unless the admin wants it to be removed.
This patch changes the scheme so that the system RNG once allocated
is never removed unless a specifically requested.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Currently for skcipher IV generators they must provide givencrypt
as that is the whole point. We are currently replacing skcipher
IV generators with explicit IV generators. In order to maintain
backwards compatibility, we need to allow the IV generators to
still function as a normal skcipher when the RNG Is not present
(e.g., in the initramfs during boot). IOW everything but givencrypt
and givdecrypt will still work but those two will fail.
Therefore this patch assigns a default givencrypt that simply
returns an error should it be NULL.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Should be CRYPTO_AKCIPHER instead of AKCIPHER
Reported-by: Andreas Ruprecht <andreas.ruprecht@fau.de>
Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Merge the mvebu/drivers branch of the arm-soc tree which contains
just a single patch bfa1ce5f38 ("bus:
mvebu-mbus: add mv_mbus_dram_info_nooverlap()") that happens to be
a prerequisite of the new marvell/cesa crypto driver.
Add a new rsa generic SW implementation.
This implements only cryptographic primitives.
Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com>
Added select on ASN1.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add Public Key Encryption API.
Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com>
Made CRYPTO_AKCIPHER invisible like other type config options.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
