ossl: Fix some bugs in the fallback AES-GCM implementation

gcm_*_aesni() are used when the AVX512 implementation is not available.
Fix two bugs which manifest when handling operations spanning multiple
segments:
- Avoid underflow when the length of the input is smaller than the
  residual.
- In gcm_decrypt_aesni(), ensure that we begin the operation at the
  right offset into the input and output buffers.

Reviewed by:	jhb
Fixes:		9b1d87286c ("ossl: Add a fallback AES-GCM implementation using AES-NI")
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D42838
This commit is contained in:
Mark Johnston 2023-11-29 15:08:12 -05:00
parent 0fac350c54
commit 47d767dab5

View file

@ -459,7 +459,7 @@ gcm_encrypt_aesni(struct ossl_gcm_context *ctx, const unsigned char *in,
size_t bulk = 0, res;
int error;
res = (AES_BLOCK_LEN - ctx->gcm.mres) % AES_BLOCK_LEN;
res = MIN(len, (AES_BLOCK_LEN - ctx->gcm.mres) % AES_BLOCK_LEN);
if ((error = gcm_encrypt(ctx, in, out, res)) != 0)
return error;
@ -621,12 +621,12 @@ gcm_decrypt_aesni(struct ossl_gcm_context *ctx, const unsigned char *in,
size_t bulk = 0, res;
int error;
res = (AES_BLOCK_LEN - ctx->gcm.mres) % AES_BLOCK_LEN;
res = MIN(len, (AES_BLOCK_LEN - ctx->gcm.mres) % AES_BLOCK_LEN);
if ((error = gcm_decrypt(ctx, in, out, res)) != 0)
return error;
bulk = aesni_gcm_decrypt(in, out, len, &ctx->aes_ks, ctx->gcm.Yi.c,
ctx->gcm.Xi.u);
bulk = aesni_gcm_decrypt(in + res, out + res, len - res, &ctx->aes_ks,
ctx->gcm.Yi.c, ctx->gcm.Xi.u);
ctx->gcm.len.u[1] += bulk;
bulk += res;