Fix broken STARTTLS when SharedMemoryKey is enabled.

OpenSSL 1.1 API patch for sendmail had a bug which
prevented sm_RSA_generate_key() function from working.
This function is used to generate a temporary RSA key
for a shared memory region used for TLS processing.
Note that 12.0 and 12.1-RELEASE include this bug.

This affects only if SM_CONF_SHM compile-time
option (enabled by default) and SharedMemoryKey
run-time option (not enabled by default) in a .cf file are
specified.  The latter corresponds to confSHARED_MEMORY_KEY in
a .mc file.

PR:		242861
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D23734
This commit is contained in:
Hiroki Sato 2020-02-27 19:40:29 +00:00
parent 5d481ad8df
commit 9b429e2192
Notes: svn2git 2020-12-20 02:59:44 +00:00
svn path=/head/; revision=358404

View file

@ -745,18 +745,15 @@ sm_RSA_generate_key(num, e)
{
RSA *rsa = NULL;
BIGNUM *bn_rsa_r4;
int rc;
bn_rsa_r4 = BN_new();
rc = BN_set_word(bn_rsa_r4, RSA_F4);
if ((bn_rsa_r4 != NULL) && BN_set_word(bn_rsa_r4, RSA_F4) && (rsa = RSA_new()) != NULL)
if ((bn_rsa_r4 != NULL) && BN_set_word(bn_rsa_r4, e) && (rsa = RSA_new()) != NULL)
{
if (!RSA_generate_key_ex(rsa, RSA_KEYLENGTH, bn_rsa_r4, NULL))
if (!RSA_generate_key_ex(rsa, num, bn_rsa_r4, NULL))
{
RSA_free(rsa);
rsa = NULL;
}
return NULL;
}
BN_free(bn_rsa_r4);
return rsa;