cli: tunnels can sporadically become unresponsive (#181285)

Last iteration I moved some RPC logic to use Tokios "codecs" to give
them cancellation safety. These operate on streams of input data.

While this worked at first, I failed to take into account that the byte
buffer we read from the stream could have _more_ data than just the
current message under load scenarios. We were discarding any extra data
from the subsequent message. In most cases caused the next message
"length" to be read from the middle of the next message, which usually
(when parsed as a u32) was some number of gigabytes, then causing the
connection to stall forever.

Fixes #181284
This commit is contained in:
Connor Peet 2023-05-01 17:34:11 -07:00 committed by GitHub
parent 368167ff7f
commit 06ff8779bd
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23

View file

@ -3,6 +3,7 @@
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
use bytes::Buf;
use tokio::{
io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt, BufReader},
pin,
@ -124,8 +125,8 @@ impl tokio_util::codec::Decoder for U32PrefixedCodec {
return Ok(None);
}
let msg = src[U32_SIZE..].to_vec();
src.resize(0, 0);
let msg = src[U32_SIZE..required_len].to_vec();
src.advance(required_len);
Ok(Some(msg))
}
}