DeflateStream, GZipStream, and CryptoStream handling of partial and zero-byte reads

[stephentoub]

DeflateStream, GZipStream, and CryptoStream handling of partial and zero-byte reads

DeflateStream, GZipStream, and CryptoStream diverged from typical Stream.Read and Stream.ReadAsync behavior in two ways. First, they did not complete the operation until either the buffer passed to the read operation was completely filled or the end of the stream was reached, and second, as wrapper streams they didn't delegate zero-length buffer functionality to the stream they wrap. Both of these issues have been addressed.

Version introduced

.NET 6.0

Old behavior

When Stream.Read or Stream.ReadAsync was called on one of these streams with a buffer of length N, the operation would not complete until N bytes had been read from the stream or until the underlying stream they wrap returned 0 from a call to its read, indicating no more data is available.

Also, when Stream.Read or Stream.ReadAsync was called on one of these streams with a buffer of length 0, the operation would succeed immediately, sometimes without doing a zero-length read on the stream it wraps.

New behavior

When Stream.Read or Stream.ReadAsync is called on one of these streams with a buffer of length N, the operation will complete once at least one byte has been read from the stream or once the underlying stream they wrap returns 0 from a call to its read, indicating no more data is available.

Also, when Stream.Read or Stream.ReadAsync is called on one of these streams with a buffer of length 0, the operation will succeed once a call with a non-zero buffer would succeed.

Reason for change

The streams might not have returned from a read operation even if data had been successfully read, which meant they couldn't readily be used in any bidirectional communication situation where messages smaller than the buffer size were being used. This could lead to deadlocks in an application, unable to read the data from the stream necessary to continue the operation. It could also lead to arbitrary slowdowns, with the consumer unable to process available data while waiting for additional data to arrive.

Also, it's common in highly scalable applications to use zero-byte reads as a way of delaying buffer allocation until a buffer is needed. An application can issue a read with an empty buffer, and when that read completes, data should soon be available to consume, such that the application can then issue the read again, this time with a buffer to receive the data. By delegating to the wrapped stream if no already decompressed or transformed data is available, these streams now inherit any such behavior of the streams they wrap.

Recommended action

If an application depends on the buffer being completely filled before progressing, it can perform the read in a loop to regain the behavior.

int totalRead = 0;
while (totalRead < buffer.Length)
{
    int bytesRead = stream.Read(buffer.Slice(totalRead));
    if (bytesRead == 0) break;
    totalRead += bytesRead;
}

In general, code should not make any assumptions about a Stream.Read or ReadAsync operation reading as much as was requested. The call returns the number of bytes read, which may be less than what was requested.

If an application depends on a zero-byte read completing immediately without waiting, it can check the buffer length itself and skip the call entirely:

int bytesRead = 0;
if (!buffer.IsEmpty)
{
    bytesRead = stream.Read(buffer);
}

In general, code should expect that a Stream.Read or ReadAsync call may not complete until at least a byte of data is available for consumption (or the stream reaches its end), regardless of how many bytes were requested.

Category

• ASP.NET Core
• C#
• Code analysis
• Core .NET libraries
• Cryptography
• Data
• Debugger
• Deployment
• Globalization
• Interop
• JIT
• LINQ
• Managed Extensibility Framework (MEF)
• MSBuild
• Networking
• Printing
• SDK
• Security
• Serialization
• Visual Basic
• Windows Forms
• Windows Presentation Foundation (WPF)
• XML, XSLT

Affected APIs

DeflateStream.Read
DeflateStream.ReadAsync
DeflateStream.BeginRead

GZipStream.Read
GZipStream.ReadAsync
GZipStream.BeginRead

CryptoStream.Read
CryptoStream.ReadAsync
CryptoStream.BeginRead

---

Issue metadata

• Issue type: breaking-change

[danmoseley]

linking related
dotnet/runtime#53502
dotnet/runtime#53644