Tracking issue for optional io::Read/io::Write methods

[thaliaarchi]

This tracks the status of specializing optional methods in implementations of io::Read and io::Write in the standard library.

Read and Write have many methods which are provided by default, and many implementors do not override them, which can be problematic in some cases. Platform abstractions for stdio are particularly lacking; most of which do not have read_buf, so the buffer is forced to be initialized, and some platforms which support it might be missing vectored read/write. Since the standard library is general, this should be consistent.

I am working on addressing these gaps and have created this issue to track progress for myself and reviewers. I am still a new contributor to Rust, so please let me know if I have missed something about the process.

This surveys of all implementations of io::Read and io::Write in rust-lang/rust, including for private types, by searching for \b(Read|Write) for in .rs files. This should have no false negatives, as opposed to viewing references, which would omit other targets.

Additionally, I include all platform implementations of File. These define read, read_buf, read_vectored, is_read_vectored, write, write_vectored, is_write_vectored, and flush as inherent methods. Since the other methods are not currently supported with this scheme, I have marked them with N/A.

Implementation history

PRs created for this effort and some relevant PRs from earlier:

• std: Update the std::io adaptors to proxy all methods #22428
• Add vectored read and write support #58357
• Add vectored positioned I/O on Unix #89518
• Specialize some methods of io::Chain #105917
• Specialize some io::Read and io::Write methods for VecDeque<u8> and &[u8] #110608
• Specialize many implementations of Read::read_buf_exact #122393
• Improve several Read implementations #122441
• UnixStream: override read_buf #123084
• Specialize read_exact and read_buf_exact for VecDeque #132039
• Provide optional Read/Write methods for stdio #136769
• Implement read*_exact for std:io::repeat #136818
• Implement write_all_vectored for VecDeque #136819
• Implement default methods for io::Empty and io::Sink #137051
• Forward all default methods for I/O impls #137062
• Override default Write methods for cursor-like types #137107
• Implement read_buf for zkVM stdin #137349
• Implement read_buf for WASI stdin #137353
• Implement read_buf and vectored read/write for SGX stdio #137355
• Reserve before write_fmt for owned buffers #137762

Read

pub trait Read {
    fn read(&mut self, buf: &mut [u8]) -> Result<usize>;
    fn read_buf(&mut self, buf: BorrowedCursor<'_>) -> Result<()> { … }
    fn read_buf_exact(&mut self, buf: BorrowedCursor<'_>) -> Result<()> { … }
    fn read_vectored(&mut self, bufs: &mut [IoSliceMut<'_>]) -> Result<usize> { … }
    fn is_read_vectored(&self) -> bool { … } // considered with `read_vectored`
    fn read_exact(&mut self, buf: &mut [u8]) -> Result<()> { … }
    fn read_to_end(&mut self, buf: &mut Vec<u8>) -> Result<usize> { … }
    fn read_to_string(&mut self, buf: &mut String) -> Result<usize> { … }
}

impl Read for T	buf	buf_exact	vectored	exact	to_end	to_string
&mut Read	✅1	✅2	✅3 4	✅	✅5	✅5
Box<Read>	✅1	✅2	✅3 4	✅	✅6	✅6
&[u8]	✅1	✅2	✅3 4	✅	✅	✅7
VecDeque<u8>	✅	✅8		✅8	✅7	✅7
std::io::Chain<Read, Read>	✅9	✅	✅3 9	✅	✅9	✅
std::io::Take<Read>	✅1
std::io::Empty	✅	#137051	#137051	#137051	#137051	#137051
std::io::Repeat	✅1	✅10	✅3 4	✅10	✅	✅
std::io::Cursor<AsRef<[u8]>>	✅1	✅2	✅3 4	✅	✅11	✅11
std::io::BufReader<Read>	✅1	✅2	✅3 4	✅	✅	✅
std::net::TcpStream	✅		✅3 4
&std::net::TcpStream	✅		✅3 4
std::os::unix::net::UnixStream	✅12		✅3 4
&std::os::unix::net::UnixStream	✅12		✅3 4
&std::sys::net::windows::Socket13
std::io::PipeReader	✅		✅		✅
&std::io::PipeReader	✅		✅		✅
std::fs::File	✅1		✅/✅4		✅	✅
&std::fs::File	✅1		✅/✅4		✅	✅
Arc<std::fs::File>	✅		✅		✅	✅
std::sys::hermit::fs::File	#136769	N/A	✅	N/A	N/A	N/A
std::sys::solid::fs::File	✅	N/A		N/A	N/A	N/A
std::sys::unix::fs::File	✅1	N/A	✅/✅4	N/A	N/A	N/A
std::sys::unsupported::fs::File	✅1	N/A	✅	N/A	N/A	N/A
std::sys::wasi::fs::File	✅	N/A	✅/✅4	N/A	N/A	N/A
std::sys::windows::fs::File	✅1	N/A	✅/✅4	N/A	N/A	N/A
std::sys::unix::fd::FileDesc	✅1		✅/✅4
std::sys::unix::fs::CachedFileMetadata	✅		✅		✅	✅
std::sys::windows::handle::Handle	✅1		✅
&std::sys::hermit::fd::FileDesc14
std::io::Stdin	✅	✅2	✅/✅4	✅	✅	✅
&std::io::Stdin	✅	✅2	✅	✅	✅	✅
std::io::StdinLock<'_>	✅	✅2	✅/✅4	✅	✅	✅
std::io::stdio::StdinRaw	✅	#136769	✅/✅4	#136769	✅	✅
std::process::ChildStdout	✅		✅/✅4		✅
std::process::ChildStderr	✅		✅/✅4		✅11
std::sys::hermit::stdio::Stdin	#136769		✅/✅4
std::sys::sgx::stdio::Stdin	#137355		#137355
std::sys::solid::stdio::Stdin15
std::sys::teeos::stdio::Stdin15
std::sys::uefi::stdio::Stdin
std::sys::unix::stdio::Stdin	✅		✅/✅4
std::sys::unsupported::stdio::Stdin	#136769		#136769
std::sys::wasi::stdio::Stdin	✅16		✅/✅4
std::sys::windows::stdio::Stdin
std::sys::xous::stdio::Stdin15
std::sys::zkvm::stdio::Stdin	✅17		❌18
cargo::util::FileLock
cargo::util::LimitErrorReader<Read>

Write

pub trait Write {
    fn write(&mut self, buf: &[u8]) -> Result<usize>;
    fn write_vectored(&mut self, bufs: &[IoSlice<'_>]) -> Result<usize> { … }
    fn is_write_vectored(&self) -> bool { … } // considered with `write_vectored`
    fn write_all(&mut self, buf: &[u8]) -> Result<()> { … }
    fn write_all_vectored(&mut self, bufs: &mut [IoSlice<'_>]) -> Result<()> { … }
    fn write_fmt(&mut self, fmt: fmt::Arguments<'_>) -> Result<()> { … }
    fn flush(&mut self) -> Result<()>;
}

impl Write for T	vectored	all	all_vectored	fmt
&mut Write	✅3 4	✅5	✅19	✅5
Box<Write>	✅3 4	✅6	✅19	✅6
&mut [u8]	✅3 4	✅5	✅20
Vec<u8>	✅3 4	✅5	✅20	#137762
VecDeque<u8>	✅7	✅	✅20	#137762
std::io::Cursor<&mut [u8]>	✅3 4	✅20	✅20
std::io::Cursor<&mut Vec<u8>>	✅3 4	✅20	✅20	#137762
std::io::Cursor<Vec<u8>>	✅3 4	✅20	✅20	#137762
std::io::Cursor<Box<[u8]>>	✅3 4	✅20	✅20
std::io::Cursor<[u8; N]>	✅	✅20	✅20
core::io::BorrowedCursor<'_>	✅20	✅20	✅20
std::io::BufWriter<Write>	✅3 4	✅
std::io::LineWriter<Write>	✅	✅	✅	✅
std::io::…::LineWriterShim<'_, Write>	✅	✅
std::io::Empty	✅	#137051	#137051
&std::io::Empty	✅	#137051	#137051
std::io::Sink	✅3 4	#137051	#137051
&std::io::Sink	✅	#137051	#137051
std::net::TcpStream	✅3 4
&std::net::TcpStream	✅3 4
std::os::unix::net::UnixStream	✅3 4
&std::os::unix::net::UnixStream	✅3 4
std::io::PipeWriter	✅
&std::io::PipeWriter	✅
std::fs::File	✅/✅4
&std::fs::File	✅/✅4
Arc<std::fs::File>	✅
std::sys::hermit::fs::File	✅	N/A	N/A	N/A
std::sys::solid::fs::File		N/A	N/A	N/A
std::sys::unix::fs::File	✅	N/A	N/A	N/A
std::sys::unsupported::fs::File	✅	N/A	N/A	N/A
std::sys::wasi::fs::File	✅/✅4	N/A	N/A	N/A
std::sys::windows::fs::File	✅/✅4	N/A	N/A	N/A
std::sys::unix::fs::CachedFileMetadata	✅
std::io::Stdout	✅	✅/✅4	✅	✅
&std::io::Stdout	✅	✅	✅	✅
&std::io::StdoutLock<'_>	✅	✅/✅4	✅	❌21
std::io::stdio::StdoutRaw	✅	✅/✅4	✅	✅
std::io::Stderr	✅	✅/✅4	✅	✅
&std::io::Stderr	✅	✅	✅	✅
&std::io::StderrLock<'_>	✅	✅/✅4	✅	❌21
std::io::stdio::StderrRaw	✅	✅/✅4	✅	✅
std::process::ChildStdin	✅/✅4
&std::process::ChildStdin	✅
std::sys::hermit::stdio::Stdout	✅/✅4
std::sys::hermit::stdio::Stderr	✅/✅4
std::sys::sgx::stdio::Stdout	#137355
std::sys::sgx::stdio::Stderr	#137355
std::sys::sgx::abi::panic::SgxPanicOutput	TODO
std::sys::solid::stdio::Stdout	❌22
std::sys::solid::stdio::Stderr	❌22
std::sys::solid::stdio::PanicOutput	❌22
std::sys::teeos::stdio::Stdout	❌23
std::sys::teeos::stdio::Stderr	❌23
std::sys::uefi::stdio::Stdout
std::sys::uefi::stdio::Stderr
std::sys::unix::stdio::Stdout	✅/✅4
std::sys::unix::stdio::Stderr	✅/✅4
std::sys::unsupported::stdio::Stdout	#136769
std::sys::unsupported::stdio::Stderr	#136769
std::sys::wasi::stdio::Stdout	✅/✅4
std::sys::wasi::stdio::Stderr	✅/✅4
std::sys::windows::stdio::Stdout
std::sys::windows::stdio::Stderr
std::sys::xous::stdio::Stdout		#136769
std::sys::xous::stdio::Stderr		#136769
std::sys::xous::stdio::PanicWriter
std::sys::zkvm::stdio::Stdout	❌18
std::sys::zkvm::stdio::Stderr	❌18
test::console::OutputLocation<Write>	WIP
test::term::WinConsole<Write>	WIP
test::term::TerminfoTerminal<Write>	WIP
cargo::util::FileLock
proc_macro::bridge::buffer::Buffer		✅
rust-installer::compression::CombinedEncoder	❌	✅
rustc_errors::emitter::Buffy
rustc_errors::json::…::BufWriter	WIP

FileExt

Type	read_at	seek_read	read_vectored_at	read_exact_at	write_at	seek_write	write_vectored_at	write_all_at
std::fs::unix::fs::File	✅	N/A	✅24	default	✅	N/A	✅24	default
std::sys::wasi::fs::File	default	N/A	✅	default	default	N/A	✅	default
std::sys::windows::fs::File	N/A	✅	N/A	N/A	N/A	✅	N/A	N/A

Socket

Type	read	read_buf	read_vectored	recv_from	peek	peek_from	write	write_vectored
std::sys::net::hermit::Socket	✅	✅	✅/✅4	✅	✅	✅	✅	✅/✅4
std::sys::net::solid::Socket	✅	✅	✅	✅	✅	✅		✅
std::sys::net::unix::Socket	✅	✅	✅3 4	✅	✅	✅	✅	✅3 4
std::sys::net::wasip2::Socket	✅	✅	default	✅	✅	✅	private	default
std::sys::net::windows::Socket	✅	✅	✅3 4	✅	✅	✅		✅3 4
std::sys::net::sgx::{TcpStream, UdpSocket}	✅	✅	✅3 4	unsupported	unsupported (ok)	unsupported	✅	✅3 4
std::sys::net::wasip1::{TcpStream, UdpSocket}	✅	✅	✅3 4	unsupported	unsupported	unsupported	✅	✅3 4

Footnotes

1. Added in Implement most of RFC 2930, providing the ReadBuf abstraction #81156. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹² ↩¹³ ↩¹⁴

2. Added in Specialize many implementations of Read::read_buf_exact #122393. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸

3. Added in Add vectored read and write support #58357. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹² ↩¹³ ↩¹⁴ ↩¹⁵ ↩¹⁶ ↩¹⁷ ↩¹⁸ ↩¹⁹ ↩²⁰ ↩²¹ ↩²² ↩²³ ↩²⁴ ↩²⁵ ↩²⁶ ↩²⁷ ↩²⁸ ↩²⁹ ↩³⁰ ↩³¹ ↩³² ↩³³

4. Added in Add Read/Write::can_read/write_vectored #67841. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹² ↩¹³ ↩¹⁴ ↩¹⁵ ↩¹⁶ ↩¹⁷ ↩¹⁸ ↩¹⁹ ↩²⁰ ↩²¹ ↩²² ↩²³ ↩²⁴ ↩²⁵ ↩²⁶ ↩²⁷ ↩²⁸ ↩²⁹ ↩³⁰ ↩³¹ ↩³² ↩³³ ↩³⁴ ↩³⁵ ↩³⁶ ↩³⁷ ↩³⁸ ↩³⁹ ↩⁴⁰ ↩⁴¹ ↩⁴² ↩⁴³ ↩⁴⁴ ↩⁴⁵ ↩⁴⁶ ↩⁴⁷ ↩⁴⁸ ↩⁴⁹ ↩⁵⁰ ↩⁵¹ ↩⁵² ↩⁵³ ↩⁵⁴ ↩⁵⁵ ↩⁵⁶ ↩⁵⁷ ↩⁵⁸ ↩⁵⁹ ↩⁶⁰ ↩⁶¹ ↩⁶² ↩⁶³ ↩⁶⁴ ↩⁶⁵

5. Added in std: Update the std::io adaptors to proxy all methods #22428. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

6. Added in std: Stabilize portions of the io module #23010. ↩ ↩² ↩³ ↩⁴

7. Added in Specialize some io::Read and io::Write methods for VecDeque<u8> and &[u8] #110608. ↩ ↩² ↩³ ↩⁴

8. Added in Specialize read_exact and read_buf_exact for VecDeque #132039. ↩ ↩²

9. Added in Specialize some methods of io::Chain #105917. ↩ ↩² ↩³

10. Added in Implement read*_exact for std:io::repeat #136818. ↩ ↩²

11. Added in Improve several Read implementations #122441. ↩ ↩² ↩³

12. Added in UnixStream: override read_buf #123084. ↩ ↩²

13. This Read implementation is unused. I plan to remove it. ↩

14. This Read implementation is unused. I plan to remove it. ↩

15. This platform doesn't have stdin. I replace it with std::sys::unsupported::stdio::Stdin in #136769. ↩ ↩² ↩³

16. Added in Implement read_buf for WASI stdin #137353. ↩

17. Added in Implement read_buf for zkVM stdin #137349. ↩

18. zkVM has no vectored read/write syscalls. ↩ ↩² ↩³

19. Added in Forward all default methods for I/O impls #137062. ↩ ↩²

20. Added in Override default Write methods for cursor-like types #137107. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹² ↩¹³ ↩¹⁴ ↩¹⁵ ↩¹⁶

21. StdoutLock<'_> and StderrLock<'_> cannot provide write_fmt, because the user could supply a formatting implementation which uses Stdout or Stderr, leading to panicking. (See joboet's comment) ↩ ↩²

22. SOLID has no vectored version of SOLID_LOG_write. ↩ ↩² ↩³

23. TEEOS has no vectored version of KCALL_DEBUG_CMD_PUT_BYTES. ↩ ↩²

24. Added in Add vectored positioned I/O on Unix #89518. ↩ ↩²