-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add isAsync overload for Create/Open/etc. #24698
Comments
Btw is there still a big requirement to have Filestreams by default not open for asynchronous operations? |
I agree that we should make it easier to open|create files for async IO. Especially since we want to invest in
The
I like Since public partial class StreamReader : System.IO.TextReader
{
public StreamReader(string path)
+ public StreamReader(string path, System.IO.FileOptions options)
public StreamReader(string path, bool detectEncodingFromByteOrderMarks)
+ public StreamReader(string path, bool detectEncodingFromByteOrderMarks,System.IO. FileOptions options)
public StreamReader(string path, System.Text.Encoding encoding)
+ public StreamReader(string path, System.Text.Encoding encoding, System.IO.FileOptions options)
public StreamReader(string path, System.Text.Encoding encoding, bool detectEncodingFromByteOrderMarks)
+ public StreamReader(string path, System.Text.Encoding encoding, bool detectEncodingFromByteOrderMarks, System.IO.FileOptions options)
public StreamReader(string path, System.Text.Encoding encoding, bool detectEncodingFromByteOrderMarks, int bufferSize)
+ public StreamReader(string path, System.Text.Encoding encoding, bool detectEncodingFromByteOrderMarks, int bufferSize, System.IO.FileOptions options)
} public partial class StreamWriter : System.IO.TextWriter
{
public StreamWriter(string path)
+ public StreamWriter(string path, System.IO.FileOptions options)
public StreamWriter(string path, bool append)
+ public StreamWriter(string path, bool append, System.IO.FileOptions options)
public StreamWriter(string path, bool append, System.Text.Encoding encoding)
+ public StreamWriter(string path, bool append, System.Text.Encoding encoding, System.IO.FileOptions options)
public StreamWriter(string path, bool append, System.Text.Encoding encoding, int bufferSize)
+ public StreamWriter(string path, bool append, System.Text.Encoding encoding, int bufferSize, System.IO.FileOptions options)
} This should allow for: public static partial class File
{
public static System.IO.FileStream Create(string path)
+ public static System.IO.FileStream Create(string path, System.IO.FileOptions options)
public static System.IO.FileStream Create(string path, int bufferSize)
public static System.IO.FileStream Create(string path, int bufferSize, System.IO.FileOptions options)
public static System.IO.StreamWriter CreateText(string path)
+ public static System.IO.StreamWriter CreateText(string path, System.IO.FileOptions options)
public static System.IO.FileStream Open(string path, System.IO.FileMode mode)
+ public static System.IO.FileStream Open(string path, System.IO.FileMode mode, System.IO.FileOptions options)
public static System.IO.FileStream Open(string path, System.IO.FileMode mode, System.IO.FileAccess access)
+ public static System.IO.FileStream Open(string path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileOptions options)
public static System.IO.FileStream Open(string path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share)
+ public static System.IO.FileStream Open(string path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.IO.FileOptions options)
public static System.IO.FileStream OpenRead(string path) { throw null; }
+ public static System.IO.FileStream OpenRead(string path) { throw null; }
public static System.IO.StreamReader OpenText(string path) { throw null; }
+ public static System.IO.StreamReader OpenText(string path) { throw null; }
public static System.IO.FileStream OpenWrite(string path) { throw null; }
+ public static System.IO.FileStream OpenWrite(string path) { throw null; }
} note that public sealed partial class FileInfo : System.IO.FileSystemInfo
{
public System.IO.FileStream Create()
+ public System.IO.FileStream Create(System.IO.FileOptions options)
public System.IO.StreamWriter CreateText()
+ public System.IO.StreamWriter CreateText(System.IO.FileOptions options)
public System.IO.FileStream Open(System.IO.FileMode mode)
+ public System.IO.FileStream Open(System.IO.FileMode mode, System.IO.FileOptions options)
public System.IO.FileStream Open(System.IO.FileMode mode, System.IO.FileAccess access)
+ public System.IO.FileStream Open(System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileOptions options)
public System.IO.FileStream Open(System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share)
+ public System.IO.FileStream Open(System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.IO.FileOptions options)
public System.IO.FileStream OpenRead()
+ public System.IO.FileStream OpenRead(System.IO.FileOptions options)
public System.IO.StreamReader OpenText()
+ public System.IO.StreamReader OpenText(System.IO.FileOptions options)
public System.IO.FileStream OpenWrite()
+ public System.IO.FileStream OpenWrite(System.IO.FileOptions options)
} @Duranom would the proposed API meet your needs? @carlossanlop @jozkee since this task aligns with our .NET 6.0 goals, I am setting the 6.0 milestone for it |
Yes, think that will help with the misconception.
Just wondering how many use this and set the buffersize to 4096 (the default), think the |
@Duranom good point! thank you for feedback! @Duranom some time ago we have added an official template for API proposal. Is there any chance that you could update your issue description and fill the template? Then I would mark it as ready for review, introduce it in .NET Design Review Meeting and once we get the API approved, we can start coding and ship it ;) |
Updated, apologies for delay |
@Duranom thank you! |
Did you consider including public static class File
{
public StreamWriter AppendText(string path) { throw null; }
+ public StreamWriter AppendText(string path, FileOptions options) { throw null; }
}
public class FileInfo
{
public StreamWriter AppendText() { throw null; }
+ public StreamWriter AppendText(FileOptions options) { throw null; }
} |
@adamsitnik I have a suggestion and a question:
|
I've missed it. I've added it after reading your comment. 👍
done
Few things that come to my mind:
private FileStream CreateFileStream(FileStreamParameters parameters)
{
EnsureIsAsyncSetToTrue(parameters); // method modifies a copy
return new FileStream(parameters); // uses a different copy, where IsAsync might not be set to true
}
private static void EnsureIsAsyncSetToTrue(FileStreamParameters parameters)
{
parameters.IsAsync = true;
} |
I question whether this is actually a beneficial thing to do. Even with all of the work being invested in FileStream this release (which I'm of course very glad we're doing, having helped spearhead it :), do you have some example scenarios where you've found using FileStreams created with FileOptions.Asynchronous yields better end-to-end performance? I expect it's primarily meaningful when accessing remote files, which isn't a super common use case. The proposed helpers are all convenience, not actually enabling anything that couldn't otherwise be done with one or two more lines of code (if that), so adding these means we want to encourage more devs to use the options with File/FileInfo/StreamWriter/StreamReader.
So we're adding even more? 😉 |
@stephentoub From the examples that I've seen it's very common for the users to open the file for Some of them: https://twitter.com/buhakmeh/status/1375497431962488838 The main point of adding these overloads is just to make it easier for the end-users to use truly async file IO. |
I'm not disputing that. But before you add a bazillion overloads, prove it's actually beneficial in practice, not just in theory. You might be surprised. We're not talking here about making something that wasn't possible now possible, we're talking about making something just a smidgen easier, by adding ~30 methods. |
Looks good as proposed namespace System.IO
{
public static partial class File
{
public StreamWriter AppendText(string path, FileOptions options) => throw null;
public static System.IO.FileStream Create(string path, System.IO.FileOptions options) => throw null;
public static System.IO.StreamWriter CreateText(string path, System.IO.FileOptions options) => throw null;
public static System.IO.FileStream Open(string path, System.IO.FileMode mode, System.IO.FileOptions options) => throw null;
public static System.IO.FileStream Open(string path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileOptions options) => throw null;
public static System.IO.FileStream Open(string path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.IO.FileOptions options) => throw null;
public static System.IO.FileStream OpenRead(string path, System.IO.FileOptions options) => throw null; { throw null; }
public static System.IO.StreamReader OpenText(string path, System.IO.FileOptions options) => throw null; { throw null; }
public static System.IO.FileStream OpenWrite(string path, System.IO.FileOptions options) => throw null; { throw null; }
}
public sealed partial class FileInfo : System.IO.FileSystemInfo
{
public StreamWriter AppendText(FileOptions options) => throw null;
public System.IO.FileStream Create(System.IO.FileOptions options) => throw null;
public System.IO.StreamWriter CreateText(System.IO.FileOptions options) => throw null;
public System.IO.FileStream Open(System.IO.FileMode mode, System.IO.FileOptions options) => throw null;
public System.IO.FileStream Open(System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileOptions options) => throw null;
public System.IO.FileStream Open(System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.IO.FileOptions options) => throw null;
public System.IO.FileStream OpenRead(System.IO.FileOptions options) => throw null;
public System.IO.StreamReader OpenText(System.IO.FileOptions options) => throw null;
public System.IO.FileStream OpenWrite(System.IO.FileOptions options) => throw null;
}
public partial class StreamReader : System.IO.TextReader
{
public StreamReader(string path, System.IO.FileOptions options) => throw null;
public StreamReader(string path, bool detectEncodingFromByteOrderMarks, System.IO.FileOptions options) => throw null;
public StreamReader(string path, System.Text.Encoding encoding, System.IO.FileOptions options) => throw null;
public StreamReader(string path, System.Text.Encoding encoding, bool detectEncodingFromByteOrderMarks, System.IO.FileOptions options) => throw null;
public StreamReader(string path, System.Text.Encoding encoding, bool detectEncodingFromByteOrderMarks, int bufferSize, System.IO.FileOptions options) => throw null;
}
public partial class StreamWriter : System.IO.TextWriter
{
public StreamWriter(string path, System.IO.FileOptions options) => throw null;
public StreamWriter(string path, bool append, System.IO.FileOptions options) => throw null;
public StreamWriter(string path, bool append, System.Text.Encoding encoding, System.IO.FileOptions options) => throw null;
public StreamWriter(string path, bool append, System.Text.Encoding encoding, int bufferSize, System.IO.FileOptions options) => throw null;
}
} |
I have created a PR (#52720) to implement these new overloads. |
Wouldn't it be simpler / more flexible for these overloads to be revised now that #52446 has been approved? e.g. we shouldn't have three new overloads of Open... if we want more functionality there, why wouldn't we just add one that takes the options bag? |
@stephentoub that is our plan, but specifying preallocation size and the buffer is just higher on the priority list |
I'm confused. I think Steve's asking/suggesting that these new overloads not be added now, since the the functionality is present in the options bag one. So if it's the plan, the PR and this issue would get closed. Unless the plan is do the work for this, then when adding the options bag remove these overloads, but that sounds like a more complicated plan. |
By plan, I meant re-designing the API accepted in #24698 (comment) by adopting the new namespace System.IO
{
public static partial class File
{
- public StreamWriter AppendText(string path, FileOptions options)
+ public StreamWriter AppendText(string path, FileStreamOptions options)
- public static FileStream Create(string path, FileOptions options)
+ public static FileStream Create(string path, FileStreamOptions options)
- public static StreamWriter CreateText(string path, FileOptions options)
+ public static StreamWriter CreateText(string path, FileStreamOptions options)
- public static FileStream Open(string path, FileMode mode, FileOptions options)
- public static FileStream Open(string path, FileMode mode, FileAccess access, FileOptions options)
- public static FileStream Open(string path, FileMode mode, FileAccess access, FileShare share, FileOptions options)
+ public static FileStream Open(string path, FileMode mode, FileStreamOptions options)
- public static FileStream OpenRead(string path, FileOptions options)
+ public static FileStream OpenRead(string path, FileStreamOptions options)
- public static StreamReader OpenText(string path, FileOptions options)
+ public static StreamReader OpenText(string path, FileStreamOptions options)
- public static FileStream OpenWrite(string path, FileOptions options)
+ public static FileStream OpenWrite(string path, FileStreamOptions options)
}
public sealed partial class FileInfo : FileSystemInfo
{
- public StreamWriter AppendText(FileOptions options)
+ public StreamWriter AppendText(FileStreamOptions options)
- public FileStream Create(FileOptions options)
+ public FileStream Create(FileStreamOptions options)
- public StreamWriter CreateText(FileOptions options)
+ public StreamWriter CreateText(FileStreamOptions options)
- public FileStream Open(FileMode mode, FileOptions options)
- public FileStream Open(FileMode mode, FileAccess access, FileOptions options)
- public FileStream Open(FileMode mode, FileAccess access, FileShare share, FileOptions options)
+ public FileStream Open(FileMode mode, FileStreamOptions options)
- public FileStream OpenRead(FileOptions options)
+ public FileStream OpenRead(FileStreamOptions options)
- public StreamReader OpenText(FileOptions options)
+ public StreamReader OpenText(FileStreamOptions options)
- public FileStream OpenWrite(FileOptions options)
+ public FileStream OpenWrite(FileStreamOptions options)
}
public partial class StreamReader : TextReader
{
- public StreamReader(string path, FileOptions options)
+ public StreamReader(string path, FileStreamOptions options)
- public StreamReader(string path, bool detectEncodingFromByteOrderMarks, FileOptions options)
- public StreamReader(string path, System.Text.Encoding encoding, FileOptions options)
- public StreamReader(string path, System.Text.Encoding encoding, bool detectEncodingFromByteOrderMarks, FileOptions options)
- public StreamReader(string path, System.Text.Encoding encoding, bool detectEncodingFromByteOrderMarks, int bufferSize, FileOptions options)
+ public StreamReader(string path, System.Text.Encoding encoding, bool detectEncodingFromByteOrderMarks, FileStreamOptions options)
}
public partial class StreamWriter : TextWriter
{
- public StreamWriter(string path, FileOptions options)
+ public StreamWriter(string path, FileStreamOptions options)
- public StreamWriter(string path, bool append, FileOptions options)
- public StreamWriter(string path, bool append, System.Text.Encoding encoding, FileOptions options)
- public StreamWriter(string path, bool append, System.Text.Encoding encoding, int bufferSize, FileOptions options)
+ public StreamWriter(string path, System.Text.Encoding encoding, FileStreamOptions options)
}
} |
Trusting that all of the removals are not yet implemented, or were to be new for .NET 6, the update looks good as proposed. namespace System.IO
{
public static partial class File
{
- public StreamWriter AppendText(string path, FileOptions options)
+ public StreamWriter AppendText(string path, FileStreamOptions options)
- public static FileStream Create(string path, FileOptions options)
+ public static FileStream Create(string path, FileStreamOptions options)
- public static StreamWriter CreateText(string path, FileOptions options)
+ public static StreamWriter CreateText(string path, FileStreamOptions options)
- public static FileStream Open(string path, FileMode mode, FileOptions options)
- public static FileStream Open(string path, FileMode mode, FileAccess access, FileOptions options)
- public static FileStream Open(string path, FileMode mode, FileAccess access, FileShare share, FileOptions options)
+ public static FileStream Open(string path, FileMode mode, FileStreamOptions options)
- public static FileStream OpenRead(string path, FileOptions options)
+ public static FileStream OpenRead(string path, FileStreamOptions options)
- public static StreamReader OpenText(string path, FileOptions options)
+ public static StreamReader OpenText(string path, FileStreamOptions options)
- public static FileStream OpenWrite(string path, FileOptions options)
+ public static FileStream OpenWrite(string path, FileStreamOptions options)
}
public sealed partial class FileInfo : FileSystemInfo
{
- public StreamWriter AppendText(FileOptions options)
+ public StreamWriter AppendText(FileStreamOptions options)
- public FileStream Create(FileOptions options)
+ public FileStream Create(FileStreamOptions options)
- public StreamWriter CreateText(FileOptions options)
+ public StreamWriter CreateText(FileStreamOptions options)
- public FileStream Open(FileMode mode, FileOptions options)
- public FileStream Open(FileMode mode, FileAccess access, FileOptions options)
- public FileStream Open(FileMode mode, FileAccess access, FileShare share, FileOptions options)
+ public FileStream Open(FileMode mode, FileStreamOptions options)
- public FileStream OpenRead(FileOptions options)
+ public FileStream OpenRead(FileStreamOptions options)
- public StreamReader OpenText(FileOptions options)
+ public StreamReader OpenText(FileStreamOptions options)
- public FileStream OpenWrite(FileOptions options)
+ public FileStream OpenWrite(FileStreamOptions options)
}
public partial class StreamReader : TextReader
{
- public StreamReader(string path, FileOptions options)
+ public StreamReader(string path, FileStreamOptions options)
- public StreamReader(string path, bool detectEncodingFromByteOrderMarks, FileOptions options)
- public StreamReader(string path, System.Text.Encoding encoding, FileOptions options)
- public StreamReader(string path, System.Text.Encoding encoding, bool detectEncodingFromByteOrderMarks, FileOptions options)
- public StreamReader(string path, System.Text.Encoding encoding, bool detectEncodingFromByteOrderMarks, int bufferSize, FileOptions options)
+ public StreamReader(string path, System.Text.Encoding encoding, bool detectEncodingFromByteOrderMarks, FileStreamOptions options)
}
public partial class StreamWriter : TextWriter
{
- public StreamWriter(string path, FileOptions options)
+ public StreamWriter(string path, FileStreamOptions options)
- public StreamWriter(string path, bool append, FileOptions options)
- public StreamWriter(string path, bool append, System.Text.Encoding encoding, FileOptions options)
- public StreamWriter(string path, bool append, System.Text.Encoding encoding, int bufferSize, FileOptions options)
+ public StreamWriter(string path, System.Text.Encoding encoding, FileStreamOptions options)
}
} |
When reviewing #52720 I've realized that I've made a mistake in the re-design proposal (#24698 (comment)) that got approved. There is no need for - public static FileStream Open(string path, FileMode mode, FileStreamOptions options)
+ public static FileStream Open(string path, FileStreamOptions options) @bartonjs can I get this change accepted without presenting it in the review meeting? |
Makes sense to me, but you probably want As the video freely admits, we didn't really look at this issue very much. It was "we approved some new overloads before, this is simplifying them for the new options type, LGTM". As long as you stay in that same principle (not making a breaking change, aren't adding net new things), you're still within the bounds we reviewed 😄. |
Maybe I'm missing something but why not add I'd never even noticed the |
Background and Motivation
When using
System.IO.File
and/orSystem.IO.FileInfo
for opening, creating and writing files you do not have the option to properly open FileStreams for use with async methods.A very common mistake/issue that occurs because of it is that files are opened through methods like
File.OpenWrite
ornew FileInfo(x).Open()
and the FileStream is then used withasync/await
operations likeWriteAsync
orReadAsync
. However when this is done you will incur a performance hit as the stream is not configured to be used in this way.The proper way to use
FileStreams
with async await is to open aFileStream
through the constructor with and supplying the argumentisAsync
with true. HoweverSystem.IO.File
andSystem.IO.FileInfo
expose a lot of convenient shorthands but lack the possibility to open the streams properly for async actions. leading also to the misconception that async await with file IO is actually not good.Proposed API
@adamsitnik proposed the use of a
FileOptions
and supply additional overloads toSystem.IO.StreamReader
,System.IO.StreamWriter
,System.IO.File
andSystem.IO.FileInfo
so not only Files can be opened withFileOptions.Asynchronous
but also withFileOptions.SequentialScan
.Comment: #24698 (comment)
note that
File.Create(string path, int bufferSize, System.IO.FileOptions options)
already exists.Usage Examples
Alternative Designs
In the beginning an idea was put for exposing additional async API methods next to the existing methods and that would return return the stream in a
Task
/ValueTask
and guide the developer immediately into the async environment, also taken into account that a lot of .NET Native only exposed async variants in the API's with IO file streams.However as was mentioned in comments that streams are opened with constructors and would require a lot of changes and provide no additional benefit and more overhead.
Risks
The text was updated successfully, but these errors were encountered: