Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(2/4) Override Stream ReadAsync/WriteAsync Analyzer #4726

Merged
merged 25 commits into from
May 18, 2021

Conversation

NewellClark
Copy link
Contributor

@NewellClark NewellClark commented Jan 21, 2021

Fixes issue #33789.

I ran the analyzer against dotnet/runtime and found several violations:

  • DeflateStream, ChunkedMemoryStream, RequestStream, and NetworkStreamWrapper were all trivial, and have been dealt with in this PR.
  • CryptoStream was non-trivial, and is being worked on in this PR.
  • DeflateManagedStream is internal, and calls into the IFileFormatReader interface, which is a dead interface and is being removed in this PR. Once that gets merged, I'll memorify DeflateManagedStream, which will have no extra copies. Edit: I've memorified DeflateManagedStream (PR).
  • WebSocketHttpListenerDuplexStream is also internal, and from what I could find, ReadAsync/WriteAsync are only ever called from WebSocketBase, which only calls the array-based override. Unfortunately, WebSocketHttpListenerDuplexStream also calls back into WebSocket, and it passes the array it was given into multiple different methods on WebSocket, and some methods on WebSocketBuffer. Providing memory-based overrides would require almost a complete rewrite of WebSocketBuffer as well as changes to code that consumes it. Since nothing currently calls the memory-based overrides on WebSocketHttpListenerDuplexStream, I think we should strongly consider suppressing the warning.

Note that I'm using CA1840 as the diagnostic ID because I used CA1839 in another analyzer that has yet to be merged in. Not sure how that type of situation is typically handled.

@NewellClark NewellClark requested a review from a team as a code owner January 21, 2021 18:54
@NewellClark
Copy link
Contributor Author

@mavasani Sorry about that. I've retargeted this PR to the correct branch.

@codecov
Copy link

codecov bot commented Jan 21, 2021

Codecov Report

Merging #4726 (4d7967b) into release/6.0.1xx (0f43809) will increase coverage by 0.01%.
The diff coverage is 98.43%.

@@                 Coverage Diff                 @@
##           release/6.0.1xx    #4726      +/-   ##
===================================================
+ Coverage            95.56%   95.57%   +0.01%     
===================================================
  Files                 1210     1212       +2     
  Lines               276014   277225    +1211     
  Branches             16724    16737      +13     
===================================================
+ Hits                263785   264971    +1186     
- Misses               10007    10020      +13     
- Partials              2222     2234      +12     

return containingType.GetMembers(methodName)
.SingleOrDefault(symbol => symbol is IMethodSymbol m && IsMatch(m, argumentTypes)) as IMethodSymbol;

static bool IsMatch(IMethodSymbol method, ITypeSymbol[] argumentTypes)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Evangelink Was there a helper method that does this?

Copy link
Contributor Author

@NewellClark NewellClark Jan 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The helper method requires instances of ParameterInfo, which themselves require INamedTypeSymbol instances. It was significantly shorter to write it out my way, which allows any ITypeSymbol to be used. I did originally write it out using the helper method, but it was significantly longer and more verbose.

NewellClark and others added 3 commits January 22, 2021 10:29
…ideStreamMemoryBasedAsyncOverrides.cs

Co-authored-by: Youssef Victor <31348972+Youssef1313@users.noreply.github.com>
If there are multiple definitions of the same overridden method, the analyzer can crash due to uses of `SingleOrDefault` instead of `FirstOrDefault`.

To fix, I refactored `GetOverload` and `GetSymbolForOverridingMethod` into immutable-array-returning methods, and simply used `FirstOrDefault` or `SingleOrDefault` at each call-site as appropriate.
@@ -1510,4 +1510,14 @@
<value>and all other platforms</value>
<comment>This call site is reachable on: 'windows' 10.0.2000 and later, and all other platforms</comment>
</data>
<data name="ProvideStreamMemoryBasedAsyncOverridesDescription" xml:space="preserve">
<value>The default implementation of '{2}' delegates to '{1}', which is innefficient as it requires an extra copy. It also results in extra allocations as it returns Task instead of ValueTask. Consider moving your implementation to '{2}' and have '{1}' delegate to '{2}'.</value>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like you missed the {0} in this declaration.

Copy link
Contributor Author

@NewellClark NewellClark Jan 23, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used the {0} in the message field, but not in the description.
Edit: I've changed the description so it includes the {0} argument.


context.RegisterSymbolAction(AnalyzeNamedType, SymbolKind.NamedType);

void AnalyzeNamedType(SymbolAnalysisContext context)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except if I have missed something, this local func could be marked as static.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It uses the symbols struct produced by the call to TryGetRequiredSymbols(...) on line 55.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad!

Comment on lines 101 to 104
return Diagnostic.Create(
Rule,
violatingType.Locations[0],
violatingType.Locations.Skip(1),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use the CreateDiagnostic helper method.

Suggested change
return Diagnostic.Create(
Rule,
violatingType.Locations[0],
violatingType.Locations.Skip(1),
return violatingType.CreateDiagnostic(
Rule,

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've fixed this in the latest commit.


// We know that there are no compiler errors on streamType, so we use 'SingleOrDefault'.

var readAsyncArrayMethod = GetOverloads(streamType, ReadAsyncName, byteArrayType, int32Type, int32Type, cancellationTokenType).SingleOrDefault();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would still avoid the SingleOrDefault here as it makes it harder to debug if you end up with actually more than one item.

What's your view on this @mavasani ?

Copy link
Contributor Author

@NewellClark NewellClark Jan 23, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I figured SingleOrDefault since we're looking up methods on framework types that are in metadata, and therefore could never contain compiler errors. Now that I think about it though, if this analyzer is running against System.Private.CoreLib, these framework types could actually contain compiler errors, and we definitely don't want the analyzer to crash if that happens. I suppose we could detect whether streamType is in source code or metadata, but it's probably not worth the hassle. I'll change this to FirstOrDefault for now.

Edit: I've switched to FirstOrDefault

{
public class ProvideStreamMemoryBasedAsyncOverridesTests
{
#region Reports Diagnostic
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[cosmetic] @mavasani Shall we keep introducing these regions?

Comment on lines +27 to +36
string code = $@"
{CSUsings}
namespace Testopolis
{{
public class {{|#0:FooStream|}} : Stream
{{
{CSAbstractMembers}
{CSReadAsyncArray}
}}
}}";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[cosmetic] I am fine if you keep it this way but I find the double curly braces hard to read.

Suggested change
string code = $@"
{CSUsings}
namespace Testopolis
{{
public class {{|#0:FooStream|}} : Stream
{{
{CSAbstractMembers}
{CSReadAsyncArray}
}}
}}";
string code = @"
" + CSUsings + @"
namespace Testopolis
{
public class {|#0:FooStream|} : Stream
{
" + CSAbstractMembers + @"
" + CSReadAsyncArray + @"
}
}";

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going forward, I will do it your way.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's wait for @sharwell or @mavasani opinion here.

@NewellClark
Copy link
Contributor Author

NewellClark commented Jan 24, 2021

This should be linked to issue #33789.

Comment on lines +111 to +112
var int32Type = compilation.GetSpecialType(SpecialType.System_Int32);
var byteType = compilation.GetSpecialType(SpecialType.System_Byte);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mavasani Can these be null under any case?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, they can be null for broken compilation references. We should have a null check for these types as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mavasani Fixed. Sorry I missed this earlier.

Comment on lines +250 to +253
VerifyVB.Diagnostic(Rule)
.WithLocation(0)
.WithLocation(1)
.WithArguments("River", displayArrayMethod, displayMemoryMethod)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very confusing. Is this supposed to be two diagnostics?
I believe that only one diagnostic is passed to the test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The location that is underlined when this diagnostic is reported is the name of the class in the class declaration. If the class is a partial class, all partial declarations need to be underlined. That is what this test is checking: that when you have multiple partial declarations in the same file, all partial declarations will be underlined for a single violation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NewellClark If I understand correctly, you're passing only one diagnostic. When you chain WithLocation calls, only the latest one takes effect.

I believe if you removed the chain and kept the latest WithLocation call only, the test will still pass. Could you confirm that locally on your machine?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just tested it. The test fails if we remove any of the WithLocation calls, and the error message reports the diagnostics at the removed locations as the unexpected diagnostics.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NewellClark Good to know! Thanks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Youssef1313 by the way, would it be possible to have this pull request linked to issue 33789?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NewellClark The issue won't be auto-closed when the PR is merged. But a Microsoft employee will close it when he merges your PR.

Comment on lines +297 to +299
.WithLocation(0)
.WithLocation(1)
.WithLocation(2)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as my previous comment, but this time we are checking whether all locations are reported when the partials are in separate files.

@@ -1510,4 +1510,14 @@
<value>and all other platforms</value>
<comment>This call site is reachable on: 'windows' 10.0.2000 and later, and all other platforms</comment>
</data>
<data name="ProvideStreamMemoryBasedAsyncOverridesDescription" xml:space="preserve">
<value>The default implementation of '{2}' delegates to '{1}', which is innefficient as it requires an extra copy. It also results in extra allocations as it returns Task instead of ValueTask. Consider moving {0}'s implementation to '{2}' and have '{1}' delegate to '{2}'.</value>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<value>The default implementation of '{2}' delegates to '{1}', which is innefficient as it requires an extra copy. It also results in extra allocations as it returns Task instead of ValueTask. Consider moving {0}'s implementation to '{2}' and have '{1}' delegate to '{2}'.</value>
<value>The default implementation of '{2}' delegates to '{1}', which is innefficient as it requires an extra copy. It also results in extra allocations as it returns 'Task' instead of 'ValueTask'. Consider moving {0}'s implementation to '{2}' and have '{1}' delegate to '{2}'.</value>

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, you cannot have format arguments {0}, {1}, etc. in description string, it is a static description per-descriptor, not something instantiated per-diagnostic. You should instead follow what @carlossanlop did in a similar analyzer and use more generic description string that is applicable for all violations.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

Comment on lines 58 to 60
context.RegisterSymbolAction(AnalyzeNamedType, SymbolKind.NamedType);

void AnalyzeNamedType(SymbolAnalysisContext context)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
context.RegisterSymbolAction(AnalyzeNamedType, SymbolKind.NamedType);
void AnalyzeNamedType(SymbolAnalysisContext context)
context.RegisterSymbolAction(AnalyzeNamedType, SymbolKind.NamedType);
return;
// Local functions
void AnalyzeNamedType(SymbolAnalysisContext context)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

@@ -2,3 +2,4 @@

Rule ID | Missing Help Link | Title |
--------|-------------------|-------|
CA1840 | <https://docs.microsoft.com/dotnet/fundamentals/code-analysis/quality-rules/ca1840> | Provide memory-based overrides of async methods when subclassing 'Stream' |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ID has already been used, please choose a new one.

Copy link
Contributor Author

@NewellClark NewellClark Apr 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed ID to ca1844

Copy link
Contributor

@mavasani mavasani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks great. Please retarget to preview4 branch.

- Change ID to CA1842
@NewellClark NewellClark changed the base branch from release/6.0.1xx-preview1 to release/6.0.1xx-preview4 April 20, 2021 14:34
@mavasani
Copy link
Contributor

@NewellClark Sorry for not getting back to this PR review sooner, but unfortunately even preview4 is now locked. Can you please retarget to https://github.com/dotnet/roslyn-analyzers/tree/release/6.0.1xx? Promise this one is the last branch retarget request. Again, very sorry for this additional work.

Can you also address the remaining review comments? I should be able to merge once both of these are taken care of.

@NewellClark NewellClark changed the base branch from release/6.0.1xx-preview4 to release/6.0.1xx May 17, 2021 15:34
Fix possible crash when primitive types can't be loaded
@NewellClark
Copy link
Contributor Author

@NewellClark Sorry for not getting back to this PR review sooner, but unfortunately even preview4 is now locked. Can you please retarget to https://github.com/dotnet/roslyn-analyzers/tree/release/6.0.1xx? Promise this one is the last branch retarget request. Again, very sorry for this additional work.

Can you also address the remaining review comments? I should be able to merge once both of these are taken care of.

@mavasani Fixed. Should I retarget for all my other analyzer PRs that are targeted to preview4?

@mavasani
Copy link
Contributor

Should I retarget for all my other analyzer PRs that are targeted to preview4?

Yes please. preview4 branch is now locked.


return violatingType.CreateDiagnostic(
Rule,
violatingType.ToDisplayString(SymbolDisplayFormat.MinimallyQualifiedFormat),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We normally use just the type/method .Name in diagnostic messages. MinimallyQualifiedFormat actually produces a qualified name, which lead to very long names.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mavasani Fixed. I've changed the message so it makes sense with simple names (the message was originally worded with the assumption that the arguments of each overload would be displayed).

Copy link
Contributor

@mavasani mavasani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

I can merge once you change the diagnostic messages to use simple names instead of ToDisplayString.

- Remove usages of `MinimallyQualifiedFormat`
- Change message so it makes sense with simple names
Copy link
Contributor

@mavasani mavasani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@mavasani mavasani merged commit 4bda814 into dotnet:release/6.0.1xx May 18, 2021
@mavasani
Copy link
Contributor

@NewellClark Can you please help us in creating documentation PRs for all the new .NET6 analyzers that you have contributed? Please see the item 4. in https://github.com/dotnet/roslyn-analyzers/blob/main/GuidelinesForNewRules.md.

Also tagging @gewarren as we now have 8 new undocumented CA rules which have been implemented in the release\6.0.1xx branch for .NET6: https://github.com/dotnet/roslyn-analyzers/blob/release/6.0.1xx/src/NetAnalyzers/Core/AnalyzerReleases.Unshipped.md.

@NewellClark
Copy link
Contributor Author

@NewellClark Can you please help us in creating documentation PRs for all the new .NET6 analyzers that you have contributed? Please see the item 4. in https://github.com/dotnet/roslyn-analyzers/blob/main/GuidelinesForNewRules.md.

Also tagging @gewarren as we now have 8 new undocumented CA rules which have been implemented in the release\6.0.1xx branch for .NET6: https://github.com/dotnet/roslyn-analyzers/blob/release/6.0.1xx/src/NetAnalyzers/Core/AnalyzerReleases.Unshipped.md.

Absolutely. I'm on it.

@NewellClark NewellClark mentioned this pull request May 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants