[Proposal] Only allow Lexical Contextual Keywords in the Language

[jmarolf]

Only Allow Lexical Keywords in the Language

• Proposed
• Prototype:
• Implementation:
• Specification:

Summary

Today there are keywords in the language that cannot be understood with just lexical information such as var or nameof. These keywords operate this way for backwards compatibility reasons. I propose that we change the language so that there exists no keywords that cannot be determined via lexical analysis.

Motivation

In general, C# strives to be a language that is explicit about program behavior and normally required the developer to write out what their intention is without ambiguities. This makes is a language that is easy for someone to read and understand once you learn it as there is little implicit behavior. I believe having this "gotcha" where keywords are only keywords if nothing else in scope is so-named makes the language harder to read and reason about in general.

There is also the reality that the design goals around C# Language versions and .NET Framework versions have changed. In the past it was paramount that developers could take a new language version update without updating the framework version they were targeting. With language features being increasingly tied to the runtime this makes less sense. We now strongly encourage developers to update both the language version and target framework together.

Detailed design

There is an existing concept in the language called "contextual keywords". I am not proposing doing away with this concept altogether just changing it so that a keywords "contextual-ness" is always able to be determined lexically. Take new (at the time of this writing) keyword record. We can still know if we are referring to the record keyword or some identifier named record based on the lexical context, there is no ambiguity. However var, according to the spec, requires us to check if there are types named var in scope:

spec

In the context of a local variable declaration, the identifier var acts as a contextual keyword. When the local_variable_type is specified as var and no type named var is in scope,

Similarly nameof requires checking if a there are any identifiers called nameof in scope

spec

Because nameof is not a reserved keyword, a nameof expression is always syntactically ambiguous with an invocation of the simple name nameof. For compatibility reasons, if a name lookup of the name nameof succeeds, the expression is treated as an invocation_expression -- regardless of whether the invocation is legal. Otherwise it is a nameof_expression.

The implementation of this proposal would remove wording from the spec around name lookup collisions, and have a compliant compiler be able to fully determine keywords given only parsing information.

The following keywords would now error if developers attempted to use them as anything other than a keyword

• var
• nameof
• dynamic
• _

Drawbacks

This is a breaking change, if anyone was relying on this behavior in their code it would no longer compile. For most cases where a type named var or a method named nameof is in use the developer would need to change the usages to @var and @nameof.

Alternatives

We could opt to keep _ as a contextual keyword that depends on name lookup rules as this is the change most likely to break real-world programs (see discussion on #1064)

[RikkiGibson]

I feel like it's worth mentioning that the "migration path" for most cases where a type named var or a method named nameof is in use is to change the usages to @var and @nameof.

[jmarolf]

ah, I'll add that to the Drawbacks section

[HaloFour]

Java:

[CyrusNajmabadi]

I believe that @jaredpar's position on this is that we could tie this to language version upgrades, similar to the breaking change we took for 'record' parsing. If we do this, then we can take this codepaths out for later versions, and effectively limit support for previous versions.

[jmarolf]

you are right that nameof doesn't ever come into play, var on the other hand?

[CyrusNajmabadi]

well, we have no formatting rules that would be affected by 'var', so i'm not sure if it's relevant.

HOwever, say that 'var' was relevant for some formatting rule. I woudl say it would 100% acceptable for our formatter to say:

"you wrote var x, so i'm doing to assume that that var is the type-inferred var and not some real type you defined in your system".

Our formatter doesn't have to support the pathological. Specifically, people going out of their way to subvert the language. If users do pathological stuff and want the formatter to support that, we're welcome to say: sorry, too bad, so sad :) We would want to support the complex though. So if you do have an array with 10,000 elements in it... well, that's extreme and complex, but not unreasonable. So we should try to support if we can. But people who abuse things like contextual keywords, with their own semantic types that conflict with them? I do not think we have to care about that case at all.

[jmarolf]

Our formatter doesn't have to support the pathological.

right, my point is that all formatters need to consider this today. If they just blindly use the APIs they for roslyn and expect to get keywords for this, they won't. It's by no means insurmountable, especially for us who have had years to tune things. If someone wants to color code on their blog what should they do if they want to handle contextual keywords? They can't just call roslyn apis and expect things to work. They need to learn about the contextual rules and decide what heuristics make sense for there use cases. I would prefer if "Is this a keyword" was a remarkably simple question to answer that required no education for users of roslyns apis.

Admittedly this desire only serves a small portion of the C# community and I think you could most definitely argue that this alone is not a strong enough argument to justify the change

[jmarolf]

If the roslyn apis had these heuristics built in, or had helpers that said "right but please honey, this is definitely a keyword" then I think I wouldn't care. But this is something everyone writing tooling for C# needs to care about and think about today.

[CyrusNajmabadi]

I would prefer if "Is this a keyword" was a remarkably simple question to answer that required no education for users of roslyns apis.

I'm 100% ok with that :)

[CyrusNajmabadi]

The key motivator here is tooling. There are several tools such as dotnet-format which operate on syntax, but because of the way keywords work in the language today these tools need to get all the command-line arguments to the compiler to be accurate which typically means calling msbuild. Tooling authors are faced with an unfortunate choice of building slow but accurate tools or building fast but inaccurate ones.

I woudl be surprised if any formatting tooling actually cared about this (including for the nameof case). In practice nameof will 100% of the time be the C# language feature. So it's totally fine for our tools to assume as such. The same with 'var', 'dynamic' and the other semantic contextual keywords.

So i don't see this really buying much at the tooling level TBH, as tehse tools shoudl care about the practical case, not the pathological.

[jmarolf]

in Visual Studio sure because we expect user interaction in some form or fashion. Tools like dotnet-format operate over millions of lines of code and people expect it to not just use heuristics to make decisions since there is no human interaction.

[jmarolf]

I've moves this proposal to an issue here