proposal: Go 2: reform the variable redeclaration syntax to avoid some confusions and inconveniences

[zigo101]

(The following content is a rewritten and an improvement version of this comment)

Abstract

This proposal tries to avoid some problems caused by short variable declarations (a.k.a., variable redeclarations), by introducing a scope viewer identifier concept and using labels as scope viewer identifiers.

Background

We all know that, although short variable declarations (x, y := a, b) do solve some problems, they also brings some new ones. The new problems include:

1. break the one way to do a specified thing principle in Go. We all know that short declarations overlap too much with standard declarations in functionalities.
2. short declarations brings some confusions to Go programmers, in particular to new gophers.
   • It is not very clear to get which identifiers are new declared and which are redeclared at the first glance.
   • Almost every gophers ever dropped in the famous variable shadowing trap caused by using short declarations. A real world example: Update negotiator.go Azure/go-ntlmssp#24, and an older issue.
   • Not all L-values suitable to act as target values are allowed to show up as the target values in short declarations. proposal: spec: let := support any l-value that = supports #30318. (Another similar one.)
   • The original intention of short variable declarations is to make code less verbose. However, it makes code more verbose in several scenarios.

Proposal

Proposal part 1: scope viewers

The proposal proposes that we can re-use labels as scope viewer identifiers. The following code shows what are scope viewer identifiers:

func f() {
	var a = 1
x:
	{
	y:
		var a = 2
	z:
		{
			var a = 3
			
			// x:a <=> y:a
			println(x:a, y:a, z:a, a) // 1 1 2 3
		}
	}
}

Proposal part 2: syntax sugars

There are two sugars for the scope:identifier notation.

1. :identifier (without the scope prefix), means the innermost declared identifier (it is equivalent to *&identifier).
2. ::identifier, means the package-level declared identifier.

(Digression 1: is it good to view package import names as scope viewer identifiers? So that we can use aPkg::ExportIdentifier to use exported resources.)

(Digression 2: the proposal may also discard the above described scope viewer concept and only contain the following content, by replacing :identifier with *&identifier and using *&identifier as redeclared items.)

Proposal part 3: avoid using short declarations by using scope viewer identifiers

The proposal proposes that we can relax the restriction of the left items in standard variable declarations. Now all left items must be pure identifiers. We can relax the rules as selectors, dereferences, element indexing, and the above mentioned scope:identifier forms are allowed to show up as target values in the left items of a standard variable declaration, as long as there is at least one pure identifier in the left items as well. When the declaration is executed at run time, the new variables represented by pure identifiers are initialized, others are re-assigned with new values. The following is an example without using labels:

 type T struct {
        i *int
        s []int
 }
 var t T
 var p = new(int)

 func bar() {
       // "err" is new declared, others are not.
       var *p, t.i, t.s[1], :p, err = 1, p, 2, t.i, errors.New("bar")
       // *:p is not valid. In other words, :id must show up independently.
       ...
 }

Another more common use case:

    package bar

    func foo() {
       var x, err = f()
       ...
       // Here "err" means the "err" declared above.
       var y, z, :err = g()
s:
       ...
       {
           // The "err" also means the "err" declared above.
           var w, :err = h()
           ...
           // This "err" is a new declared one.
           var m, n, err = k()
           ...
           
           {
           	// This "err" means the first error declared above.
           	var p, q, s:err = l()
           	...
           }
       }
    }

The := redeclaration syntax should be kept ONLY in the simple statements in control flows, just for aesthetics reason. In other words, ... := ... is totally a shorthand of var ... = ... in the simple statements in control flows.

    if a, b := c, d; a {
    }
    //<=>
    if var a, b = c, d; a { // some ugly
    }
    // However, personally, I also think the ugly version is acceptable.

Rationale

Please see the above backgroud part.

Compatibility

The idea should be Go 1 compatible. What I mean here is the above proposed new redeclaration syntax may coexist with the current standard and short variable declarations.

It is easy to let go fix translate short variable declarations into the above proposed new syntax form. The short declarations in the simple statements in control flows don't need to be translated.

If it needs, the current short declaration uses can be removed eventually.

Implementation

Compiler parsers should consider the new syntax form.

One new statement might need to be added to go/ast standard package.

[zigo101]

Aha, I suddenly realized that scope:id might cause some difficulties in parsing container indexing and re-slicing uses. So there are two ways to avoid this:

1. only allow scope:id to show up as the target items in variable declarations.
2. use *&id (or :id) to represent a re-declared target item.

(Edit: is scope'id a better choice?)

[ianlancetaylor]

For language change proposals, please fill out the template at https://go.googlesource.com/proposal/+/refs/heads/master/go2-language-changes.md .

When you are done, please reply to the issue with @gopherbot please remove label WaitingForInfo.

Thanks!

[zigo101]

Er, I tried to search this template before posting the proposal, but I only found this one instead.

OK, the following uses the new template form:

================================================================

Would you consider yourself a novice, intermediate, or experienced Go programmer?

Experienced (at least in grammar, ;D)

What other languages do you have experience with?

C, C++, AS3

Would this change make Go easier or harder to learn, and why?

Easier. At least less confusions for new gophers, and it removes some inconveniences in Go programming.

Has this idea, or one like it, been proposed before?

There are some similar ones, but none of them proposes to retire short variable declarations
(the current variable redeclaration syntax). This one suggests to retire short variable declaration syntax.

(I decided to remove the scope viewer concept from this proposal. The scope viewer concept is mainly used to make the :identifier notation look naturally. The usefulness of Itself is small actually.) (edit: I decided to add it back, with a new form scope'identifer)

This proposal also reuses the the labels concepts as scope viewers. Here is an example:

func f() {
	var a = 1
x:
	{
	y:
		var a = 2
	z:
		{
			var a = 3
			
			// x'a <=> y'a
			println(x'a, y'a, z'a, a) // 1 1 2 3
		}
	}
}

Before, it would be written as the following (less readable in my opinion):

func f() {
	var a = 1
	var p1 = &a
	{
		var a = 2
		var p2 = &a
		{
			var a = 3
			
			// x'a <=> y'a
			println(*p1, *p1, *p2, a) // 1 1 2 3
		}
	}
}

Who does this proposal help, and why?

It removes some confusions for new gophers, and it removes some inconveniences in Go programming.

What is the proposed change?

The proposal proposes to retire short variable declarations
(the current variable re-declaration syntax) add a scope viewer concept.

If the scope in scope'id is omitted as 'id, it means the inner most declared id.

Precisely, it proposes the following line

a, b := ... // a is new declared and b is re-declared.

can be expressed as a less confused way:

var a, 'b = ...

And it recommends the following code

x, y := ... // x and y are both new declared

should be written as (already supported now):

var x, y = ...

for better readability.

The third, for continence, it also proposes that legal L-values in general assignments
can also show up as the left items in variable declarations/specifications.
For example:

var aPkg.Exp, aValue.Sel, *ptr, contianer[k], scope'redeclared, 'reDeclared, newDeclared = ...

Please note, the scope viewer concept is an optional feature in this proposal. If it is not ignored, then there is an alternative form for 'identifier: *&identifier.

• The benefit of *&identifier is we don't need to invent a new expression form. It is supported now!
• The drawback of *&identifier is it is some verbose.

Is this change backward compatible?

Yes (for what I can tell now). Here what I mean is this proposal doesn't require to retire short variable declarations immediately. Short variable declarations may be retired eventually.

Show example code before and after the change.

An example (in a survey, 21 of 35 participants think the following program prints 6,
which is really the expected result, however the actual result is 4):

package main

func f(n int) (r int) {
  a, r := n-1, n+1
  if a+a == r {
    c, r := n, n*n
    r = r-c
  }
  return r
}

func main() {
  println(f(3))
}

The correct version (some verbose):

package main

func f(n int) (r int) {
  a, r := n-1, n+1 // might be also a function call
  if a+a == r {
    var c int // If the the type of c declared is in another package, we must import the pacakge.
    c, r = n, n*n
    r = r-c
  }
  return r
}

func main() {
  println(f(3))
}

After the change (It is less confused and still avoids verboseness):

package main

func f(n int) (r int) {
  var a, r = n-1, n+1
  if a+a == r {
    var c, 'r = n, n*n
    r = r-c
  }
  return r
}

func main() {
  println(f(3))
}

What is the cost of this proposal? (Every language change has a cost).

How many tools (such as vet, gopls, gofmt, goimports, etc.) would be affected?

Many, but the impact is expected to be smaller than introducing type aliases, and it is easy to update these tools.

What is the compile time cost?

Very small.

What is the run time cost?

None.

How would the language spec change?

Add a new variable declaration+assignment hybrid statement.

VarDecl     = "var" ( VarSpec | "(" { VarSpec ";" } ")" ) .
VarSpec     = ExpressionList "=" ExpressionList .

Each left-hand side operand must be addressable, a map index expression,
or (for = assignments only) the blank identifier. Operands may be parenthesized.
In particular, there must at least one left-hand side operand is a pure identifier.

Orthogonality: how does this change interact or overlap with existing features?

It avoids one variable declaration form.

The scope viewer concept is good in orthogonality with labels.

Is the goal of this change a performance improvement?

No.

Does this affect error handling?

No.

Is this about generics?

Not.

[zigo101]

@gopherbot please remove label WaitingForInfo.

[ianlancetaylor]

The proposal described in the template (#38388 (comment)) seems significantly different from the proposal described in the original post on this issue. For example, there are no scope viewers in the template discussion. What is the actual proposal here? Thanks.

The idea of annotating the variables that should be newly declared in a := statement has been raised before, in #377.

[zigo101]

There is a problem in the form scope:id from the original post, when the form shows up in re-slice and element indexing expressions, so I removed it from the new template version. OK, I decided to add it back with a new form scope'id, so that it can show up in re-slice and element indexing expressions.

The idea of annotating the variables that should be newly declared in a := statement has been raised before, in #377.

This is half true. None of the ideas in #377 propose to retire the current re-declaration syntax form.

[ianlancetaylor]

Removing a widely used feature like redeclarations on := will be so far in the future that there is little point in discussing it. We can discuss adding some new feature. If over time that feature replaces redeclarations, then years from now we can remove redeclarations.

go fix is not a panacea here, because go fix can't fix all the existing documentation and written examples and blog posts.

[zigo101]

The

var scope'redeclared, 'redeclared, newDeclared = ...

form can coexist with the current redeclarations.

go fix can't fix all the existing documentation and written examples and blog posts.

Yes, this is true. It has happened for "named type" definition and the behavior of fmt.Print(aMap). It is really hard to totally avoid such situations.

[ianlancetaylor]

Sorry, we're still not clear on what the actual proposal is here. It's hard to track the changes in this issue.

In

:p, err := F()

is p a new identifier? Or is :p a reference to an identifier in an enclosing scope, where that identifier must exist?

Are scoped variable references still required by this proposal? Can this proposal be described in just a few succinct lines?

Thanks.

[zigo101]

@ianlancetaylor
The :id notation has been changed to 'id. The reason is 'id may show up in slice and index expressions, but :id may not. Sorry, the change might not show up in the mail list.

Here, I summarize the proposal simply. The proposal suggests three changes to Go.

1.

Recommend to replace the current re-declaration syntax form

newdeclared, redeclared := ...

with a clearer form.

var newDeclared, 'reDeclared = ...

The new clearer form shows which identifiers are re-declared clearly.

2.

Support "variable declaration + assignment" hybrid statement, as long as there is at least one pure identifier target item. For example,

var aPkg.Exp, aValue.Sel, *ptr, contianer[k], 'reDeclared, newDeclared = ...

3.

Use labels as scope viewers. An example:

func f() {
	var a = 1
x:
	{
	y:
		var a = 2
	z:
		{
			var a = 3
			
			// x'a <=> y'a
			println(x'a, y'a, z'a, a) // 1 1 2 3
		}
	}
}

Some ideas in this proposal have been mentioned in other issue threads, but this one differs from them by recommending to retire the current re-declaration syntax form.

[zigo101]

I'm sorry, I just mis-wrote

var newDeclared, 'reDeclared = ...

as

var newDeclared, 'reDeclared := ...

[ianlancetaylor]

Thanks. Your change 1 and 2 from #38388 (comment) seem to be adequately covered by #377.

I'm not clear on when change 3 would be useful. If you want to refer to a variable in an outer scope, don't shadow it. It seems to me that it is rather confusing to shadow a variable from an outer scope if you then need to refer to it. Introducing a way to get around that shadowing seems like adding complexity to avoid a complex situation, when there is already an easy way to simplify the situation instead by renaming the variable.

[zigo101]

#377 doesn't cover the change 2, #30318 does.

Without considering the change 3, this proposal can be viewed as a merge of #377 and #30318. And this proposal differs from them by recommending to retire the current re-declaration syntax form.

[ianlancetaylor]

As I mentioned above, though, we really can't remove the current re-declaration syntax. So the best we could do to "retire" it would be to advise people writing new code to use a different form.

[zigo101]

So the best we could do to "retire" it would be to advise people writing new code to use a different form.

Sorry for my bad description. In fact, this is exactly the way that this current proposal suggests.

[ianlancetaylor]

As discussed above, parts of this are covered by #377. The idea of referencing variables in different labeled scopes is new, but does not seem like a feature that is essential to the language. People can instead rename variables. Also, emoji voting for this proposal is negative.

For those reasons, this proposal is itself a likely decline. We can pursue further discussions on #377. Leaving open for four weeks for final comments.

[zigo101]

The other two issues don't pursue retiring the := token.
But ok, it is not bad to discuss this in the other issues.

[ianlancetaylor]

No change in consensus. Closing.