proposal: Go 2: expression scoped variables

[dolmen]

I propose to extend the expression syntax to allow to declare variables scoped to a sub-expression.

Rationale

Go already allows to declare variables scoped to some statements:

if _, err := check(); err != nil {
    ....
}

switch l := len(x); l {
case 0:
case 1:
default:
}

I propose to allow to define variables scoped to an expression by allowing the same syntax as if/switch in any non-constant expression.

Here is the grammar:

VarDeclExpr = "(" ShortVarDecl ";" Expression ")" .

( v1, v2 := e1, e2 ; expr ) would become a shorter alternative to:

(func() T {
    v1, v2 := e1, e2
    return expr
}())

e1 and e2 are evaluated first. expr can reference variables v1, v2. v1, v2 are not accessible outside expr.

Examples:

ptrTo3 := ( x := 3; &x )
ptrToNow := (now := time.Now(); &now)
squareF := ( x := F(); x*x )
ok := (_, err := f.Write(); err != nil)
fmt.Printf("%T %s\n", V, (b, _ := json.Marshal(V); b))

Note: if and switch allow SimpleStmt (which is more than just ShortVarDecl) as the first part. I think that inside an expression context only the ShortVarDecl is relevant. However this syntax could be extended later.

Would you consider yourself a novice, intermediate, or experienced Go programmer?

Experienced.

What other languages do you have experience with?

C, Perl, JavaScript, SQL, shell, Rebol, Caml, Prolog, Pascal, x86 assembly...

Would this change make Go easier or harder to learn, and why?

Neither. This change generalizes existing syntax by reusing constructions which were so far limited to specific contexts.

Has this idea, or one like it, been proposed before?

AFAIK, no.

However this provides a more generic alternative to proposals #45624 (expression to create pointer to simple types) and #45653 (allow taking address of value returned by function):

ptrTo3 := (x := 3; &x)
ptrToNow := (now := time.Now(); &now)

Who does this proposal help, and why?

People annoyed by the difficulty of allocating pointers to simple values.

People annoyed by the difficulty of calling a function that returns multiple values in a context where a single value is wanted.

What is the proposed change?

See above.

Is this change backward compatible?

Yes.

Show example code before and after the change.

See above.

What is the cost of this proposal? (Every language change has a cost).

Fairly small compiler update compared to some others underway. Will need to touch documentation, spec, perhaps some examples.

Can you describe a possible implementation?

As this is syntaxic sugar, it might be implemented by the compiler as an AST rewrite:

( v1, v2 := e1, e2 ; expr ) ->

(func() T {
    v1, v2 := e1, e2
    return expr
}())

How would the language spec change?

The definition of Expression would be extended:

PrimaryExpr =
	Operand |
	Conversion |
	MethodExpr |
	PrimaryExpr Selector |
	PrimaryExpr Index |
	PrimaryExpr Slice |
	PrimaryExpr TypeAssertion |
	PrimaryExpr Arguments  |
	VarDeclExpr .                       // <----
VarDeclExpr = "(" ShortVarDecl ";" Expression ")" .

Orthogonality: how does this change interact or overlap with existing features?

As this reuses syntax known to every Go programmer, orthogonality is improved.

Is the goal of this change a performance improvement?

No.

Does this affect error handling?

This could allow to drop values in expressions that return multiple values. This is useful when passing values to functions and either the value or the error is wanted.

justError := (_, err := strconv.ParseInt(s); err)
justValue := (n, _ := strconv.ParseInt(s); n)

fmt.Printf("%T %s\n", V, (b, _ := json.Marshal(V); b))

Is this about generics?

No.

[gudvinr]

Pointer thing related to #45624 I think.

Extracting single value from functions looks clunky. IMO, it's not really worth saving extra line for the sake of saving extra line.
These expressions can become ugly and unreadable very fast when you jam everything into single line.

You can just write first couple examples like this:

_, justError := strconv.ParseInt(s)
justValue, _ := strconv.ParseInt(s)

And if you just want to omit an error and you need to do that more than once, you can always write something like func mustJsonMarshal(v interface{}) []byte that should be inlined by compiler.

[dolmen]

@gudvinr That was just an example. The use case for expression-scoped variables is of course NOT in expressions which can also be written as simple statements.

The power of expression-scoped variables is in deeply embedded expressions such as:

• arguments to functions/methods calls
• initializers of structures (especially package level variables)
• multiple uses of one sub-expresssion in an expression

This proposal just adds syntactic sugar like #45624 does, because as mentioned, alternative syntax already exists.
Also this is a niche use case like #45624, but my proposal is about a syntax familiar to Go programmers that can be used for more cases than just pointer expressions.

[gudvinr]

That was just an example.

Well, examples usually tend to help you explain why someone needs this. Basically, what you calling "annoyance" can be called as "clarity" by other people.

Pointer to value and pointer to function result addressed in other issues. In these cases it is something that already exists. Even grammar changes aren't required.

Other ones become unreadable really fast when you start to use parts of real-word code instead of single-characted method names. Especially when used in function calls. Or used rarely in something you might encounter or write.

[quenbyako]

what about something like this:

jsonStr, err := (b, err := json.MarshalIdent(someObj, "", "  "); string(b), err)

Does this proposal accept multiple return values as var expressions?

[ycalansy]

I believe this proposal is just a variant of lambda expression (Python / Kotlin) / closure (Rust), and it has been discussed in a separate issue #21498.

[dolmen]

@ycalansy No, this proposal isn't about lambda expression. Lambda expression are used to define function. Here, this is about expression to evaluate immediately.

[dolmen]

@quenbyako

Does this proposal accept multiple return values as var expressions?

No. I considered to only extend single expressions (called PrimaryExpr in the Go specification). Because list of expressions (ExpressionList) are not expressions (Expression). For example, you can't wrap an ExpressionList with parentheses to use in the same context as an Expression in the current Go specification.

List of expressions are mostly used in statements (Assignment, VarDecl, ShortVarDecl) and I think that expression-scoped variable bring low value in such context as a separate statement is more readable. Expression-scoped variable are mostly useful when the statement that includes the expression spans multiple lines.

[dolmen]

In the proposal I suggested PrimaryExpr as the extension point for the grammar, but Operand is a better place as this is where expressions wrapped in parentheses are defined:

Operand     = Literal | OperandName | "(" Expression ")"  | VarDeclExpr .
VarDeclExpr = "(" ShortVarDecl ";" Expression ")" .

[ycalansy]

@dolmen Imagine if you have lambda expression in Go, you can simply achieve what you want in this proposal by writing code similar to the following:

Your proposal:

squareF := ( x := F(); x*x )

Rust:

fn main() {
    let nine = (|x: i8| x * x)(do_something());
    println!("{}", nine);
}

fn do_something() -> i8 {
    3
}

Python:

def do_something():
   return 3

nine = (lambda x: x * x)(do_something())
print(nine)

However, I am not a big fan of this kind of syntax because it reduces the readability of the code.

[dolmen]

@ycalansy I know what are lambda expressions. And this proposal is NOT about lambda expressions. In fact, I'm against lambda expression (#21498) without the func keyword as they hide function definitions. I like the explicit func keyword used to define function as this is explicit. And I know that lambda expressions will be abused. To keep Go simple to read we must avoid to have 2 syntaxes for the same meaning that allow different writing styles.

But I too often write short lived functions (often closures) just to workaround shortcomings in Go syntax. I call a function short lived if it is defined and immediately executed in the same statement (function declaration followed by a list of arguments).

Here are some use cases of short lived functions:

• variable scoping (when I need stricter scope that the englobing function)
• defer scoping (when I need stricter scope that the englobing function)
• complex expressions which need to save intermediate results for reuse (this proposal is for this use case)

I think that the func keyword is pollution for short lived functions and that syntax would be helpful to remove func from those expressions.

This proposal is about adding syntax for the last case and so remove one misleading use of the func keyword.

[mibk]

I've ran a few times into a situation where a SimpleStmt couldn't have been used, so I needed to nest the code in an outer if, which made the code a bit uglier (IMO). E.g.:

if x != nil {
	if _, ok := x.Foo(); ok {
		// …
	}
}

Under this proposal I could use:

if x != nil && (_, ok := x.Foo(); ok) {
	// …
}

Other times I needed to use the value omitted in the previous snipped:

if x != nil && (v, ok := x.Foo(); ok) {
	v.Bar()
	// …
}

which wouldn't be possible under this proposal:

I propose to allow to define variables scoped to an expression […]

So a question is, whether it needs to be scoped to that expression.

---

An argument against this proposal is that it introduces yet another way how to achieve the same thing, which is generally a bad thing. In order to return just a single value from a call to a function returning more than one value the idiom is to write something like:

func _() error {
	// …
	_, err := q.Exec()
	return err
}

Now an alternative idiom would be a possibility:

func _() error {
	// …
	return (_, err := q.Exec(); err)
}

for people who find it tempting to reduce the number of lines. Surely one might find many other examples where this proposal makes the language more complex than it needs to be.

I can't decide whether I'd be for or against this proposal.

[quenbyako]

@mibk well, this example

if x != nil {
	if _, ok := x.Foo(); ok {
		// …
	}
}

could be written as

if _, ok := x.Foo(); ok && x != nil {
    // …
}

So it's not required to accept this proposal for if cases, BUT in this situation

func _() error {
  // …
  _, err := q.Exec()
  return err
}

this feature could be useful.

Main problem is that go is not so expressive as other languages, so i think this is why this proposal exist. But honestly, for last example its way more useful to use if-style in return like return _, err := q.Exec(); err, which is, well, more obvious and it looks more go-way imho

[mibk]

@mibk well, this example

if x != nil {
	if _, ok := x.Foo(); ok {
		// …
	}
}

could be written as

if _, ok := x.Foo(); ok && x != nil {
    // …
}

That' only possible when it's safe to call x.Foo() on nil, which most of the time isn't the case.

[quenbyako]

btw, good point, i didn't even think about that.

[ianlancetaylor]

Note that this would permit considerable complexity in expressions.

var a int
a = (a := (a := a; a + 1); a + 1))

I think that would set a to 2.

It's true that we have this feature in if and switch statements, but they are syntactically limited there: you can't have a recursive short variable declaration, as this new feature would permit.

[dolmen]

It's true that we have this feature in if and switch statements, but they are syntactically limited there: you can't have a recursive short variable declaration, as this new feature would permit.

Well, such obfuscation is already available:

var a int
a = func() int { a := func() int { a := a; return a + 1 }(); return a + 1 }()

Go playground

[ianlancetaylor]

True, there are many ways to complicate code. Still, that doesn't really seem comparable to me.

[atdiar]

Looks really unreadable to me. And perhaps for no obvious advantage: in the case of dealing with error, a union type would be clearer and would force the user to do the error check most probably.

But yes, your proposal seems to impact readability quite a lot, quite unfortunately amongst other things.

[ianlancetaylor]

Go is not a language that places a high value on terseness. It prefers clarity.

One of the primary goals mentioned above (allocation of pointers) can be addressed through helper functions.

Having a shorter notation for function literals may help some of these cases; see #21498.

Based on the discussion above, and the emoji voting, this is a likely decline. Leaving open for four weeks for final comments.

[ianlancetaylor]

No further comments.