spec: allow nested def statements and lambda expressions

[alandonovan]

No description provided.

[stepancheg]

Probably worth clarifying that:

• it is not possible to assign to a captured variable (unlike Python with nonlocal keyword)
• captured values can be mutated until captured values are frozen (similar to functions capturing globals)

I have mixed feelings about this proposal.

I have never seen any good code with nested def statements.

Lambdas are somewhat more useful, but anonymous functions in 99% of cases are used in parameters to map and filter functions, which is not the issue with Python/Starlark list comprehensions expressions.

But I heard some people want both nested defs and lambdas.

[alandonovan]

Probably worth clarifying that ...

Done, x2

I have mixed feelings about this proposal.

My two concerns are that some people will try to write Scheme or Haskell in Starlark, and that the memory footprint of a struct containing only closures (to simulate an opaque object) is greater than necessary. (It's approximately 15 words per closure in the Java implementation.) The first problem may be ameliorated by the lack of recursion, and the second can be addressed by optimizations (e.g. combining defaults and freevars Tuples into a single array; grouping all the per-module stuff behind a single reference).

I have never seen any good code with nested def statements.

I don't know what code you've seen, but I've seen tons of good and bad code with nested def statements, and in the absence of classes (the other way to combine code+data), they provide a much needed form of expressiveness. They also help to avoid polluting the toplevel namespace with helper functions that could be local.

[brandjon]

Python 3 does not accept an unparenthesized tuple in either a for or if clause, and it does not accept an unparenthesized lambda in a for clause.

[alandonovan]

Regarding tuples, Python has never allowed an Expression (unparenthesized tuple) as the operand of an 'if' clause, nor would anyone want such a thing, so we needn't mention it. However, Python2 used to allow an Expression as the operand of 'for', and it's a common and natural thing to want, hence we point out that we follow Python3 in using a Test (tuples must be parenthesized).

By itself, using Test would permit lambda to be the operand of a for clause, but Python3 specifically rejects lambda there, I assume to avoid a parsing ambiguity that would arising from a lambda body that is a conditional expression.

I think the wording is correct.

The spec does need to be more explicit about precedence rules around if/else expressions, but I don't think there's anything relevant to say about it here.

[brandjon]

I'm in a rush so not checking the full grammar atm, but aren't there two cases of the concrete grammar production for lambda, that differ in the expressiveness of the body?

[alandonovan]

Yes, but it's really to enforce the correct precedence of if/else expressions. I've added a note in Conditional Expressions.

[alandonovan]

During parsing, the if operator, considered as a postfix operator on
the "true" expression, has higher precedence than else (a prefix
operator on the "false" expression), which in turn has higher
precedence than the lambda prefix operator.

a if b else (c if d else e)          # parens are redundant                                          
(a if b else c) if d else e          # parens are required                                           
                                                                                                     
lambda: (a if b else c)              # parens are redunant                                           
(lambda: a) if b else c              # parens are required                                           
                                                                                                     
a if b else lambda: (c if d else e)  # parens are redundant                                          
a if b else (lambda: c if d else e)  # parens are required                                           
(a if b else lambda: c) if d else e  # parens are required                                           

[brandjon]

value of each free variable

"Value" is wrong, it's the cell (or whatever the user-facing terminology for this concept would be).

[alandonovan]

Changed to "reference to each free variable appearing in the body". Added an example.

[brandjon]

name

Despite it being "anonymous". Probably fine to leave as is though.

[alandonovan]

I think it's fine.