Editorial: Eliminate "weird returns" from tail-call handling

[jmdyck]

As in issue #2400, a "weird return" is when an operation performing a Return transfers control to somewhere other than the operation's caller. This PR discusses weird returns in the spec's handling of tail calls, and suggests a way to eliminate them.

In EvaluateCall, the last few steps say:

 7. Let _result_ be Call(_func_, _thisValue_, _argList_).
 8. Assert: If _tailPosition_ is *true*, the above call will not return here, but
    instead evaluation will continue as if the following return has already occurred.
 9. Assert: ....
10. Return _result_.

As an example, say that function A has a tail-call to function B. So in step 8, tailPosition is true when func is B, and "the above call" presumably refers to the invocation of Call(B, ...) in step 7. So the weird return might be:

• the Returns in the Call operation, or
• the Returns in the [[Call]] internal method that Call(B, ...) invokes.

But none of these has a Note to confirm the weird returns.

Nor is there a Note to say where they return to. When step 8 says "as if the following return has already occurred", it presumably refers to step 10, and yet it can't be that Call(B) (or B.[[Call]]) simply returns to wherever step 10 normally returns (i.e., to some definition of the Evaluation SDO), because that wouldn't make any difference. In order for tail-call handling to work, we must "skip over" (working outward):

• the Return step of EvaluateCall,
• the Return steps of arbitrarily many Evaluation invocations for the portions of A's body that contain B,
• the Return step of EvaluateBody for A,
• the Return step of OrdinaryCallEvaluateBody for A, and
• A.[[Call]]'s step that removes A's context from the stack. (This step is the crucial one to "skip", since PrepareForTailCall (for B) already removed A's context from the stack.)

I suppose the interpretation that makes the most sense (in the world of weird returns) would be for Call(B, ...) to return to wherever Call(A, ...) would have returned to (and so on recursively, if appropriate). However:

• I'm pretty sure there's nothing in the spec that tells me this.

• If that were the intended interpretation, I'd expect EvaluateCall's step 8 to say something more like:

 8. Assert: If _tailPosition_ is *true*, the above call will not return here, but
    instead will return to this point in an outer execution of this operation.

• There's an alternative interpretation, in which B.[[Call]] is deemed to return to wherever A.[[Call]] would have returned. It's probably equivalent to the interpretation given above, but it's a bit fishy that I can't tell which was intended.

• Note that neither Call nor [[Call]] actually "know" if the call in question is a tail call, so they theoretically wouldn't be able to identify the occasions when a Return will be a weird return. (Although EvaluateCall could be tweaked to pass down that information.)

• Maybe readers are supposed to intuit this interpretation based on the stack-pop that PrepareForTailCall does. But if we use the assumed mental model from my original comment in Editorial: Notes that tell you where a Return step returns to #2400, PrepareForTailCall's removal of A's context from the execution context stack would cause PrepareForTailCall to modify its "operation return-target". Modify it to what is unclear, since A's context was never really "resumed", but that's kind of beside the point, since I think it's pretty clear that PrepareForTailCall must simply return to its caller (EvaluateCall). Perhaps there's a different mental model that covers both the cases in Editorial: Notes that tell you where a Return step returns to #2400 and the ones here, but I don't think it's worth looking for. I think we should eliminate weird returns.

---

If we want to adopt a model in which weird returns do not occur, then the spec will have to use a different mechanism to handle tail calls. This PR suggests one way to do so. The changes are fairly minor.

As far as I can tell, the crucial thing is to prevent double-popping. That is, if PrepareForTailCall removes a context from the execution context stack, then the step that would 'normally' remove it must be prevented from doing so. I think the simplest way to accomplish this is to have these removal steps (which all occur in [[Call]] methods) first check if the running execution context is the one they 'expect', and pop it only if it is.

[devsnek]

I like this approach 👍

[bakkot]

Closing in favor of #2495.