Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JIT: IP mapping for method doesn't contain IL offset corresponding to the one of the sequence points #9064

Closed
kbaladurin opened this issue Oct 5, 2017 · 9 comments
Assignees
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI bug
Milestone

Comments

@kbaladurin
Copy link
Member

For following example:

using System;

class DivNumbers
{
    static void Main()
    {
        int x = 0;
        foreach (int number in SomeNumbers()) {
            x++;
        }
        x++;
        x++;
    }
      
    public static System.Collections.IEnumerable SomeNumbers()  
    {
        yield return 1;
        yield return 2;
        yield return 3;
        yield return 4;
        yield return 5;
        yield return 6;
        yield return 7;
        yield return 8;
    }
}

pdb contains following sequence points for MoveNext method:

<method containingType="DivNumbers+&lt;SomeNumbers&gt;d__1" name="MoveNext">
      <sequencePoints>
        <entry offset="0x0" hidden="true" document="1" />
        <entry offset="0x5d" startLine="16" startColumn="5" endLine="16" endColumn="6" document="1" />
        <entry offset="0x5e" startLine="17" startColumn="9" endLine="17" endColumn="24" document="1" />
        <entry offset="0x73" hidden="true" document="1" />
        <entry offset="0x7a" startLine="18" startColumn="9" endLine="18" endColumn="24" document="1" />
        <entry offset="0x8f" hidden="true" document="1" />
        <entry offset="0x96" startLine="19" startColumn="9" endLine="19" endColumn="24" document="1" />
        <entry offset="0xab" hidden="true" document="1" />
        <entry offset="0xb2" startLine="20" startColumn="9" endLine="20" endColumn="24" document="1" />
        <entry offset="0xc7" hidden="true" document="1" />
        <entry offset="0xce" startLine="21" startColumn="9" endLine="21" endColumn="24" document="1" />
        <entry offset="0xe3" hidden="true" document="1" />
        <entry offset="0xea" startLine="22" startColumn="9" endLine="22" endColumn="24" document="1" />
        <entry offset="0xff" hidden="true" document="1" />
        <entry offset="0x106" startLine="23" startColumn="9" endLine="23" endColumn="24" document="1" />
        <entry offset="0x11b" hidden="true" document="1" />
        <entry offset="0x122" startLine="24" startColumn="9" endLine="24" endColumn="24" document="1" />
        <entry offset="0x137" hidden="true" document="1" />
        <entry offset="0x13e" startLine="25" startColumn="5" endLine="25" endColumn="6" document="1" />
      </sequencePoints>
      <scope startOffset="0x0" endOffset="0x140" />
    </method>

But IP mapping for compiled MoveNext method doesn't contain IL offset for the last sequence point (at the line 25):

IP mapping count : 44
IL offs PROLOG : 0x00000000 ( STACK_EMPTY )
IL offs NO_MAP : 0x00000017 ( STACK_EMPTY )
IL offs 0x0000 : 0x0000002B ( STACK_EMPTY )
IL offs 0x0007 : 0x00000035 ( STACK_EMPTY )
IL offs 0x0031 : 0x00000056 ( STACK_EMPTY )
IL offs 0x0033 : 0x00000059 ( STACK_EMPTY )
IL offs 0x0035 : 0x0000005C ( STACK_EMPTY )
IL offs 0x0037 : 0x0000005F ( STACK_EMPTY )
IL offs 0x0039 : 0x00000065 ( STACK_EMPTY )
IL offs 0x003B : 0x0000006B ( STACK_EMPTY )
IL offs 0x0040 : 0x00000071 ( STACK_EMPTY )
IL offs 0x0045 : 0x00000077 ( STACK_EMPTY )
IL offs 0x004A : 0x0000007D ( STACK_EMPTY )
IL offs 0x004F : 0x00000083 ( STACK_EMPTY )
IL offs 0x0056 : 0x0000008E ( STACK_EMPTY )
IL offs 0x005D : 0x00000099 ( STACK_EMPTY )
IL offs 0x005E : 0x0000009A ( STACK_EMPTY )
IL offs 0x006A : 0x000000C9 ( STACK_EMPTY )
IL offs 0x0073 : 0x000000D9 ( STACK_EMPTY )
IL offs 0x007A : 0x000000E4 ( STACK_EMPTY )
IL offs 0x0086 : 0x00000113 ( STACK_EMPTY )
IL offs 0x008F : 0x00000123 ( STACK_EMPTY )
IL offs 0x0096 : 0x0000012E ( STACK_EMPTY )
IL offs 0x00A2 : 0x0000015D ( STACK_EMPTY )
IL offs 0x00AB : 0x0000016D ( STACK_EMPTY )
IL offs 0x00B2 : 0x00000178 ( STACK_EMPTY )
IL offs 0x00BE : 0x000001A7 ( STACK_EMPTY )
IL offs 0x00C7 : 0x000001B7 ( STACK_EMPTY )
IL offs 0x00CE : 0x000001C2 ( STACK_EMPTY )
IL offs 0x00DA : 0x000001F1 ( STACK_EMPTY )
IL offs 0x00E3 : 0x00000201 ( STACK_EMPTY )
IL offs 0x00EA : 0x0000020C ( STACK_EMPTY )
IL offs 0x00F6 : 0x0000023B ( STACK_EMPTY )
IL offs 0x00FF : 0x0000024B ( STACK_EMPTY )
IL offs 0x0106 : 0x00000256 ( STACK_EMPTY )
IL offs 0x0112 : 0x00000285 ( STACK_EMPTY )
IL offs 0x011B : 0x00000292 ( STACK_EMPTY )
IL offs 0x0122 : 0x0000029D ( STACK_EMPTY )
IL offs 0x012E : 0x000002CC ( STACK_EMPTY )
IL offs NO_MAP : 0x000002D9 ( STACK_EMPTY )
IL offs EPILOG : 0x000002DE ( STACK_EMPTY )
IL offs 0x0137 : 0x000002E4 ( STACK_EMPTY )
IL offs NO_MAP : 0x000002F1 ( STACK_EMPTY )

IL code of the MoveNext method is following:

IL to import:
IL_0000  02                ldarg.0     
IL_0001  7b 01 00 00 04    ldfld        0x4000001
IL_0006  0a                stloc.0     
IL_0007  06                ldloc.0     
IL_0008  45 09 00 00 00 02 00 00 00 04 00 00 00 06 00 00 00 08 00 00 00 0a 00 00 00 0f 00 00 00 14 00 00 00 19 00 00 00 1e 00 00 00 switch      
IL_0031  2b 21             br.s         33 (IL_0054)
IL_0033  2b 21             br.s         33 (IL_0056)
IL_0035  2b 3c             br.s         60 (IL_0073)
IL_0037  2b 56             br.s         86 (IL_008f)
IL_0039  2b 70             br.s         112 (IL_00ab)
IL_003b  38 87 00 00 00    br           135 (IL_00c7)
IL_0040  38 9e 00 00 00    br           158 (IL_00e3)
IL_0045  38 b5 00 00 00    br           181 (IL_00ff)
IL_004a  38 cc 00 00 00    br           204 (IL_011b)
IL_004f  38 e3 00 00 00    br           227 (IL_0137)
IL_0054  16                ldc.i4.0    
IL_0055  2a                ret         
IL_0056  02                ldarg.0     
IL_0057  15                ldc.i4.m1   
IL_0058  7d 01 00 00 04    stfld        0x4000001
IL_005d  00                nop         
IL_005e  02                ldarg.0     
IL_005f  17                ldc.i4.1    
IL_0060  8c 12 00 00 01    box          0x1000012
IL_0065  7d 02 00 00 04    stfld        0x4000002
IL_006a  02                ldarg.0     
IL_006b  17                ldc.i4.1    
IL_006c  7d 01 00 00 04    stfld        0x4000001
IL_0071  17                ldc.i4.1    
IL_0072  2a                ret         
IL_0073  02                ldarg.0     
IL_0074  15                ldc.i4.m1   
IL_0075  7d 01 00 00 04    stfld        0x4000001
IL_007a  02                ldarg.0     
IL_007b  18                ldc.i4.2    
IL_007c  8c 12 00 00 01    box          0x1000012
IL_0081  7d 02 00 00 04    stfld        0x4000002
IL_0086  02                ldarg.0     
IL_0087  18                ldc.i4.2    
IL_0088  7d 01 00 00 04    stfld        0x4000001
IL_008d  17                ldc.i4.1    
IL_008e  2a                ret         
IL_008f  02                ldarg.0     
IL_0090  15                ldc.i4.m1   
IL_0091  7d 01 00 00 04    stfld        0x4000001
IL_0096  02                ldarg.0     
IL_0097  19                ldc.i4.3    
IL_0098  8c 12 00 00 01    box          0x1000012
IL_009d  7d 02 00 00 04    stfld        0x4000002
IL_00a2  02                ldarg.0     
IL_00a3  19                ldc.i4.3    
IL_00a4  7d 01 00 00 04    stfld        0x4000001
IL_00a9  17                ldc.i4.1    
IL_00aa  2a                ret         
IL_00ab  02                ldarg.0     
IL_00ac  15                ldc.i4.m1   
IL_00ad  7d 01 00 00 04    stfld        0x4000001
IL_00b2  02                ldarg.0     
IL_00b3  1a                ldc.i4.4    
IL_00b4  8c 12 00 00 01    box          0x1000012
IL_00b9  7d 02 00 00 04    stfld        0x4000002
IL_00be  02                ldarg.0     
IL_00bf  1a                ldc.i4.4    
IL_00c0  7d 01 00 00 04    stfld        0x4000001
IL_00c5  17                ldc.i4.1    
IL_00c6  2a                ret         
IL_00c7  02                ldarg.0     
IL_00c8  15                ldc.i4.m1   
IL_00c9  7d 01 00 00 04    stfld        0x4000001
IL_00ce  02                ldarg.0     
IL_00cf  1b                ldc.i4.5    
IL_00d0  8c 12 00 00 01    box          0x1000012
IL_00d5  7d 02 00 00 04    stfld        0x4000002
IL_00da  02                ldarg.0     
IL_00db  1b                ldc.i4.5    
IL_00dc  7d 01 00 00 04    stfld        0x4000001
IL_00e1  17                ldc.i4.1    
IL_00e2  2a                ret         
IL_00e3  02                ldarg.0     
IL_00e4  15                ldc.i4.m1   
IL_00e5  7d 01 00 00 04    stfld        0x4000001
IL_00ea  02                ldarg.0     
IL_00eb  1c                ldc.i4.6    
IL_00ec  8c 12 00 00 01    box          0x1000012
IL_00f1  7d 02 00 00 04    stfld        0x4000002
IL_00f6  02                ldarg.0     
IL_00f7  1c                ldc.i4.6    
IL_00f8  7d 01 00 00 04    stfld        0x4000001
IL_00fd  17                ldc.i4.1    
IL_00fe  2a                ret         
IL_00ff  02                ldarg.0     
IL_0100  15                ldc.i4.m1   
IL_0101  7d 01 00 00 04    stfld        0x4000001
IL_0106  02                ldarg.0     
IL_0107  1d                ldc.i4.7    
IL_0108  8c 12 00 00 01    box          0x1000012
IL_010d  7d 02 00 00 04    stfld        0x4000002
IL_0112  02                ldarg.0     
IL_0113  1d                ldc.i4.7    
IL_0114  7d 01 00 00 04    stfld        0x4000001
IL_0119  17                ldc.i4.1    
IL_011a  2a                ret         
IL_011b  02                ldarg.0     
IL_011c  15                ldc.i4.m1   
IL_011d  7d 01 00 00 04    stfld        0x4000001
IL_0122  02                ldarg.0     
IL_0123  1e                ldc.i4.8    
IL_0124  8c 12 00 00 01    box          0x1000012
IL_0129  7d 02 00 00 04    stfld        0x4000002
IL_012e  02                ldarg.0     
IL_012f  1e                ldc.i4.8    
IL_0130  7d 01 00 00 04    stfld        0x4000001
IL_0135  17                ldc.i4.1    
IL_0136  2a                ret         
IL_0137  02                ldarg.0     
IL_0138  15                ldc.i4.m1   
IL_0139  7d 01 00 00 04    stfld        0x4000001
IL_013e  16                ldc.i4.0    
IL_013f  2a                ret         

During compilation statement corresponding last sequence point was removed:

Removing statement [000029] in BB22 as useless:
               [000029] ------------             *  STMT      void  (IL 0x13E...0x13F)
               [000028] ------------             \--*  RETURN    int   
               [000027] ------------                \--*  CNS_INT   int    0

I've fugured out that this issue was introduced by dotnet/coreclr#13792
Before this changes IP mapping contains all sequence points:

IP mapping count : 52
IL offs PROLOG : 0x00000000 ( STACK_EMPTY )
IL offs NO_MAP : 0x00000023 ( STACK_EMPTY )
IL offs 0x0000 : 0x00000037 ( STACK_EMPTY )
IL offs 0x0007 : 0x00000041 ( STACK_EMPTY )
IL offs 0x0031 : 0x00000062 ( STACK_EMPTY )
IL offs 0x0033 : 0x00000065 ( STACK_EMPTY )
IL offs 0x0035 : 0x00000068 ( STACK_EMPTY )
IL offs 0x0037 : 0x0000006E ( STACK_EMPTY )
IL offs 0x0039 : 0x00000074 ( STACK_EMPTY )
IL offs 0x003B : 0x0000007A ( STACK_EMPTY )
IL offs 0x0040 : 0x00000080 ( STACK_EMPTY )
IL offs 0x0045 : 0x00000086 ( STACK_EMPTY )
IL offs 0x004A : 0x0000008C ( STACK_EMPTY )
IL offs 0x004F : 0x00000092 ( STACK_EMPTY )
IL offs 0x0054 : 0x00000098 ( STACK_EMPTY )
IL offs 0x0056 : 0x000000A2 ( STACK_EMPTY )
IL offs 0x005D : 0x000000AD ( STACK_EMPTY )
IL offs 0x005E : 0x000000AE ( STACK_EMPTY )
IL offs 0x006A : 0x000000DD ( STACK_EMPTY )
IL offs 0x0071 : 0x000000E8 ( STACK_EMPTY )
IL offs 0x0073 : 0x000000F4 ( STACK_EMPTY )
IL offs 0x007A : 0x000000FF ( STACK_EMPTY )
IL offs 0x0086 : 0x0000012E ( STACK_EMPTY )
IL offs 0x008D : 0x00000139 ( STACK_EMPTY )
IL offs 0x008F : 0x00000145 ( STACK_EMPTY )
IL offs 0x0096 : 0x00000150 ( STACK_EMPTY )
IL offs 0x00A2 : 0x0000017F ( STACK_EMPTY )
IL offs 0x00A9 : 0x0000018A ( STACK_EMPTY )
IL offs 0x00AB : 0x00000196 ( STACK_EMPTY )
IL offs 0x00B2 : 0x000001A1 ( STACK_EMPTY )
IL offs 0x00BE : 0x000001D0 ( STACK_EMPTY )
IL offs 0x00C5 : 0x000001DB ( STACK_EMPTY )
IL offs 0x00C7 : 0x000001E7 ( STACK_EMPTY )
IL offs 0x00CE : 0x000001F2 ( STACK_EMPTY )
IL offs 0x00DA : 0x00000221 ( STACK_EMPTY )
IL offs 0x00E1 : 0x0000022C ( STACK_EMPTY )
IL offs 0x00E3 : 0x00000238 ( STACK_EMPTY )
IL offs 0x00EA : 0x00000243 ( STACK_EMPTY )
IL offs 0x00F6 : 0x00000272 ( STACK_EMPTY )
IL offs 0x00FD : 0x0000027D ( STACK_EMPTY )
IL offs 0x00FF : 0x00000289 ( STACK_EMPTY )
IL offs 0x0106 : 0x00000294 ( STACK_EMPTY )
IL offs 0x0112 : 0x000002C3 ( STACK_EMPTY )
IL offs 0x0119 : 0x000002CE ( STACK_EMPTY )
IL offs 0x011B : 0x000002D7 ( STACK_EMPTY )
IL offs 0x0122 : 0x000002E2 ( STACK_EMPTY )
IL offs 0x012E : 0x00000311 ( STACK_EMPTY )
IL offs 0x0135 : 0x0000031C ( STACK_EMPTY )
IL offs 0x0137 : 0x00000325 ( STACK_EMPTY )
IL offs 0x013E : 0x00000330 ( STACK_EMPTY )
IL offs NO_MAP : 0x00000337 ( STACK_EMPTY )
IL offs EPILOG : 0x0000033D ( STACK_EMPTY )
@kbaladurin
Copy link
Member Author

cc @Dmitri-Botcharnikov @ayuckhulk

@kbaladurin
Copy link
Member Author

@dotnet/jit-contrib

@kbaladurin
Copy link
Member Author

The last sequence point corresponds to return statement that was deleted during returns merging:

Thread 1 "corerun" hit Breakpoint 4, Compiler::fgRemoveStmt (this=0x6eb270, block=0x6f1e50, node=0x6f3378, updateRefCount=true)
    at /media/kbaladurin/data/dotnet/forked/coreclr-1/src/jit/flowgraph.cpp:9888
9888	    if (opts.compDbgCode && stmt->gtPrev != stmt && stmt->gtStmtILoffsx != BAD_IL_OFFSET)
(gdb) p block->bbNum
$16 = 22
(gdb) bt
#0  Compiler::fgRemoveStmt (this=0x6eb270, block=0x6f1e50, node=0x6f3378, updateRefCount=true) at /media/kbaladurin/data/dotnet/forked/coreclr-1/src/jit/flowgraph.cpp:9888
dotnet/coreclr#1  0x00007ffff0bc73ff in (anonymous namespace)::MergedReturns::Merge (this=0x7fffffffa170, returnBlock=0x6f1e50, searchLimit=2)
    at /media/kbaladurin/data/dotnet/forked/coreclr-1/src/jit/flowgraph.cpp:8541
dotnet/coreclr#2  0x00007ffff0ba9360 in (anonymous namespace)::MergedReturns::Record (this=0x7fffffffa170, returnBlock=0x6f1e50) at /media/kbaladurin/data/dotnet/forked/coreclr-1/src/jit/flowgraph.cpp:8276
dotnet/coreclr#3  0x00007ffff0ba85cb in Compiler::fgAddInternal (this=0x6eb270) at /media/kbaladurin/data/dotnet/forked/coreclr-1/src/jit/flowgraph.cpp:8834
dotnet/coreclr#4  0x00007ffff0cfb096 in Compiler::fgMorph (this=0x6eb270) at /media/kbaladurin/data/dotnet/forked/coreclr-1/src/jit/morph.cpp:17812
dotnet/coreclr#5  0x00007ffff0b61571 in Compiler::compCompile (this=0x6eb270, methodCodePtr=0x7fffffffaa68, methodCodeSize=0x7fffffffafdc, compileFlags=0x7fffffffaa80)
    at /media/kbaladurin/data/dotnet/forked/coreclr-1/src/jit/compiler.cpp:4540
...

Block 22:

Importing BB22 (PC=311) of '<SomeNumbers>d__1:MoveNext():bool:this'
    [ 0] 311 (0x137) ldarg.0
    [ 1] 312 (0x138) ldc.i4.m1 -1
    [ 2] 313 (0x139) stfld 04000001

               [000026] ------------             *  STMT      void  (IL 0x137...  ???)
               [000023] ------------             |  /--*  CNS_INT   int    -1
               [000025] -A-XG-------             \--*  ASG       int   
               [000024] ---XG--N----                \--*  FIELD     int    <>1__state
               [000022] ------------                   \--*  LCL_VAR   ref    V00 this         

    [ 0] 318 (0x13e) ldc.i4.0 0
    [ 1] 319 (0x13f) ret

               [000029] ------------             *  STMT      void  (IL 0x13E...  ???)
               [000028] ------------             \--*  RETURN    int   
               [000027] ------------                \--*  CNS_INT   int    0

Maybe we should to disable returns merging for debug mode. What is the proper way to fix it?

@RussKeldorph
Copy link
Contributor

@dotnet/jit-contrib @JosephTremoulet

@JosephTremoulet
Copy link
Contributor

We sometimes must merge returns for correctness, and have been doing so since before dotnet/coreclr#13792. Evidently, we were doing it in a way that preserved the sequence points before, and that's what dotnet/coreclr#13792 broke. In particular, dotnet/coreclr#13792 added a code path that separates out returns of constants, and that new path must be dropping the sequence point. Note that we have an open question/issue about what's the best behavior for this return merging/separation in minopts and debug (#13915). I'm not sure exactly what IR constraints "preserve the sequence points" translates to, presumably someone on @dotnet/jit-contrib does. So possibly one option is to enforce those constraints on the separate-constant-return path. And I believe the logic for creating non-constant merged returns was refactored but largely unchanged, so another option may be bypassing the constant return check in debug, so that debug always does the temp-introducing return merging that has been preserving sequence points correctly.

@kbaladurin
Copy link
Member Author

@JosephTremoulet thank you for response! I think it's better to figure out how JIT guarantees sequence point preserving (especially for return blocks) and apply this approach in the separate-constant-return path. And if it's impossible we can use second option and bypass the constant return check in debug.

@kbaladurin
Copy link
Member Author

Could anyone explain how JIT guarantees sequence point preserving (especially for return blocks)? Thanks!

@AndyAyersMS
Copy link
Member

Generally speaking, when generating debuggable code, the jit will not do any optimizations or flow transformations. So the sequence point information that is captured initially when the jit IR is created is preserved in the IR throughout compilation.

However return merging is a special case as the jit must also honor the constraints imposed by the runtime. So for this case some flow transformation may be required even when generating debuggable code.

Most returns logically split into two parts: assign the return value, then execute the epilog. The runtime constraint is on the number of epilogs. So the general idea is that the return sequence point associates with the return value assignment, which remain distinct, one per return point. Then the number of epilogs can be altered to suit the constraints of the runtime.

Likely the recent change to merge returns of constants effectively broke this one-to-one mapping and so some return sequence points can get lost. The fix is likely to disable the same return value merging for debuggable codegen.

@kbaladurin
Copy link
Member Author

@AndyAyersMS Thank you for explanation!

AndyAyersMS referenced this issue in AndyAyersMS/coreclr Oct 20, 2017
If we merge constant returns into a common point we may lose track of sequence
points. So inhibit this when we are generating debuggable code.

Fixes #14339.
AndyAyersMS referenced this issue in dotnet/coreclr Oct 23, 2017
If we merge constant returns into a common point we may lose track of sequence
points. So inhibit this when we are generating debuggable code.

Fixes #14339.
@msftgits msftgits transferred this issue from dotnet/coreclr Jan 31, 2020
@msftgits msftgits added this to the 2.1.0 milestone Jan 31, 2020
@ghost ghost locked as resolved and limited conversation to collaborators Dec 20, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI bug
Projects
None yet
Development

No branches or pull requests

5 participants