runtime: goroutine stack exceeds 1000000000-byte limit

[andreas-schwab]

When building 1.18beta1 on riscv64 with QEMU linux-user emulation, the compiler runs out of stack space.

https://build.opensuse.org/package/live_build_log/openSUSE:Factory:RISCV/go1.18/standard/riscv64

Building Go toolchain3 using go_bootstrap and Go toolchain2.
internal/unsafeheader
internal/goexperiment
internal/goos
internal/goarch
runtime/internal/atomic
internal/cpu
internal/race
sync/atomic
unicode
internal/abi
runtime/internal/math
runtime/internal/sys
internal/bytealg
unicode/utf8
internal/itoa
math/bits
# unicode
runtime: goroutine stack exceeds 1000000000-byte limit
runtime: sp=0xc0411ff620 stack=[0xc021200000, 0xc041200000]
fatal error: stack overflow

runtime stack:
runtime.throw({0x9a8902, 0xe})
        /usr/lib64/go/1.18/src/runtime/panic.go:992 +0x58
runtime.newstack()
        /usr/lib64/go/1.18/src/runtime/stack.go:1101 +0x544
runtime.morestack()
        /usr/lib64/go/1.18/src/runtime/asm_riscv64.s:201 +0x58

goroutine 1 [running]:
cmd/compile/internal/types.(*Sym).Linksym(0xc000893360)
        /usr/lib64/go/1.18/src/cmd/compile/internal/types/sym.go:75 +0xc fp=0xc0411ff620 sp=0xc0411ff620 pc=0x1a272c
cmd/compile/internal/ir.(*Name).Linksym(...)
        /usr/lib64/go/1.18/src/cmd/compile/internal/ir/name.go:237
cmd/compile/internal/staticinit.(*Schedule).StaticAssign(0xc0411ffbf0, 0xc001c215f0, 0x18, {0xb28760, 0xc0017f8900}, 0xc0009a1d50)
        /usr/lib64/go/1.18/src/cmd/compile/internal/staticinit/sched.go:275 +0x1d4 fp=0xc0411ff6f8 sp=0xc0411ff620 pc=0x74127c
cmd/compile/internal/staticinit.(*Schedule).StaticAssign.func1({0x6, 0x199f100}, 0xc001c215f0, 0x18, {0xb28760, 0xc0017f8900})
        /usr/lib64/go/1.18/src/cmd/compile/internal/staticinit/sched.go:203 +0x68 fp=0xc0411ff768 sp=0xc0411ff6f8 pc=0x741f80
cmd/compile/internal/staticinit.(*Schedule).StaticAssign(0xc0411ffbf0, 0xc001c215f0, 0x0, {0xb28760, 0xc0017f8a00}, 0xc0009a1b90)
        /usr/lib64/go/1.18/src/cmd/compile/internal/staticinit/sched.go:293 +0xd58 fp=0xc0411ff840 sp=0xc0411ff768 pc=0x741e00
cmd/compile/internal/staticinit.(*Schedule).StaticAssign.func1({0x6, 0x199a050}, 0xc001c215f0, 0x0, {0xb28760, 0xc0017f8a00})
        /usr/lib64/go/1.18/src/cmd/compile/internal/staticinit/sched.go:203 +0x68 fp=0xc0411ff8b0 sp=0xc0411ff840 pc=0x741f80
cmd/compile/internal/staticinit.(*Schedule).StaticAssign(0xc0411ffbf0, 0xc000b05110, 0x0, {0xb27c70, 0xc0017f71d0}, 0xc0009ecbd0)
        /usr/lib64/go/1.18/src/cmd/compile/internal/staticinit/sched.go:254 +0x618 fp=0xc0411ff988 sp=0xc0411ff8b0 pc=0x7416c0
cmd/compile/internal/staticinit.(*Schedule).tryStaticInit(0xc0411ffbf0, {0xb27f90, 0xc0017f7220})
        /usr/lib64/go/1.18/src/cmd/compile/internal/staticinit/sched.go:76 +0x170 fp=0xc0411ff9e8 sp=0xc0411ff988 pc=0x7401b0
cmd/compile/internal/staticinit.(*Schedule).StaticInit(0xc0411ffbf0, {0xb27f90, 0xc0017f7220})
        /usr/lib64/go/1.18/src/cmd/compile/internal/staticinit/sched.go:49 +0x34 fp=0xc0411ffa60 sp=0xc0411ff9e8 pc=0x73feb4
cmd/compile/internal/pkginit.Task()
        /usr/lib64/go/1.18/src/cmd/compile/internal/pkginit/init.go:98 +0x81c fp=0xc0411ffc18 sp=0xc0411ffa60 pc=0x8970e4
cmd/compile/internal/gc.Main(0x9d2620)
        /usr/lib64/go/1.18/src/cmd/compile/internal/gc/main.go:265 +0xe98 fp=0xc0411fff10 sp=0xc0411ffc18 pc=0x8b3d08
main.main()
        /usr/lib64/go/1.18/src/cmd/compile/main.go:55 +0x138 fp=0xc0411fff80 sp=0xc0411fff10 pc=0x8db868
runtime.main()
        /usr/lib64/go/1.18/src/runtime/proc.go:255 +0x244 fp=0xc0411fffd8 sp=0xc0411fff80 pc=0x48974
runtime.goexit()
        /usr/lib64/go/1.18/src/runtime/asm_riscv64.s:494 +0x4 fp=0xc0411fffd8 sp=0xc0411fffd8 pc=0x77304
encoding
unicode/utf16
internal/goversion
math
runtime
# runtime
runtime: goroutine stack exceeds 1000000000-byte limit
runtime: sp=0xc04107d910 stack=[0xc021080000, 0xc041080000]
fatal error: stack overflow

runtime stack:
runtime.throw({0x9a8902, 0xe})
        /usr/lib64/go/1.18/src/runtime/panic.go:992 +0x58
runtime.newstack()
        /usr/lib64/go/1.18/src/runtime/stack.go:1101 +0x544
runtime.morestack()
        /usr/lib64/go/1.18/src/runtime/asm_riscv64.s:201 +0x58

goroutine 1 [running]:
cmd/compile/internal/types2.(*Checker).multiExpr(0xc00105c000, 0xc002064000, {0xb25850, 0xc000930780})
        /usr/lib64/go/1.18/src/cmd/compile/internal/types2/expr.go:1669 +0xc fp=0xc04107d910 sp=0xc04107d910 pc=0x7dfacc
cmd/compile/internal/types2.(*Checker).exprList(0xc00105c000, {0xc04107dde8, 0x1, 0x1}, 0x0)
        /usr/lib64/go/1.18/src/cmd/compile/internal/types2/call.go:246 +0x74 fp=0xc04107d9e8 sp=0xc04107d910 pc=0x7c6a04
cmd/compile/internal/types2.(*Checker).assignVars(0xc00105c000, {0xc04107ddf8, 0x1, 0x1}, {0xc04107dde8, 0x1, 0x1})
        /usr/lib64/go/1.18/src/cmd/compile/internal/types2/assignments.go:392 +0x50 fp=0xc04107dab8 sp=0xc04107d9e8 pc=0x7be9a0
cmd/compile/internal/types2.(*Checker).stmt(0xc00105c000, 0x3, {0xb25340, 0xc0009307c0})
        /usr/lib64/go/1.18/src/cmd/compile/internal/types2/stmt.go:451 +0x38c fp=0xc04107df60 sp=0xc04107dab8 pc=0x8063e4
cmd/compile/internal/types2.(*Checker).stmtList(0xc00105c000, 0x3, {0xc000609960, 0x1, 0x1})
[  647s]        /usr/lib64/go/1.18/src/cmd/compile/internal/types2/stmt.go:124 +0xdc fp=0xc04107dfb0 sp=0xc04107df60 pc=0x80491c
cmd/compile/internal/types2.(*Checker).stmt(0xc00105c000, 0x3, {0xb253d0, 0xc000930740})
        /usr/lib64/go/1.18/src/cmd/compile/internal/types2/stmt.go:544 +0x15c8 fp=0xc04107e458 sp=0xc04107dfb0 pc=0x807620
cmd/compile/internal/types2.(*Checker).stmt(0xc00105c000, 0x3, {0xb256a0, 0xc0009289b0})
        /usr/lib64/go/1.18/src/cmd/compile/internal/types2/stmt.go:563 +0x10d8 fp=0xc04107e900 sp=0xc04107e458 pc=0x807130
cmd/compile/internal/types2.(*Checker).stmtList(0xc00105c000, 0x3, {0xc0009347a0, 0x2, 0x2})
        /usr/lib64/go/1.18/src/cmd/compile/internal/types2/stmt.go:124 +0xdc fp=0xc04107e950 sp=0xc04107e900 pc=0x80491c
cmd/compile/internal/types2.(*Checker).stmt(0xc00105c000, 0x3, {0xb253d0, 0xc000930480})
        /usr/lib64/go/1.18/src/cmd/compile/internal/types2/stmt.go:544 +0x15c8 fp=0xc04107edf8 sp=0xc04107e950 pc=0x807620
cmd/compile/internal/types2.(*Checker).stmt(0xc00105c000, 0x0, {0xb255e0, 0xc000928960})
        /usr/lib64/go/1.18/src/cmd/compile/internal/types2/stmt.go:653 +0xd0c fp=0xc04107f2a0 sp=0xc04107edf8 pc=0x806d64
cmd/compile/internal/types2.(*Checker).stmtList(0xc00105c000, 0x0, {0xc0007b3f80, 0x8, 0x8})
        /usr/lib64/go/1.18/src/cmd/compile/internal/types2/stmt.go:124 +0xdc fp=0xc04107f2f0 sp=0xc04107f2a0 pc=0x80491c
cmd/compile/internal/types2.(*Checker).funcBody(0xc00105c000, 0xc000108480, {0xc0004c4f78, 0x13}, 0xc00136f000, 0xc00092dd40, {0x0, 0x0})
        /usr/lib64/go/1.18/src/cmd/compile/internal/types2/stmt.go:45 +0x2e0 fp=0xc04107f4e8 sp=0xc04107f2f0 pc=0x804100
cmd/compile/internal/types2.(*Checker).funcDecl.func1()
        /usr/lib64/go/1.18/src/cmd/compile/internal/types2/decl.go:814 +0x5c fp=0xc04107f530 sp=0xc04107f4e8 pc=0x7d47cc
cmd/compile/internal/types2.(*Checker).processDelayed(0xc00105c000, 0x0)
        /usr/lib64/go/1.18/src/cmd/compile/internal/types2/check.go:355 +0x38 fp=0xc04107f5b8 sp=0xc04107f530 pc=0x7cb028
cmd/compile/internal/types2.(*Checker).checkFiles(0xc00105c000, {0xc00063dc00, 0x7d, 0x7d})
        /usr/lib64/go/1.18/src/cmd/compile/internal/types2/check.go:304 +0x180 fp=0xc04107f618 sp=0xc04107f5b8 pc=0x7cac90
cmd/compile/internal/types2.(*Checker).Files(...)
        /usr/lib64/go/1.18/src/cmd/compile/internal/types2/check.go:276
cmd/compile/internal/types2.(*Config).Check(0xc001037400, {0x40007fea6c, 0x7}, {0xc00063dc00, 0x7d, 0x7d}, 0xc001037450)
        /usr/lib64/go/1.18/src/cmd/compile/internal/types2/api.go:425 +0x80 fp=0xc04107f658 sp=0xc04107f618 pc=0x7bba90
cmd/compile/internal/noder.checkFiles({0xc00017e400, 0x7d, 0x7d})
        /usr/lib64/go/1.18/src/cmd/compile/internal/noder/irgen.go:65 +0x3e4 fp=0xc04107f7c8 sp=0xc04107f658 pc=0x83b14c
cmd/compile/internal/noder.check2({0xc00017e400, 0x7d, 0x7d})
        /usr/lib64/go/1.18/src/cmd/compile/internal/noder/irgen.go:78 +0x38 fp=0xc04107f908 sp=0xc04107f7c8 pc=0x83b430
cmd/compile/internal/noder.LoadPackage({0xc00012a140, 0x7d, 0x94})
        /usr/lib64/go/1.18/src/cmd/compile/internal/noder/noder.go:90 +0x300 fp=0xc04107fc18 sp=0xc04107f908 pc=0x83f3e8
cmd/compile/internal/gc.Main(0x9d2620)
        /usr/lib64/go/1.18/src/cmd/compile/internal/gc/main.go:191 +0xbd8 fp=0xc04107ff10 sp=0xc04107fc18 pc=0x8b3a48
main.main()
        /usr/lib64/go/1.18/src/cmd/compile/main.go:55 +0x138 fp=0xc04107ff80 sp=0xc04107ff10 pc=0x8db868
runtime.main()
        /usr/lib64/go/1.18/src/runtime/proc.go:255 +0x244 fp=0xc04107ffd8 sp=0xc04107ff80 pc=0x48974
runtime.goexit()
        /usr/lib64/go/1.18/src/runtime/asm_riscv64.s:494 +0x4 fp=0xc04107ffd8 sp=0xc04107ffd8 pc=0x77304
go tool dist: FAILED: /home/abuild/rpmbuild/BUILD/go/pkg/tool/linux_riscv64/go_bootstrap install -gcflags=all= -ldflags=all= -v -a -i cmd/asm cmd/cgo cmd/compile cmd/link: exit status 2

[randall77]

That doesn't make a whole lot of sense to me. Those stacks all look small. It's also suspicious that it just happened to run out of stack space it two totally separate code paths.

I'm more suspicious of QEMU here, frankly. We've had QEMU emulation problems before. Does this run fine on real hardware? If so, it is almost certainly a QEMU issue.

How are you bootstrapping this? One of the failures at least was from the bootstrap process. Are you bootstrapping the build of 1.18 with an already built 1.18?

[andreas-schwab]

This is surely an incompatibility with qemu, but it may be an issue due to difference in the memory layout.

[randall77]

My guess would be TLS.

[cherrymui]

Maybe related to #35470 . The stack bounds check needs to be not preemptible. (When #35470 was fixed the RISCV64 ports was not completed.)

[gopherbot]

Change https://golang.org/cl/373434 mentions this issue: cmd/internal/obj/riscv: mark stack bounds check prologue nonpreemptible

[andreas-schwab]

Thanks, with that fix the package now builds sucessfully.