StackOverflowError (during training)

[StepHaze]

I try to implement a new game.
Scripts.test_game and Scripts.dummy_run were passed OK.

After I run julia --project -e 'using AlphaZero; Scripts.train("best")',
I get this error message:

Initializing a new AlphaZero environment

Initial report

Number of network parameters: 1,544,262
Number of regularized network parameters: 1,539,712
Memory footprint per MCTS node: 240 bytes

Running benchmark: AlphaZero against MCTS (1000 rollouts)

Progress:   2%|█▌                                                                           |  ETA: 1:25:28StackOverflowError:StackOverflowError:

Stacktrace:
[1]
Stacktrace:Array

 @  [1]./ boot.jl:457Array [inlined]

@  ./ [2]boot.jl:457  [inlined]Array

 @  [2]./ boot.jl:466Array [inlined]

@  ./ [3]boot.jl:466  [inlined]similar

 @  [3]./ array.jl:378similar [inlined]

@  ./ [4]array.jl:378  [inlined]similar

 @  [4]./ abstractarray.jl:783similar [inlined]

@  ./ [5]abstractarray.jl:783  [inlined]_unsafe_getindex

( #unused# [5]:: IndexLinear_unsafe_getindex, (A#unused#::::VectorIndexLinear{Int64}, , AI::::VectorBase.LogicalIndex{Int64}{Int64, StaticArrays.SVector{5, Bool}}, )I
:: @ Base.LogicalIndexBase{Int64, StaticArrays.SVector{5, Bool}} )./
multidimensional.jl:851 @ Base
./ [6]multidimensional.jl:851 _getindex

 @  [6]./ multidimensional.jl:839_getindex [inlined]

@  ./ [7]multidimensional.jl:839  [inlined]getindex

 @  [7]./ abstractarray.jl:1218getindex [inlined]

@  ./ [8]abstractarray.jl:1218  [inlined]available_actions

( game [8]:: AlphaZero.Examples.Best.GameEnvavailable_actions)(
game @ ::AlphaZero.GameInterfaceAlphaZero.Examples.Best.GameEnv )~/Downloads/Ju/AlphaZero.jl/src/
game.jl:320 @ AlphaZero.GameInterface
/Downloads/Ju/AlphaZero.jl/src/ [9]game.jl:320 run_simulation!
( env [9]:: AlphaZero.MCTS.Envrun_simulation!{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}(, envgame::::AlphaZero.MCTS.EnvAlphaZero.Examples.Best.GameEnv{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}; , ηgame::::VectorAlphaZero.Examples.Best.GameEnv{Float64}; , ηroot::::VectorBool{Float64}),
root @ ::AlphaZero.MCTSBool )/Downloads/Ju/AlphaZero.jl/src/
mcts.jl:204 @ AlphaZero.MCTS
/Downloads/Ju/AlphaZero.jl/src/[10]mcts.jl:204 run_simulation!
( env[10]:: AlphaZero.MCTS.Envrun_simulation!{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}(, envgame::::AlphaZero.MCTS.EnvAlphaZero.Examples.Best.GameEnv{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}; , ηgame::::VectorAlphaZero.Examples.Best.GameEnv{Float64}; , ηroot::::VectorBool{Float64}), (repeats 11814 times)root
:: @ BoolAlphaZero.MCTS) (repeats 11814 times)/Downloads/Ju/AlphaZero.jl/src/
mcts.jl:218 @ AlphaZero.MCTS
/Downloads/Ju/AlphaZero.jl/src/[11]mcts.jl:218 explore!
( env[11]:: AlphaZero.MCTS.Envexplore!{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}(, envgame::::AlphaZero.MCTS.EnvAlphaZero.Examples.Best.GameEnv{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}, , nsimsgame::::Int64AlphaZero.Examples.Best.GameEnv),
nsims @ ::AlphaZero.MCTSInt64 )/Downloads/Ju/AlphaZero.jl/src/
mcts.jl:243 @ AlphaZero.MCTS
/Downloads/Ju/AlphaZero.jl/src/[12]mcts.jl:243 think
( p[12]:: MctsPlayerthink{AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}}(, pgame::::MctsPlayerAlphaZero.Examples.Best.GameEnv{AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}}),
game @ ::AlphaZeroAlphaZero.Examples.Best.GameEnv )/Downloads/Ju/AlphaZero.jl/src/
play.jl:198 @ AlphaZero
~/Downloads/Ju/AlphaZero.jl/src/[13]play.jl:198 think

 @ [13]~/Downloads/Ju/AlphaZero.jl/src/ play.jl:259think [inlined]

@  ~/Downloads/Ju/AlphaZero.jl/src/[14]play.jl:259  [inlined]play_game

( gspec[14]:: AlphaZero.Examples.Best.GameSpecplay_game, (playergspec::::TwoPlayersAlphaZero.Examples.Best.GameSpec{MctsPlayer{AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.Batchifier.BatchedOracle{AlphaZero.Batchifier.var"#8#9"}}}, MctsPlayer{AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}}}, ; playerflip_probability::::TwoPlayersFloat64{MctsPlayer{AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.Batchifier.BatchedOracle{AlphaZero.Batchifier.var"#8#9"}}}, MctsPlayer{AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}}});
flip_probability @ ::AlphaZeroFloat64 )~/Downloads/Ju/AlphaZero.jl/src/
play.jl:308 @ AlphaZero
/Downloads/Ju/AlphaZero.jl/src/[15]play.jl:308 (::AlphaZero.var"#simulate_game#70"{TwoPlayers{MctsPlayer{AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.Batchifier.BatchedOracle{AlphaZero.Batchifier.var"#8#9"}}}, MctsPlayer{AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}}}, AlphaZero.Benchmark.var"#5#9"{ProgressMeter.Progress}, Simulator{AlphaZero.Benchmark.var"#4#8"{Env{AlphaZero.Examples.Best.GameSpec, ResNet, NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}}, AlphaZero.Benchmark.Duel}, AlphaZero.Benchmark.var"#net#6"{Env{AlphaZero.Examples.Best.GameSpec, ResNet, NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}}, AlphaZero.Benchmark.Duel}, typeof(record_trace)}, AlphaZero.Examples.Best.GameSpec, SimParams})
( sim_id[15]:: Int64(::AlphaZero.var"#simulate_game#70"{TwoPlayers{MctsPlayer{AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.Batchifier.BatchedOracle{AlphaZero.Batchifier.var"#8#9"}}}, MctsPlayer{AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}}}, AlphaZero.Benchmark.var"#5#9"{ProgressMeter.Progress}, Simulator{AlphaZero.Benchmark.var"#4#8"{Env{AlphaZero.Examples.Best.GameSpec, ResNet, NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}}, AlphaZero.Benchmark.Duel}, AlphaZero.Benchmark.var"#net#6"{Env{AlphaZero.Examples.Best.GameSpec, ResNet, NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}}, AlphaZero.Benchmark.Duel}, typeof(record_trace)}, AlphaZero.Examples.Best.GameSpec, SimParams}))(
sim_id @ ::AlphaZeroInt64 )/Downloads/Ju/AlphaZero.jl/src/
simulations.jl:232 @ AlphaZero
~/Downloads/Ju/AlphaZero.jl/src/[16]simulations.jl:232 macro expansion

 @ [16]~/Downloads/Ju/AlphaZero.jl/src/ util.jl:189macro expansion [inlined]

@  ~/Downloads/Ju/AlphaZero.jl/src/[17]util.jl:189  [inlined](::AlphaZero.Util.var"#9#10"{AlphaZero.var"#68#69"{AlphaZero.Benchmark.var"#5#9"{ProgressMeter.Progress}, Simulator{AlphaZero.Benchmark.var"#4#8"{Env{AlphaZero.Examples.Best.GameSpec, ResNet, NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}}, AlphaZero.Benchmark.Duel}, AlphaZero.Benchmark.var"#net#6"{Env{AlphaZero.Examples.Best.GameSpec, ResNet, NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}}, AlphaZero.Benchmark.Duel}, typeof(record_trace)}, AlphaZero.Examples.Best.GameSpec, SimParams, AlphaZero.var"#48#49"{Channel{Any}}, AlphaZero.var"#make#65"{Channel{Any}}}, UnitRange{Int64}, typeof(vcat), ReentrantLock})

( )[17]
@ (::AlphaZero.Util.var"#9#10"{AlphaZero.var"#68#69"{AlphaZero.Benchmark.var"#5#9"{ProgressMeter.Progress}, Simulator{AlphaZero.Benchmark.var"#4#8"{Env{AlphaZero.Examples.Best.GameSpec, ResNet, NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}}, AlphaZero.Benchmark.Duel}, AlphaZero.Benchmark.var"#net#6"{Env{AlphaZero.Examples.Best.GameSpec, ResNet, NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}}, AlphaZero.Benchmark.Duel}, typeof(record_trace)}, AlphaZero.Examples.Best.GameSpec, SimParams, AlphaZero.var"#48#49"{Channel{Any}}, AlphaZero.var"#make#65"{Channel{Any}}}, UnitRange{Int64}, typeof(vcat), ReentrantLock})AlphaZero.Util( )~/.julia/packages/ThreadPools/hwwUU/src/
macros.jl:261 @ AlphaZero.Util ~/.julia/packages/ThreadPools/hwwUU/src/macros.jl:261

Please help

[StepHaze]

Ok. I took scripts\mcts.jl, changed "tictactoe" to my game and ran it.
After few moves against AI, I got the error message

ERROR: LoadError: StackOverflowError:
Stacktrace:
[1] objectid
@ ./reflection.jl:302 [inlined]
[2] hash
@ ./hashing.jl:25 [inlined]
[3] hash(t::Tuple{AlphaZero.Examples.Best.Board, Int64}, h::UInt64)
@ Base ./tuple.jl:417
[4] hash
@ ./namedtuple.jl:195 [inlined]
[5] hash
@ ./hashing.jl:20 [inlined]
[6] hashindex
@ ./dict.jl:169 [inlined]
[7] ht_keyindex(h::Dict{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.StateInfo}, key::NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}})
@ Base ./dict.jl:284
[8] haskey
@ ./dict.jl:552 [inlined]
[9] state_info(env::AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}, state::NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}})
@ AlphaZero.MCTS ~/Downloads/Ju/AlphaZero.jl/src/mcts.jl:166
[10] run_simulation!(env::AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}, game::AlphaZero.Examples.Best.GameEnv; η::Vector{Float64}, root::Bool)
@ AlphaZero.MCTS ~/Downloads/Ju/AlphaZero.jl/src/mcts.jl:205
[11] run_simulation!(env::AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}, game::AlphaZero.Examples.Best.GameEnv; η::Vector{Float64}, root::Bool) (repeats 23785 times)
@ AlphaZero.MCTS ~/Downloads/Ju/AlphaZero.jl/src/mcts.jl:218
[12] explore!(env::AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}, game::AlphaZero.Examples.Best.GameEnv, nsims::Int64)
@ AlphaZero.MCTS ~/Downloads/Ju/AlphaZero.jl/src/mcts.jl:243
[13] think(p::MctsPlayer{AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}}, game::AlphaZero.Examples.Best.GameEnv)
@ AlphaZero ~/Downloads/Ju/AlphaZero.jl/src/play.jl:202
[14] select_move
@ ~/Downloads/Ju/AlphaZero.jl/src/play.jl:49 [inlined]
[15] select_move(p::TwoPlayers{MctsPlayer{AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}}, Human}, game::AlphaZero.Examples.Best.GameEnv, turn::Int64)
@ AlphaZero ~/Downloads/Ju/AlphaZero.jl/src/play.jl:265
[16] interactive!(game::AlphaZero.Examples.Best.GameEnv, player::TwoPlayers{MctsPlayer{AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}}, Human})
@ AlphaZero ~/Downloads/Ju/AlphaZero.jl/src/play.jl:364
[17] interactive!
@ ~/Downloads/Ju/AlphaZero.jl/src/play.jl:375 [inlined]
[18] interactive!(game::AlphaZero.Examples.Best.GameSpec, white::MctsPlayer{AlphaZero.MCTS.Env{NamedTuple{(:board, :curplayer), Tuple{AlphaZero.Examples.Best.Board, Int64}}, AlphaZero.MCTS.RolloutOracle{AlphaZero.Examples.Best.GameSpec}}}, black::Human)
@ AlphaZero ~/Downloads/Ju/AlphaZero.jl/src/play.jl:377
[19] top-level scope
@ ~/Downloads/Ju/AlphaZero.jl/scripts/mcts.jl:8
[20] include(fname::String)
@ Base.MainInclude ./client.jl:451
[21] top-level scope
@ none:1
in expression starting at /home/haze/Downloads/Ju/AlphaZero.jl/scripts/mcts.jl:8

Please give me the right direction.

[EngrStudent]

I am getting overflow of gpu memory errors too.

[EngrStudent]

I re-ran using tic-tac toe game, and kept the System Monitor and the nvidia-smi tool to track hardware usage before the freeze. My video froze, and the memory was at very high usage, so I had the thought that it was a memory overflow problem.

I then looked through the paramters (params.jl) and found a "memory buffer size" variable that I reduced by an order of magnitude (80k --> 8k). The code ran without crashing but learning was poor. I increased size to 40k, and it both ran and learned. There is a trade-off between "small enough to not crash" and "big enough to not act like a lobotomy". Manually hand-holding that is going to be a pain, but it is one way to limp forward.

[jonathan-laurent]

Thanks for the feedback. I will be adding an option to store memory buffer samples on disk.

[smart-fr]

Thanks for the feedback. I will be adding an option to store memory buffer samples on disk.

Yes please! 😘