genetic backtesting results in many active sims and TypeErrors #1372
Comments
|
just tried running again, but this time specified the following strategies and so far no scrolling or extra running jobs. |
|
also did another test with these Backtesting strategy bollinger,crossover_vwap,dema,macd,momentum,rsi,sar,speed,srsi_macd,stddev,ta_ema,ta_macd,ta_macd_ext,ta_ppo,ta_trix,ta_ultosc,trend_bollinger,trend_ema,trendline,trust_distrust,wavetrend and still no scrolling (been about 5 minutes) so i'm thinking the issue could be with the cci_rsi strategy. |
|
srsi_macd also generates 👍 Bad output detected TypeError: Cannot read property 'currency_capital' of undefined TypeError: Cannot read property 'currency_capital' of undefined |
|
add forex_analytics also to this: |
|
I was also getting this This may be necessary to check for nulls in darwin.js, but I questioned why it was happening. I wondered if the problem was perhaps something to do with many simultaneous fs.fileReadSync happening during the many simulations being run (as node has some issues with many file handles ). So I just wondered what is the proper course here? I may be able to submit a PR but need some direction as to how / where to implement it. |
|
got some Bad output detected TypeError: Cannot read property 'currency_capital' of undefined |
|
Not sure if this applies. But I run multiple backtesters on the same directory (1 system running, multiple logins, each running in the same directory). If I start any of them within the same minute, I think this results in multiple instances trying to use the same generation cache in /simulations/ . My solution is to to wait 1 minute before starting each backtester, that seems to eliminate the currency_capital and replace errors. If my theory is correct the solution may be to make the directory more specifically named, and not just based on time which I think it is now. |
|
I just got the currency_capital undefined while running on a single backtest. I was used a population of 5 and maxCores of 5 so execution of generations are fast. |
|
with the latest pull i am now seeing that the sims are indeed giving an error, shown below. My guess is this error condition may not be handled nicely by darwin, resulting in some hung sims, or what darwin thinks is hung. Doing a test now i see 241 sims active but ps aux shows about 33. When this happens thats when the constant scrolling happens with all the percentages of each sim. Couldn't find simulationResults for 829 TypeError: Cannot read property 'currency_capital' of undefined |
|
tested with a lower pop size. I see there are 20 active sims, supposed to be limited to 16. The first 4 sims in the list 27,31,39,42 did error out as i saw the above output. The issue here seems to be that darwin is waiting for these bad sims to finish even though they have errored out. I'm still not too familiar with the code to know where to look, but hopefully someone can use this as a clue as to what's going on. 27: 82.6%, 31: 54.0%, 39: 98.6%, 42: 62.0%, 71: 85.3%, 82: 96.0%, 83: 59.1%, 86: 79.3%, 87: 62.8%, 89: 61.2%, 92: 79.3%, 93: 44.5%, 94: 80.5%, 95: 37.5%, 96: 30.5%, 97: 60.3%, 98: 84.0%, 99: 40.8%, 100: 19.7%, 101: 227: 82.6%, 31: 54.0%, 39: 98.6%, 42: 62.0%, 71: 87.5%, 82: 98.6%, 83: 61.6%, 86: 81.8%, 87: 64.9%, 89: 63.9%, 92: 85.0%, 93: 47.5%, 94: 84.2%, 95: 42.1%, 96: 37.7%, 97: 69.2%, 98: 93.9%, 99: 49.2%, 100: 23.5%, 101: 127: 82.6%, 31: 54.0%, 39: 98.6%, 42: 62.0%, 71: 90.2%, 82: 99.9%, 83: 63.1%, 86: 84.9%, 87: 69.2%, 89: 68.2%, 92: 93.2%, 93: 50.4%, 94: 95.3%, 95: 46.2%, 96: 43.1%, 97: 76.3%, 98: 97.5%, 99: 54.9%, 100: 28.1%, 101: 227: 82.6%, 31: 54.0%, 39: 98.6%, 42: 62.0%, 83: 82.5%, 87: 89.9%, 89: 91.9%, 93: 75.0%, 95: 83.1%, 96: 84.9%, 100: 66.6%, 101: 81.4%, 102: 50.0%, 103: 92.2%, 104: 19.2%, 105: 46.1%, 106: 45.0%, 107: 70.6%, 108: 0.0%,27: 82.6%, 31: 54.0%, 39: 98.6%, 42: 62.0%, 83: 85.0%, 87: 92.1%, 89: 93.6%, 93: 78.5%, 95: 86.0%, 96: 91.6%, 100: 71.3%, 101: 84.8%, 102: 59.0%, 103: 96.0%, 104: 25.2%, 105: 56.8%, 106: 57.6%, 107: 71.2%, 108: 10.8%27: 82.6%, 31: 54.0%, 39: 98.6%, 42: 62.0%, 83: 90.5%, 87: 97.4%, 89: 98.4%, 93: 86.1%, 95: 95.2%, 96: 97.9%, 100: 79.7%, 101: 97.7%, 102: 74.7%, 104: 33.1%, 105: 75.7%, 106: 87.5%, 107: 72.6%, 108: 28.7%, 109: 16.0%27: 82.6%, 31: 54.0%, 39: 98.6%, 42: 62.0%, 83: 98.9%, 93: 98.4%, 100: 93.8%, 104: 46.4%, 107: 75.2%, 108: 66.3%, 109: 37.3%, 110: 34.4%, 111: 1.6%, 112: 50.0%, 113: 81.2%, 114: 26.7%, 115: 40.8%, 116: 0.4% |
|
closing this out. I reduced the number of sims per darwin, mainly since i'm testing a few different coins. I've also removed some of the strategies that were giving errors, although i'll probably keep testing all to see which ones give errors. |
System information
happens with our without using nice
Describe the problem
While running genetic backtesting i've seeing way more jobs than the 16 i have the core limit set to. This was actually happening well before the introduction of the maxCores parameter. The main issue here is that it results in the sim window scrolling constantly, so when a final output is finally reached and a new generation starts the results of the previous run scroll way up. So sometimes there is no way for me to
view the output results of the sim. Darwin usually starts out only having 16 jobs and after a few error out with some TypeErrors thats when the number of active jobs start increasing. I just kicked off a test and managed to capture a nice log that shows the increase of jobs. I've noticed that the strategy that seems to cause most of these errors is cci_srsi. Adding to that, if i leave out cci_srsi and specifiy a bunch of strategies like "--use_stretegies=bollinger,dema,macd,momentum,rsi,sar,stddev,ta_ema,ta_macd,ta_macd_ext,ta_ppo,tra_trix,ta_ultosc,trend_bollinger,trend_ema,trendline,trust_distrust,wavetrend"
i still see the cci_srsi strategy being tested for some reason.
Source code / logs
./scripts/genetic_backtester/darwin.js --selector=gdax.BTC-USD --days=2 --currency_capital=300 --use_strategies=all --population=101 --period_length=15m --generateLaunch="true" --orde
r_type=maker --maxCores=16
--==Zenbot 4 Genetic Backtester v0.2.2==--
Sun, 18 Feb 2018 19:12:07 GMT
Backtesting strategy all ...
Creating population of 101 ...
=== Simulating generation 0 ===
Backfilling (might take some time) ...
./zenbot.sh backfill --days=2 gdax.BTC-USD
100: 80.8%, 101: 81.5%, 102: 50.7%, 103: 53.4%, 104: 54.5%, 105: 36.1%, 106: 39.7%, 107: 35.8%, 108: 32.8%, 109: 27.1%, 110: 26.0%, 111: 16.0%, 112: 9.5%, 113: 0.7%, 114: 0.3%
Done: 99, Active: 16, Remaining: 1804, Completion: 5.4% Best Balance(10): 323.1653, Slowest(114) ETA: 12m 12sCouldn't find simulationResults for 109
Bad output detected TypeError: Cannot read property 'currency_capital' of undefined
TypeError: Cannot read property 'currency_capital' of undefined
at processOutput (/home/ryan/zenbot/scripts/genetic_backtester/darwin.js:460:15)
at ChildProcess.proc.on (/home/ryan/zenbot/scripts/genetic_backtester/darwin.js:356:16)
at emitTwo (events.js:126:13)
at ChildProcess.emit (events.js:214:7)
100: 87.5%, 101: 87.8%, 102: 54.7%, 103: 58.1%, 104: 56.0%, 105: 37.2%, 106: 43.4%, 107: 37.3%, 108: 35.6%, 109: 27.1%, 110: 32.8%, 111: 25.9%, 112: 17.5%, 113: 7.9%, 114: 9.5%
Done: 99, Active: 17, Remaining: 1803, Completion: 5.5% Best Balance(10): 323.1653, Slowest(113) ETA: 0m 40sCouldn't find simulationResults for 113
Bad output detected TypeError: Cannot read property 'currency_capital' of undefined
TypeError: Cannot read property 'currency_capital' of undefined
at processOutput (/home/ryan/zenbot/scripts/genetic_backtester/darwin.js:460:15)
at ChildProcess.proc.on (/home/ryan/zenbot/scripts/genetic_backtester/darwin.js:356:16)
at emitTwo (events.js:126:13)
at ChildProcess.emit (events.js:214:7)
102: 61.9%, 103: 75.4%, 104: 71.6%, 105: 46.0%, 106: 58.9%, 107: 48.3%, 108: 56.1%, 109: 27.1%, 110: 40.2%, 111: 46.3%, 112: 61.1%, 113: 7.9%, 114: 49.6%, 115: 48.3%, 116: 28.1%, 117: 15.1%, 118: 10.9%, 119: 8.5%
Done: 101, Active: 18, Remaining: 1800, Completion: 5.7% Best Balance(10): 323.1653, Slowest(113) ETA: 1m 50sCouldn't find simulationResults for 103
Bad output detected TypeError: Cannot read property 'currency_capital' of undefined
TypeError: Cannot read property 'currency_capital' of undefined
at processOutput (/home/ryan/zenbot/scripts/genetic_backtester/darwin.js:460:15)
at ChildProcess.proc.on (/home/ryan/zenbot/scripts/genetic_backtester/darwin.js:356:16)
at emitTwo (events.js:126:13)
at ChildProcess.emit (events.js:214:7)
102: 65.6%, 103: 75.4%, 104: 76.7%, 105: 51.6%, 106: 63.2%, 107: 50.5%, 108: 60.6%, 109: 27.1%, 110: 43.6%, 111: 52.3%, 112: 73.0%, 113: 7.9%, 114: 59.0%, 115: 59.1%, 116: 39.5%, 117: 18.1%, 118: 32.0%, 119: 23.4%, 1102: 66.8%, 103: 75.4%, 104: 77.9%, 105: 54.4%, 106: 64.6%, 107: 51.7%, 108: 61.6%, 109: 27.1%, 110: 44.3%, 111: 54.0%, 112: 75.2%, 113: 7.9%, 114: 63.0%, 115: 62.5%, 116: 44.2%, 117: 19.4%, 118: 35.2%, 119: 27.5%, 1102: 68.2%, 103: 75.4%, 104: 79.1%, 105: 56.4%, 106: 66.0%, 107: 52.3%, 108: 62.6%, 109: 27.1%, 110: 44.6%, 111: 56.7%, 112: 77.0%, 113: 7.9%, 114: 67.3%, 115: 66.6%, 116: 48.8%, 117: 21.4%, 118: 40.5%, 119: 32.2%, 1102: 69.4%, 103: 75.4%, 104: 80.3%, 105: 58.5%, 106: 67.9%, 107: 53.2%, 108: 63.1%, 109: 27.1%, 110: 45.6%, 111: 59.1%, 112: 79.2%, 113: 7.9%, 114: 71.6%, 115: 72.0%, 116: 53.0%, 117: 23.2%, 118: 44.9%, 119: 37.4%, 1102: 71.1%, 103: 75.4%, 104: 80.9%, 105: 60.9%, 106: 70.7%, 107: 54.8%, 108: 64.4%, 109: 27.1%, 110: 47.0%, 111: 60.1%, 112: 80.4%, 113: 7.9%, 114: 75.2%, 115: 73.0%, 116: 56.6%, 117: 24.9%, 118: 49.8%, 119: 45.6%, 1102: 72.2%, 103: 75.4%, 104: 81.2%, 105: 62.6%, 106: 72.6%, 107: 55.4%, 108: 65.4%, 109: 27.1%, 110: 47.3%, 111: 61.0%, 112: 81.6%, 113: 7.9%, 114: 77.1%, 115: 74.7%, 116: 57.9%, 117: 25.4%, 118: 54.1%, 119: 51.9%, 1102: 74.2%, 103: 75.4%, 104: 81.8%, 105: 63.9%, 106: 73.7%, 107: 56.3%, 108: 66.4%, 109: 27.1%, 110: 48.3%, 111: 62.2%, 112: 82.8%, 113: 7.9%, 114: 79.2%, 115: 76.4%, 116: 60.3%, 117: 26.2%, 118: 59.5%, 119: 55.2%, 120: 51.3%
Done: 101, Active: 19, Remaining: 1799, Completion: 5.9% Best Balance(10): 323.1653, Slowest(113) ETA: 3m 36sCouldn't find simulationResults for 105
Bad output detected TypeError: Cannot read property 'currency_capital' of undefined
TypeError: Cannot read property 'currency_capital' of undefined
at processOutput (/home/ryan/zenbot/scripts/genetic_backtester/darwin.js:460:15)
at ChildProcess.proc.on (/home/ryan/zenbot/scripts/genetic_backtester/darwin.js:356:16)
at emitTwo (events.js:126:13)
at ChildProcess.emit (events.js:214:7)
102: 76.2%, 103: 75.4%, 104: 83.0%, 105: 64.6%, 106: 75.6%, 107: 57.8%, 108: 67.9%, 109: 27.1%, 110: 49.0%, 111: 63.5%, 112: 83.8%, 113: 7.9%, 114: 80.5%, 115: 78.4%, 116: 63.1%, 117: 26.7%, 118: 62.4%, 119: 58.8%, 1102: 76.8%, 103: 75.4%, 104: 83.3%, 105: 64.6%, 106: 77.5%, 107: 59.3%, 108: 69.2%, 109: 27.1%, 110: 50.0%, 111: 64.7%, 112: 84.7%, 113: 7.9%, 114: 82.6%, 115: 79.4%, 116: 66.8%, 117: 27.2%, 118: 66.3%, 119: 61.8%, 1102: 76.8%, 103: 75.4%, 104: 83.9%, 105: 64.6%, 106: 80.1%, 107: 61.8%, 108: 70.5%, 109: 27.1%, 110: 50.7%, 111: 66.1%, 112: 86.8%, 113: 7.9%, 114: 83.5%, 115: 81.1%, 116: 69.6%, 117: 28.7%, 118: 70.6%, 119: 66.5%, 120: 62.2%, 121: 0.0%
Done: 101, Active: 20, Remaining: 1798, Completion: 5.9% Best Balance(10): 323.1653, Slowest(113) ETA: 4m 11sCouldn't find simulationResults for 115
Bad output detected TypeError: Cannot read property 'currency_capital' of undefined
TypeError: Cannot read property 'currency_capital' of undefined
at processOutput (/home/ryan/zenbot/scripts/genetic_backtester/darwin.js:460:15)
at ChildProcess.proc.on (/home/ryan/zenbot/scripts/genetic_backtester/darwin.js:356:16)
at emitTwo (events.js:126:13)
at ChildProcess.emit (events.js:214:7)
102: 77.4%, 103: 75.4%, 104: 85.1%, 105: 64.6%, 106: 81.0%, 107: 63.0%, 108: 71.5%, 109: 27.1%, 110: 51.4%, 111: 67.6%, 112: 88.4%, 113: 7.9%, 114: 84.7%, 115: 81.1%, 116: 72.2%, 117: 29.7%, 118: 74.3%, 119: 69.8%, 120: 65.3%, 121: 7.2%
Done: 101, Active: 21, Remaining: 1797, Completion: 5.9% Best Balance(10): 323.1653, Slowest(113) ETA: 4m 23s^C
cat sim_115.json
{"iteration":115,"commandString":"./zenbot.sh sim gdax.BTC-USD --period_length=15m --min_periods=31 --markdown_buy_pct=-0.3207612716497521 --markup_sell_pct=-0.7515464269977876 --order_type=maker --sell_stop_pct=40 --buy_stop_pct=0 --profit_stop_enable_pct=1 --profit_stop_pct=8 --ema_acc=0.4436321727900675 --cci_periods=149 --rsi_periods=110 --srsi_periods=40 --srsi_k=26 --srsi_d=40 --oversold_rsi=10 --overbought_rsi=71 --oversold_cci=99 --overbought_cci=21 --constant=0.026275451722087888 --backtester_generation=115 --strategy=cci_srsi --days=2 --currency_capital=300 --generateLaunch=true --maxCores=16 --filename=none","queryStart":"2018-02-16T10:45:00.000Z","queryEnd":"2018-02-18T19:13:40.860Z"}ryan@coin2:~/zenbot/simulations/backtest_201802181112/gen_0$
The text was updated successfully, but these errors were encountered: