dotnet core 2.0 application aborts

[pavlexander]

I used to have this issue that 40%+ of a time when I deserialized a JSON (which contained Symbol objects as keys in a dictionary) - my application would "abort". The issue was only reproducible on Linux environment and not on Windows.

The issue was fixed by changing the Symbol data type to string and de-serializing "string" instead. This was easy solution, since string can be easily converted back to Symbol later on. However, when I tried to do so - the problem was back. With a bit of testing I was able to simplify the testing approach to reproduce the issue.

Here is the code:

static void Main(string[] args)
{
	Console.WriteLine("Start");
	
	List<string> symbolsStr = new List<string>() {
		"MODETH", "MTHETH", "MTLETH", "NANOETH", "NAVETH", "NEBLETH", "NEOETH",
		"NULSETH", "OAXETH", "OMGETH", "OSTETH", "PIVXETH", "POEETH", "POWRETH",
		"MDAETH", "PPTETH", "QTUMETH", "RCNETH", "RDNETH", "REQETH", "RLCETH",
		"SALTETH", "SNGLSETH", "SNMETH", "SNTETH", "STEEMETH", "STORJETH", "STRATETH",
		"SUBETH", "TNBETH", "QSPETH", "TNTETH", "MCOETH", "LUNETH", "CNDETH"};

	foreach (var item in symbolsStr)
	{
		var symbol = (Symbol)item;
	}

	Console.WriteLine("End");
	Environment.Exit(0);
}

And here are the results:

user@debian:/srv/scripts/myapp$ dotnet ConsoleApp.dll
Start
End
user@debian:/srv/scripts/myapp$ dotnet ConsoleApp.dll
Start
End
user@debian:/srv/scripts/myapp$ dotnet ConsoleApp.dll
Start
End
user@debian:/srv/scripts/myapp$ dotnet ConsoleApp.dll
Start
Aborted
user@debian:/srv/scripts/myapp$ dotnet ConsoleApp.dll
Start
End
user@debian:/srv/scripts/myapp$ dotnet ConsoleApp.dll
Start
End
user@debian:/srv/scripts/myapp$ dotnet ConsoleApp.dll
Start
Aborted
user@debian:/srv/scripts/myapp$ dotnet ConsoleApp.dll
Start
End
user@debian:/srv/scripts/myapp$ dotnet ConsoleApp.dll
Start
End
user@debian:/srv/scripts/myapp$ dotnet ConsoleApp.dll
Start
End
user@debian:/srv/scripts/myapp$ dotnet ConsoleApp.dll
Start
Aborted
user@debian:/srv/scripts/myapp$ dotnet ConsoleApp.dll
Start
End
user@debian:/srv/scripts/myapp$ dotnet ConsoleApp.dll
Start
End
user@debian:/srv/scripts/myapp$ dotnet ConsoleApp.dll
Start
End
user@debian:/srv/scripts/myapp$

If you look closely at the output - you will then notice that about 2/10 times there is an "Aborted" message. The "End" is never hit. No exceptions are thrown. Basically, sometimes when the object is converted from string to Symbol - .net core application crashes.

Linux is

"Debian 9", x64 bit

"dotnet --version" command output is:

2.1.3

Compiled on Windows with command:

dotnet publish -c release --runtime linux-x64

I would appreciate to hear out any advices on how to solve this issue? Or even to debug it. As You can see from the output - no actual exception is shown. The application just stops.

Also, it would be good if somebody else from the community could confirm this issue. Maybe it's just my machine issue.

Thanks.

[sonvister]

@pavlexander, I had not considered anyone serializing Symbol, but I did override ToString (I guess that counts). The approach you are using seems like the best approach regardless of any errors since the additional Symbol and Asset information is just reference data that may change at any time.

So, my best guess is that there is a race condition between the static initialization of the Cache and the implicit operator using the Cache. Perhaps the implicit operator is occasionally hitting a null Cache.

To potentially fix this, I will move the initialization of the Cache to the static constructor. If you want to test this theory (and confirm a workaround), you can try accessing Symbol.Cache directly (to initialize it) before using the implicit conversion.

[pavlexander]

I have changed the code to following (prepend cache retrieval before "start"):

var cache = Symbol.Cache;
Console.WriteLine("Start");
 List<string> symbolsStr = new List<string>() { ...

The app would still Abort occasionally. But in this case - "Start" is not being printed.

[sonvister]

@pavlexander, at least that narrows it down further. There are the BCH Symbols defined in the static constructor that are referenced by the Cache. Perhaps the Cache is being initialized before the constructor completes. Moving the Cache initialization to the static constructor could still fix that. But, as a workaround, perhaps try something that doesn't access the Cache like var symbol = Symbol.BTC_USDT; or maybe Symbol.UpdateCacheAsync (which might be a good idea regardless ...I've seen Symbols removed).

[pavlexander]

Just a moment ago tested with:

            var symbol = Symbol.MOD_ETH;
            symbol = Symbol.MTH_ETH;
            symbol = Symbol.MTL_ETH;
            symbol = Symbol.NANO_ETH;
            symbol = Symbol.NAV_ETH;
            symbol = Symbol.NEBL_ETH;
            symbol = Symbol.NEO_ETH;
            symbol = Symbol.NULS_ETH;
            symbol = Symbol.OAX_ETH;
            symbol = Symbol.OMG_ETH;
            symbol = Symbol.OST_ETH;
            symbol = Symbol.PIVX_ETH;
            symbol = Symbol.POE_ETH;
            symbol = Symbol.POWR_ETH;
// ...

            Console.WriteLine("Start");

            List<string> symbolsStr = new List<string>() {...

Same error. I will try out some more cases and see if any of them not cause abortion..

[pavlexander]

Just substituted Symbol, with Asset in the code above and checked if the issue there is for Assets as well. I couldn't reproduce the issue. At least that one we can exclude. So the issue seems to be related with Symbol initialization only. To fix the issue I tried to dispose of static Symbol constructor in the source code - it didn't resolve the issue.

[pavlexander]

retesting of this issue is a breeze now.

      static void Main(string[] args)
       {
           // compile like:
           // dotnet publish -c release -r debian-x64

           // linux command to run:
           // while sleep 0.5 ; do dotnet BinanceTestConsole.dll ; done

           //var asset = Asset.ADA; //ok
           //var symbol = Symbol.MOD_ETH; // error
           var cache = Symbol.Cache; // error
       }

For anyone interested.

The linux command will execute dotnet application once in half-a-second time. Therefore, if at least one "Aborted" is printed within 20 second interval - you can consider the test as not passed.

[pavlexander]

Alright,

To determine what's causing the issue - i started to strip down Symbol class apart, until there was almost nothing left. To say, I have got rid of:

• static costructor
• any logic inside non-static constructor
• operator implementations (==, etc)
• static dictionary (caches)
• empty Symbols (4, that were with redirects)
• and a couple of more things

Basically, what I am left with now (almost) are sections of Symbols for BTC, ETC and BNB, ..

• If I remove the BTC section only, and all the rest remain, then the issue is gone
• If I remove every section besides BTC (hence BTC is the only left), then the issue is gone as well.

When I say that the issue is gone _ I mean that I was rerunning the above code for atleast 2 minutes and there were no Abortions. I think it's a combination of all 4 sections is what's causing the issue. Could it be a dot net core limitation of some kind, for the number of static properties to be initialized in one class? Or something?

Additionally, I have tried marking Symbol class as partial and moving 4 sections into 3 different partial classes (3 and 4 sections together), leaving methods inside the first, main, fourth partial class. This did not yield any good results as well.

My discovery does not make this issue resolution any easier, @sonvister
I have no idea what to do next..

Hope that will be of some help.

[pavlexander]

First of all, I am extremely sorry so spamming. I tend to post messages as I progress on the matter of this discussion.

Secondly - the issue is now resolved :) Well, I am now not sure if there is such word as resolved now. I mean that after 5 minutes of task run - there are 0 abortions.

Following last post thematics - I continued to strip down symbol initialization.

I firstly disposed of .00x0m and made them look like .m.
After that I started to get rid of last 2 parameters, since they were the same for all symbols:
, true, new[] { OrderType.Limit, OrderType.LimitMaker, OrderType.Market, OrderType.StopLossLimit, OrderType.TakeProfitLimit });

that is when I first noticed that the issue was gone. After that I added 2 parameters back, and chnaged the signature to following (for all symbols):

public static readonly Symbol NEO_USDT = new Symbol(SymbolStatus.Trading, Asset.NEO, Asset.USDT, (0.001m, 10000000m, 0.001m), (0.001m, 10000000m, 0.001m), 10m, true, new List<OrderType>() { OrderType.Limit, OrderType.LimitMaker, OrderType.Market, OrderType.StopLossLimit, OrderType.TakeProfitLimit });

I.e. instead of using anonymous type of object new[] {} I am using List<OrderType>().
Strangely enough, with this change - the issue is now gone!

It seems like the core of the framework is having some issues when a whole bunch of anonymous objects are created. Would It be possible to apply above fix for you API, @sonvister ? As I recall there was some code which was generating Symbols dynamically. That's probably The place :)

Cheers

[sonvister]

@pavlexander, no worries, someone might find the saga interesting. It is certainly a good example of debugging (zeroing-in on the root cause). I'll create 0.2.0-alpha31 with this (unexpected) change and move the initialization of properties/fields to the static constructor (seems like a good idea).

But, first, would you confirm if new OrderType[] is sufficient (since a list is not needed and for curiosity sake). Thanks for your efforts.

[sonvister]

@pavlexander, I've released 0.2.0-alpha31 with updates for this issue (I used OrderType[], but I'll switch to List<OrderType> if necessary).

[pavlexander]

@sonvister believe it or not, but OrderType[] has the same effect as new[]. I am getting almost immediate Aborts when either of the 2 is used. No such issue with List. If possible, I would appreciate if you used the latter :)

Thank you for your library and support!

[sonvister]

@pavlexander, no problem. 0.2.0-alpha32 released with List<OrderType>. I'll reopen if the issue persists.

[pavlexander]

Thanks!
I will file a report to dotnet repository and see if Microsoft fellas can comment on this.

[sonvister]

@pavlexander, it may help to also move the initialization of all the readonly symbols to the static constructors as well in both Symbol and Asset.

[sonvister]

@pavlexander, if you are not feed-up with testing, I uploaded an alternative NuGet package with the GitHub 0.2.0-alpha32 release. I moved initialization of the symbols to the static constructor and reverted back to new [] { } (for testing) and pushed the code too. I will go back to List<OrderType> if that really is the fix. Thanks.

[pavlexander]

@sonvister, no problem.
Latest v32 revision seems to be working just fine, without exceptions. Do you have any clue what was causing aborts in the old version of code? is it the sequence of initialization?

[sonvister]

@pavlexander, by latest, are you referring to the v32 from NuGet or the v32 alternative I posted on GitHub, or both? If you are referring to the former, then it is probably working because of the List change. Could you also test the .nuget package posted on GitHub?

If you are referring to the later (or both), great, I'd say it has to do with the sequence of initialization and possible indeterminate timing of the initialization and by putting them all in the static constructor makes the initialization sequential.

[pavlexander]

@sonvister I was referring to v32 from NuGet. For the github version - should I just download the latest code source or is there a hack in VS?

[sonvister]

You could get the source code and publish a .nuget package if you prefer or just download the already published .nuget package from GitHub. From there you switch to a local NuGet package source (a configured local folder) and install Binance from there. Since the versions are the same, you may need to remove first then install.

[pavlexander]

if I understood correctly, the second approach is:

1. to get the source from git (master?)
2. compile it
3. substitute DLLs in C:\Users\Xxxx.nuget\packages\binance\0.2.0-alpha32\lib with newly compiled DLLs
4. remove and re-add Binance Nuget reference to a project (in case if I have existing one)

I have never done this before but it looks like a way to go. let me check.

[sonvister]

Well... I'll just upload 0.2.0-alpha33.

[pavlexander]

I just followed the tutorial above. The result could have never been worse :)
100% Aborted cases :)

[sonvister]

How about 0.2.0-alpha33?

[pavlexander]

After upgrading to newer version in Nuget - I was getting:

Unhandled Exception: System.IO.FileLoadException: Could not load file or assembly 'Binance, Version=0.2.0.33, Culture=neutral, PublicKeyToken=null'. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
Aborted

Any good approaches on fixing that? My solution was to clean up Binance reference, removal of Bin + Obj folders, closing and reopening VS, and then re-adding of reference v33, and only then then issue was gone.. This is not the first time I see this message and it's annoying.

As for tests - they still fail.. same 100% aborted results.

[sonvister]

I suspect that FileLoadException is due to the hacking of the NuGet package. As for v33 not working, I'll revert back to v32 code with List and call it v34. I don't feel any better about that fix, but at least it was working for you.

[pavlexander]

@sonvister the issue described in this ticket turned into something bigger. The fix is planned in 2.1.0 release of dotnetcore. After it is released - feel free to revert the data to the type of your wish. Thank you for keeping it as List type for now.