CarelinkFollower socket timeouts

[kukuchta]

For quite some time there seems to be an issue with retrieving the data from servers in timely manner using the CareLink Follower source.

As a result, the new datapoints and notifications are not downloaded for about 10-15 minutes, I can see the missed reading effects (BG display in red and so on), and then it goes back to normal, when 2 or 3 missed readings are downloaded. It happens randomly, sometimes 2-3 times per hour. There is also an exception logged:

After investigation it seems that the http client in the Carelink Follower that uses default timeout values of 10s does not make it in time with laggy Carelink server.

I have changed the timeouts to:

After some longer tests i can confirm that it solved my issue and I havent seen those Exceptions since applying this fix.

Im using one of the latest official nightly builds:

[kukuchta]

@jamorham @Navid200 Do you think we could push such change?

[user-hash]

I am a new user, but recently the Medtronic servers and the application had some connection problems.
I managed to get the Medtronic app working with a workaround few days ago, and the Carelink follower collector works without any problems.
The values in xDrip are updated 15s after the MiniMed app - grace period set at 15s.
Can you confirm the latency problem is still there?

[kukuchta]

Grace period setting does not mitigate this issue. It only changes the point in time when this collector anticipates new value. If everything is working fine, you get a reply from the server with all values or a reply with almost all values (just without latest one).

The issue is that for me it often happens that i dont get the reply from the server at all. It seemingly takes more than default 10s of timeout. Then I have to wait for another collector run and another try, successful or not.

Im using my modified xDrip for well over a month without such issues, so i believe increasing the timeout values was the right thing. It would be great if this change could be merged.

[Navid200]

@benceszasz

[benceszasz]

Every similar datasource (Nightscout, Dexcom Share, LibreLinkUp) use the default values. It is possible that CareLink servers are slower and less reliable, but before we drastically increase all timeouts (10sec -> 30sec), I think it is necessary to examine when and at what point in the communication this timeout occurs: connect, send request, receive response (server response time). Increasing the wait times can cause unnecessary delays and waits.
I have created a http communication logger that measures the time required for each communication step using a custom OkHttp EvenetListener (for example see: https://square.github.io/okhttp/features/events/). I started measurements with two EU accounts: a standalone Guardian sensor and a 780G pump, since different servers are used as datasources. After more then 12 hours the average communication time seems to be around 150 ms – 200 ms, the longest was about 350 ms.
Please check your http communication with this custom version to have some detailed information about your issue. You can download it from the below shared Google Drive folder:
https://drive.google.com/file/d/1S710wAxE2FBcNJzvi_IbmQj5u0tZAahK/view?usp=sharing
You have to enable debug logging for CareLinkFollowHC tag (set CareLinkFollowHC:d in Extra tags for logging):

This custom version's name is xDrip+ HC (com.eveningoutpost.dexdripclhc – http communication), so you don’t have to uninstall your current version. On the other hand you have to create a new care partner account in CareLink, because you can use one account only in one application.
Measurements:

• DNSlookup: time required to lookup DNS
• Connect: time required to connect to the server
• RequestHeades, RequestBody, etc.: time elapsed during these steps (see: https://square.github.io/okhttp/features/events/)
• Communication: total time of the communication
  Let me know about your results.

[kukuchta]

Thank you for taking your time to respond.

I just disabled collection in my main app, and installed yours. I will try to gather the results overnight, but I can already see that some of the responses come back as they should with Communication at 150-200ms:

but some of them just hang, and return at the timeout value:

I will try to analyze logs and maybe count these failed attempts tomorrow.

[kukuchta]

And another thing, there seems something is wrong with the log timing, the timestamps are not always in order. I havent noticed this before in any xDrip version:

After those entries there is a set of entries for 22:40 and then 22:46.