Introduction and Improved smoothing for the Libre2

[jbr7rr]

Hi all!

First a small introduction. I was diagnosed T1D in January this year, and started shortly after that on the Libre, and started right away using xDrip and I love it. I work as a embedded software engineer. And in that regard I like to contribute to xDrip.

I've noticed the need for extra smoothing and filtering on the Libre sensors and found this pull request: #1401 And the smoothing algorithm works pretty well in my limited testing so far. To me it seems the PR is abandoned by its author. Is it OK that i pick up his code and make a new PR based on his code and the review comments? I've mailed Smeuthing last week, so far no response.

What I like to do is the following:

• Create new PR for 1401 with its original intent but with review comments reworked, and look to get it merged.
• Integrate the ASG smoothing algorithm into the OOP2 Libre code and create a separate PR for this (after the first one is approved)
• Check if the Noise estimation/filtering implemented for the Libre native in PR1401 is desired for the OOP2 Libre algorithm and implement this accordingly

Please let me know what you think.

[Navid200]

I have no experience with Libre. I have no authority here.
If I were you, I would work with @tzachi-dar.
If he tells you, yes that's a good idea, I would then go ahead and do what you want.
Otherwise, you may spend a lot of time with no final outcome.

[tzachi-dar]

I would wait for @jamorham to respond as he has the final word.

AFAIR #1401 was not merged since it contains kotlin code, and there have been a question weather this would work with old devices. So, I think that first thing to do is to try and move it to java (if you decide to work on this).

Right now oop2 code has a smoothing algorithm that is based on average of the last 5 points (out of last 7 minutes).
The question is whether something better is needed.
So, first thing to remember is that OOP2 is based on the raw values. the patched app is based on the values coming from the libre algorithm, while the main issue is that the libre algorithm is using a prediction algorithm. In other words, if the raw values are 100, 100, 100, 100, 105 patched app will make this as 100, 100, 100, 100, 120 (The 120 is my guess, but it is based on the fact that there is an assumption that BG is starting to go up very fast and as a result in 10 minutes we are going to be very high). As a result the patched app needs a strong smoothing algorithm but the oop2 does not.

One thought about ASG algorithm. ASG is a brilliant thing that only people with strong mathematical background will know how to appreciate. The matrices vectors and error calculation seems very impressive.
That said, the bottom line of what they are doing is given x points (for example 5 points) is do a weighted average.
That is according to them BG = a0x0 +a1 * x1 + a2x2 +a3x3+ a4x4
The only difference from what I'm doing right now is that I choose a0=a1 =...=a4 = 0.2 (they are choosing some other numbers)
Now comes the question, can one choose other numbers that will give better results. I'm sure it is possible. This is why the "Savitzky-Golay filter" has so many articles and citation from people. But the more interesting question I think is will the difference be noticable. My guess it not.

To summarize...
If I were you I would think weather my goal is to invest time on the patched app or the oop2 path.
That said, I'm fine with having a pr submitted that will allow more smoothing options for the oop2. (obviously keep this configurable)
And as I started, please wait for @jamorham response as well.

[jbr7rr]

Thank you for your reply.

I always assumed the patched app worked with raw value's as I never used it myself. I've used the ASG smoothing for the OOP for the last week (both on the OOP2 raw values and the native/non-calibratable values). The OOP2 data can still be a bit jumpy and to me it seems that ASG can handle that better then weighted average. (see screenshot) In the coming days I will log the data of both smoothing algorithms so we can assess the difference in the algorithms.

Also #1401 fixes the Libre trend on the main graph (see screenshot as well)

Before fully committing to it I will await the reply of @jamorham and see if Kotlin is a dealbreaker. Also it might be better to break #1401 into smaller pieces

Also my goal is to invest in the oop2 path, but if we can improve both paths I would gladly do that, as for now it is also the method described in the androidAPS docs.

.

[jbr7rr]

Just a quick update: I did some additional logging in the last period. and Yes there is an advantage in using ASG, but it's responsive data but less smooth (for WA) vs laggy data and smooth (ASG, depending on parameters). So while it might be a good addition, While diving a bit deeper in the code it is a lot of work to rework it to Java. So in the mean time I'm foccusing on other aspects of the OOP2 algorithm, and will post a draft PR or new discussion soon with some improvements on noise detection with temp changes etc. And will pick this up later if desired.