Handle only the latest event with the most up to date state

[nolar]

Issue : #42

With this change, if the queue contains a batch of events, react only to the latest one — to follow the "eventual consistency" principle of Kubernetes.

The batch is time-framed to 0.1s, so that it is fast enough for all normal cases when only one event arrives, but slow enough for the initial object listing when multiple events arrive.

Still, some minimal time-frame is needed, as the streaming and parsing of the events inside of the kubernetes library is not immediate.

[zincr]

🤖 zincr found 0 problems , 0 warnings

✅ Large Commits
✅ Approvals
✅ Specification
✅ Dependency Licensing

[zincr]

🤖 zincr found 1 problem , 0 warnings

❌ Approvals
✅ Large Commits
✅ Specification
✅ Dependency Licensing

Details on how to resolve are provided below

---

Approvals

All proposed changes must be reviewed by project maintainers before they can be merged

Not enough people have approved this pull request - please ensure that 1 additional user, who have not contributed to this pull request approve the changes.

• ✅ Approved by PR author @nolar
• ❌ 1 additional approval needed
   

[samurang87]

So basically if the fetching manages to get any events, you restart it to make sure there is only one event fetched? 🤔

[nolar]

@samurang87 Yes. And getting all the events till nothing is left. All events except the last one are therefore ignored — as intended.

[parking52]

Why will the event we are interested in be fetched rapidly ? And why will the number of event decrease ?

[nolar]

@parking52 @samurang87 I didn't get your last comments. What number of events decreased? What exactly is fetched rapidly?

[nolar]

@parking52 @samurang87 See the examples in #42 — there are few objects with few events (e.g. "mycrd-expr1"), of which only the latest is of our interest.

[nolar]

@parking52 @samurang87 This is efficiently an equivalent of this logic:

while queue.qsize() > 0:
    event = await queue.get()

Except that:

• threading.Queue().qsize() on Mac OS (it is not supported), so I used to avoid qsize() and just to not think about it. It is promised to work normally in asyncio.Queue though.
• If the queue is empty (qsize == 0), we should wait for 5.0 seconds until something appears. So, while queue.qsize() > 0: is not a suitable criterion here, unless first_time is introduced. But it makes it same as not nice as double tries. In that case, it would look like this:

event = None
first_time = True
try:
    while first_time or queue.qsize() > 0:
        first_time = False
        event = await asyncio.wait_for(queue.get(), timeout=5.0)
except asyncio.TimeoutError:
    if event is None:
        break

# when (event is not None) or (no timeout on the last event), continue.