[Bug] New messages sometimes not retrieved while idle

[krackout]

Sometimes e-mail delivery is not happening until I manually refresh (pressing L); it happens in idle state only. Yet mails are mostly received in a timely fashion, I cannot make a correlation when nmail doesn't get the signal (of some kind, don't know the details) that new mail has arrived.

Yet I cannot speak of a bug, haven't got any hard evidence. I have this setting on, idle_fetch_flags=1 in case it's related. I'm thinking of changing server_timestamps=0 to 1 to see if it makes any difference.

It seems to happen mostly on my work (Microsoft 365) account. But since I constantly use it, it's not easy to be certain about it; I'll try to leave it idling in the weekend and send test e-mails to check.

[Kabouik]

Like you, I couldn't really correlate it to anything and so never reported the issue, but my impression is it's a valid one because I do experience it as well, on a work Exchange account and other non-Exchange accounts. I don't know if it is related or not, but sometimes L cannot fetch anything new, while quitting nmail and relaunching it does, as if this would trigger a "deeper" fetching. Or would put nmail out of idle?

…

On 2022-10-07 16:12 krackout ***@***.***> wrote: Sometimes e-mail delivery is not happening until I manually refresh (pressing L); it happens in idle state only. Yet mails are mostly received in a timely fashion, I cannot make a correlation when nmail doesn't get the signal (of some kind, don't know the details) that new mail has arrived. Yet I cannot speak of a bug, haven't got any hard evidence. I have this setting on, `idle_fetch_flags=1` in case it's related. I'm thinking of changing `server_timestamps=0` to 1 to see if it makes any difference. It seems to happen mostly on my work (Microsoft 365) account. But since I constantly use it, it's not easy to be certain about it; I'll try to leave it idling in the weekend and send test e-mails to check. -- Reply to this email directly or view it on GitHub: #112 You are receiving this because you are subscribed to this thread. Message ID: ***@***.***>

[d99kris]

Yes, nmail supports IMAP IDLE. After no IMAP activity for 15 seconds, nmail fetches message uids and then enters idle. Upon idle notification, nmail fetches message uids and exits idle. After 29 mins in idle state, nmail stops and restarts idle.

I have not encountered any issues relating to this myself, but reviewing the implementation I believe I've found some possible improvements. For example we could fetch message uids at every 29 min idle timeout.

However, I'll need to do some more checking and testing on this before I can prepare a fix.

[krackout]

Plus to the other possible improvements, how about reducing time in idle state to 15-20 minutes? Or perhaps leave everything as is and implement a refresh at a time interval option, set by configuration. Στις 8 Οκτωβρίου 2022 12:09:14 μ.μ. EEST, ο/η Kristofer Berggren ***@***.***> έγραψε:

…

Yes, `nmail` supports `IMAP IDLE`. After no IMAP activity for 15 seconds, `nmail` fetches message uids and then enters idle. Upon idle notification, `nmail` fetches message uids and exits idle. After 29 mins in idle state, `nmail` stops and restarts idle. I have not encountered any issues relating to this myself, but reviewing the implementation I believe I've found some possible improvements. For example we could fetch message uids at every 29 min idle timeout. However, I'll need to do some more checking and testing on this before I can prepare a fix.

[d99kris]

Yep, we can certainly make the timeout configurable.

[d99kris]

The above commit adds support for configuring idle timeout with idle_timeout in ~/.nmail/main.conf.

Will keep this issue open until other fixes have been committed.

[d99kris]

The above commit implementes more robust detection of new emails while in idle.

Will keep this issue open until a few more related fixes have been committed.

[d99kris]

The above commit adds better status reporting during idle.

Will keep this issue open until one more related fix has been committed.

[d99kris]

The above fix detects when device is returning from sleep mode, and if in idle mode, ensures we exit idle mode (and indirectly try fetch messages again before re-entering idle).

This is the last idle-related fix for this Github issue.

[d99kris]

Just like to add on - while the latest nmail idle functionality has been tested OK with several email service providers, there is one with problems: outlook.com

However, it seems to be a server-side problem according to
https://techcommunity.microsoft.com/t5/outlook/outlook-office-365-imap-idle-is-broken/m-p/3616242

Hopefully Microsoft fixes this soon. As a temporary workaround one can now set idle timeout to some lower value, effectively polling the server for new messages.