Shouldn't set flag T after moving to trash

[jgoerzen]

Expected or desired behavior

When executing the mark for trash, mu4e should move it to the Trash folder but should not flag it T.

Actual behavior

mu4e is marking the message Trashed even after moved to the Trash folder. This is not commonplace behavior and violates expectations.

Versions of mu, mu4e/emacs, operating system etc.

mu4e 0.9.18, emacs 25

Any other detail

I am the original author of OfflineIMAP and paid some attention to this.

There are three ways that mail user agents (MUAs) can delete messages:

1. Immediately and irretrievably delete it.
2. Set some flag for later expunging
3. Move it to a different folder

These days, most users are set up to use method 3. But it is actually implemented on top of method 2 in IMAP in many cases.

Option 1 is to just zap the message (rm from the Maildir, etc.) Boom, gone.

I used to use IMP (webmail), which typically worked with option 2. The messages would remain in your Inbox (or wherever), but with a special flag that they were deleted. They were normally hidden from view, but could be shown (IMP happened to do so with a strikethrough effect). Then could eventually be purged/expunged. The traditional advantage of this over option 3 was that it was clear what folder a message was deleted from. There is an IMAP operation (EXPUNGE) that permantly deletes all messages with the \Deleted flag. So to delete a message in IMAP, you set that flag and then EXPUNGE, which acts on all messages with such a mark in a folder.

Behind the scenes, when a person moves a message from their Inbox to Trash, the usual order of things is:

1. COPY it to the Trash folder
2. STORE the \Deleted flag to it in the Inbox
3. Either immediately or later, EXPUNGE the Inbox to get rid of the old copy there.

The Trash folder is cleaned up later. Perhaps automatically, perhaps manually. An "empty trash" operation would STORE the \Deleted flag to all messages, then EXPUNGE. A user might want to delete just one message from Trash.; they would STORE \Deleted to that message, then EXPUNGE.

So the standard behavior -- and this is what Thunderbird, Apple Mail, K-9 Mail on Android, and all other mail clients I've ever used do -- is to NOT add the T or \Deleted flag in Trash. If that flag is there, then the message is almost certain to be permanently removed by some other process, whether OfflineIMAP or another MUA, even when the user doesn't ask for it.

This was discussed at a Google Groups conversation [1] below, and an OfflineIMAP conversation [2], neither of which really properly dove into this.

The fix is trivial - remove the "+T" in the mu4e-mark.el. I would greatly appreciate this, as it is causing compatibility issues with mobile mail clients for me.

[1] https://groups.google.com/forum/#!topic/mu-discuss/m4ORymDlf0E
[2] OfflineIMAP/offlineimap#142

[jgoerzen]

I should also explain djb's comment in the Maildir spec. In some earlier MUAs, the "Trash" was a virtual folder that was simply built by looking for "deleted" messages in regular folders. (Or should I say "folder"; in the days of /var/mail and POP3, multiple folders weren't the thing they are now.)

In earlier days, there wasn't even that virtual trash folder concept. In the mbox world, for instance, Pine would set Status: or X-Status: to indicate if a message was deleted. [1] The Pine screen would show all messages in the folder, with deleted ones shown as "DEL", allowing the user to undelete. [2]

This is how we got that concept in Maildir and IMAP. When djb writes "the user has moved this message to the trash; the trash will be emptied by a later user action", this is applying to the source message. Whether the mail program uses a virtual trash can or what, they would mark it Trashed in the source folder.

A Maildir-native MUA could, of course, just mv it to the destination without any flag munging. In that case too, it is expected that a later user action would empty the folder (in that case, the location of the folder rather than the presence of a flag is used to identify messages to reap).

In my experience, both Dovecot and Courier have a 1-to-1 mapping between \Deleted in IMAP and T in the Maildir. [4] confirms.

There are some threads discussing this: [3] for instance

Finally, even today some MUAs (eg, mutt) are not Trash-folder-aware and continue to use the T flag for messages that are deleted. (in Mutt, D sets a flag and then $ will do the expunge, whether for Maildir or IMAP)

Hope this helps

[1] https://www.washington.edu/pine/tech-notes/low-level.html
[2] https://www.ecs.csus.edu/docs/pine.html
[3] https://www.dovecot.org/list/dovecot/2010-June/049648.html
[4] https://www.dovecot.org/list/dovecot/2010-June/049673.html

[jgoerzen]

An additional comment: when looking at search results and showing matches from a Trash folder, the UI should probably mark it strikethrough regardless of presence of T flag

[djcb]

I always saw it as both mu and offlineimap considering themselves as the owner of the Maildir; that mostly works okay, and over the years I've added a few tricks for various corner-cases. mbsync has a few others. mu doesn't make any assumptions on how we retrieve/sync the mail.

As for the trashed-flag... -- I think the current behavior is okay; people who don't want the "trashed"-flag can simply instead move messages to a folder of their choosing; however, perhaps we should document it a bit better.

[luisgerhorst]

For those seeking the behaviour implemented by most GUI mail clients (i.e. only move the message to the trash from where the server auto-deletes it automatically after some delay) - The following code overwrites d in mu4e-headers-mode to only move the message to the trash folder and do not mark it as deleted:

Update 22-03-14: Flinner points out that the following is outdated.

(setf (alist-get 'trash mu4e-marks)
      (list :char '("d" . "▼")
            :prompt "dtrash"
            :dyn-target (lambda (target msg)
                          (mu4e-get-trash-folder msg))
            :action (lambda (docid msg target)
                      ;; Here's the main difference to the regular trash mark,
                      ;; no +T before -N so the message is not marked as
                      ;; IMAP-deleted:
                      (mu4e~proc-move docid (mu4e~mark-check-target target) "-N"))))

If you want to permanently delete a message, you can still use D.

[podiki]

Thanks @luisgerhorst and @jgoerzen! I seem to finally understand and have deletion work in Gmail as expected: I can 'delete' (move to trash) a message from inbox and it will end up in trash with no extra copy in All Mail (working as it does on the web interface). Just moving the message accomplishes the same thing, but I was very confused over why the delete action was giving me odd behavior.

(Note that in Gmail I have Auto-Expunge off, and "Move message to Trash" as what should happen when a message is deleted and expunged from all IMAP folders; as well as having mbsync sync trash first if that makes any difference).

Anyway, just wanted to say thanks and make a note here for anyone else that would go crazy over not understanding deletion with accounts like Gmail.

[pkazmier]

@luisgerhorst and @jgoerzen Thank you so much for explaining this. As a new user to mu4e, I've been banging my head for the past day trying to figure out why messages I delete in mu4e are not showing up in my O365 trash folder. The reverse worked fine—messages deleted in Outlook would show up in my local trash Maildir. After using the snippet from @luisgerhorst, I can now "delete" messages in mu4e and they will be synced to my O365 instead of just being deleted. Thanks again!

I'm using mu4e <-> mbsync <-> davmail <-> O365 pipeline as my O365 does not support IMAP (enterprise policy).

[Flinner]

for me, @luisgerhorst 's solutions erros with

the function ‘mu4e~proc-move’ is not known to be defined.

so this works instead

(setf (alist-get 'trash mu4e-marks)
      (list :char '("d" . "▼")
            :prompt "dtrash"
            :dyn-target (lambda (target msg)
                          (mu4e-get-trash-folder msg))
            :action (lambda (docid msg target)
                      ;; Here's the main difference to the regular trash mark,
                      ;; no +T before -N so the message is not marked as
                      ;; IMAP-deleted:
                      (mu4e--server-move docid (mu4e--mark-check-target target) "-N"))))

[fabian-thomas]

If you want the mail to be marked as read with @Flinner modification use:
"+S-u-N" for the IMAP flags

[hokomo]

I'm using mu/mu4e with mbsync and I just ran into the same "issue", so here are my thoughts and a few comments related to mbsync (it's hard to add much to OP's great explanation though).

I agree with OP that setting the Maildir T flag, which mbsync ends up propagating as the IMAP \Deleted flag, is most likely not the behavior one would expect coming from mail clients such as Thunderbird, K-9 Mail, etc. The \Deleted flag makes the messages immediately eligible for permanent removal via IMAP's EXPUNGE command, whether initiated "locally" (e.g. through mbsync) or "remotely" (by the server itself). This doesn't match the commonly expected behavior of only moving the message to the trash folder and letting the user decide when to actually flag it as \Deleted and follow up with an EXPUNGE.

In mbsync, expunging is controlled through the Expunge option. When turned on (by setting it to Both, e.g.), mbsync will automatically perform an EXPUNGE whenever \Deleted messages are propagated, which coupled with mu4e's default behavior means that marking messages as "trashed" will actually permanently remove them, which is no different from marking them as "deleted".

I believe mbsync's Trash option could be used to define a folder where \Deleted messages would be copied before expunging. This would essentially give you the common expected behavior but in a sort of roundabout way -- rather than just directly moving the messages into the trash folder without flagging them, this would first flag them as \Deleted, then make a copy for the trash folder, and then EXPUNGE all of the \Deleted messages.

However, given the various caveats about trashing in mbsync's manpage, I decided not to use the Trash option but to just modify mu4e's default behavior like everyone above, in order to prevent it from setting the T flag in the first place. Here's an even more concise version:

(setf (plist-get (alist-get 'trash mu4e-marks) :action)
      (lambda (docid msg target)
        (mu4e--server-move docid (mu4e--mark-check-target target) "-N"))) ; Instead of "+T-N"

@djcb Maybe it would be nice to include a rundown of the above -- trashing by (just) moving vs. trashing by moving and flagging, and the connection between T, \Deleted and EXPUNGE (assuming the most common Maildir <-> IMAP syncing software such as mbsync or offlineimap) -- along with an example of customizing mu4e so that marking something for trash performs the common "trashing-by-moving".

Also, the definition of mu4e-marks might need some adjustments in order to allow for "clean" customization. Currently, the initial value of the variable is a quoted list, so mutating it like above is probably not the best thing to do. However, assigning the variable to a fresh list is technically not good either as it's defined using defconst. Elisp doesn't really enforce the constness though, so I don't know how pedantic you want to be here.

[k4r4b3y]

I just got bitten by this behavior of mu4e. It cost me hours of head scratching confusion, and only as I checked the locally trashed emails' headers did I see the trashed tag, which gave me the initial hint about the possible cause of my issue (deleting emails doesn't move them to trash and cause them to completely vanish from my mailbox). Thanks to people above suggesting the removal of the "+T" flag and posting elisp code snippets.

[phil-s]

It sounds like this behaviour could be a configuration option, as "the right thing to do" can vary based on which other mail clients are used?

[djcb]

@phil-s: the right thing to do would be to follow the Maildir standard for what "T" means; the problem is with some of those mail-syncing programs. But TBH, the details of those programs are out-of-scope for mu4e which really only cares about your local maildirs.

Anyway, I suspect they're not going to change, I have added a variable mu4e-trash-without-flag which , if non-nil, makes mu4e not add that flag.