recurring tasks with single instances edited/completed are completely ignored

[DJCrashdummy]

i haven't used/tested OpenTasks for a while because of #462... but with its support for completing single instances of recurring tasks (IMHO the essential feature, because advanced editing every once in a while can be done elsewhere) i was willingly to finally switch.

---

but sadly i had to experience a big show stopper bug:
recurring tasks seem to be completely ignored as soon as a start and due date is specified. - and unfortunately this is the case - at least in Thunderbird - as soon as you set a due date because it seems that a start date is mandatory for a recurring task. single instance was marked as completed at an other client. - if the recurring task was synced previously, it stays in OpenTasks how it was and all changes (except a complete deletion) are ignored.

some more details of this idiosyncratic bug:
(i tried my best, but this list is not very accurate any more and is propably more confusing than informative... better go to the comment with the ICS-files and proper description! ⬇️)

• if you don't specify a due date while the initial creation (editing afterwards doesn't do the trick! 🤔), the recurring task is shown.
• the date shown in OpenTasks is the start time of the whole series, no matter if you mark the fist instances as completed.
• if you add a due time afterwards, the recurring task continues to be shown(!) and the date in OpenTasks' list-view is correctly changed to the due time resp. date; but also it stays the due date (of the first instance) of the whole series even if you mark the first instances as completed.
• it gets even weirder if you start editing in OpenTasks: if you mark an instance of a recurring task as completed in OpenTasks, the list-view jumps correctly to the next due date... but i can see in Thunderbird, that a completely new single task was created (the completed one) and the one instance of the series and the series itself stays untouched.
• and the cherry on the cake with editing in OpenTasks: if you mark the first instance of a recurring task as completed in OpenTasks, you can see in Thunderbird, that again a completely new single task was created (the completed one) but the begin of the recurring task was changed to its next due date.
• finally, to make it clear: if you specified a due time on creation marked one instance of a recurring task (with Thunderbird) as completed, editing the task afterwards doesn't change anything (even unchecking all single instances)... the tasks resp. the whole series won't show up in OpenTasks! ☹️
• just to mention it: it makes no difference if the start and due date are set to the exact same time.

so a long story made short: at least in conjunction with syncing (resp. Thunderbird) it seems, the topic recurring tasks got even worse, because now you can't be sure if all tasks are shown. 😞

my setup:

• OpenTasks 1.2.4 (from F-Droid)
• tasks were created & edited with Thunderbird & Lightning (if not stated otherwise)
• the tasks for testing were weekly and monthly tasks created in a shared calendar
• sync is done via Nextcloud & DAVx5 (from F-Droid)

---

perhaps this is partly a duplicate of #896 (if so, i'm sorry), but i'm not sure because its description is not very enlightening and i can't confirm it because my observations just happen for recurring tasks - normal tasks whether they have a start and/or a due time are displayed as they should!

---

EDIT 2020-05-20: added & corrected some details and added further findings about editing in OpenTasks.
EDIT 2020-07-02: corrected/scratched wrong findings & assumptions.

[DJCrashdummy]

PS: to reproduce this bug i started from scratch (even uninstalled OpenTasks & DAVx5) and then the one recurring task which was shown (because of the initial creation without a due date, but added one later) was not shown again. but literally nothing has changed between these two attempts!
...so i'm pretty sure this issue is not on the side of Thunderbird or the ICS-file itself, but rather on the side of it's interpretation resp. its changes.

[dmfs]

Any chance you can provide an ICS file of a task to reproduce this issue?

[DJCrashdummy]

sure, in which state(s) do you want the ICS file(s)? - just an ignored one? the initial one, which is shown? the edited one, which is still shown? or even other ones with completed instance via Thunderbird and/or OpenTasks?

and how do you prefer to get them? email, github or whatever? and the files itself or (temprary) links to some cloud?

[dmfs]

Ideally a copy of every step, attached to this issue. Thank you.

[dmfs]

@DJCrashdummy any success on this yet?

[DJCrashdummy]

sorry @dmfs for replying soooo late, but unfortunately i am pretty busy the last months! ☹️

holy cow! half the OP was wrong (i updated it and the title somehow accordingly)... perhaps of presumptions raised by the mentioned other issues. 😒

---

but i'm pretty sure i finally nailed down the causing of the problems: it is, checking an instance as completed!!! 💡
so i guess something with RECURRENCE-ID, RRULE or simply matching the UIDs is wrong. 😑

this also makes now completely sense:

• a normal recurring task even with start and due time is shown perfectly fine in OpenTasks (as long as nothing is changed at a single instance, not even marked once as completed). [1]
• also editing the whole recurring master task is not a problem at all. [2]
• but as soon as a second VTODO with the same UID is present in the ICS-file (means you edited a single instance for whatever reason - see RECURRENCE-ID) [3], the single instances and any change to the master task are ignored by OpenTasks.
• even if you uncheck all single instances (and it appers like the original task), it is ignored because the single instances stay there in the ICS-file. [4]
• and now, if you just deactivate syncing this calendar in DAVx5 and activate it again, this task (which seemed perfectly fine and identical at all clients) is gone and will never show up in OpenTasks again. - although the file is still exactly the same as [4] (i even compared the checksums).

the part with editing via OpenTasks (point 4 & 5 in the OP ⬆️) was pretty accurate:

• although the 4th point of the OP is obsolete resp. self-explaining with the findings in this post... (because you can check the tasks only chronologically in OpenTasks, thus you must have else where edited a single instance if it is not the first one.)
• and if you mark the first instance of a recurring task [5] as completed, the master task is modified and "moved" (changed DTSTART & DUE among other things) to the next occurrence... and a completely new and separate completed task is created. [6] 😕
  this "workaround" kind of works as long as you solely use OpenTasks resp. a local calendar, but in conjunction with syncing, this sooner or later bites you in the ass.
  i'm not an expert regarding ICS-files, but by simply comparing the files it seems to me, that RECURRENCE-ID is missing at the marked resp. changed instance and a different UID is created... that's why a completely new task appears.

hopefully all this helps you with getting this issue fixed, and sorry for the false report in first place! 🤦
if you need something else, please don't hesitate... as you see, i do respond, it's just a matter of time (at the moment).

---

note: i had to append .txt to the file name, that the stupid github takes it! 😡
[1] task-test1 - initial recurring Thunderbird task.ics.txt
[2] task-test2 - edited recurring Thunderbird task.ics.txt
[3] task-test3 - edited recurring Thunderbird task (single edited instance: completed).ics.txt
[4] task-test4 - edited recurring Thunderbird task (single edited instance: unchecked).ics.txt
[5] task-test5 - recurring task.ics.txt
[6] task-test6 - recurring task (first instance completed by OpenTasks).ics.txt

btw: the setup didn't change (except potential updates), but OpenTasks didn't get one in the meantime and also at all other software i solely use releases in this case, no testing versions, not even a beta or a RC.

[dmfs]

Thanks for the details. Regarding the issue with the new task when completing an instance in OpenTasks ([5] and [6]), that's intended behavior. When you complete or cancel the first instance of a recurring task, the first instance is detached from the series and the series start is moved forward. The primary reason is to support Apple clients. They don't support overrides like Thunderbird and other clients do.
There will be changes/additions to this in future releases. Once relations are fully in place I'm going to link all the instances together to they still can be identified as a series. Detaching completed instances has the advantage that you save traffic by not having to sync all the past completed instances. Long running series can pile up a large number of completed tasks which is essentially just dead weight considering that they are unlikely to change after they were completed.
I may however add an option to change this "detaching" behavior on a per account basis in future releases.

Regarding the issue with Thunderbird tasks it would be interesting to see the database data. Do you happen to have root access and access to the data folder in /data/data/org.dmfs.tasks/databases? I can't reproduce the issue with my own sync app. Maybe there is some sort of misunderstanding between OpenTasks and DAVx5.

[dmfs]

I've noticed a different issue when I sync your task with my server. In my case the override is ignored. That means, when I complete the first 3 instances, the fourth instance is April 1st even though it should be May 1st because April 1st is already completed. I'll look into that.
Edit:
looks like this is currently expected: https://github.com/dmfs/opentasks/blob/master/opentasks-provider/src/main/java/org/dmfs/provider/tasks/processors/tasks/Instantiating.java#L205 (tracked via #952)

[cakexensen]

Checking in as I'm experiencing this exact problem after having created recurring tasks. I'm using OpenTasks 1.2.4 and DAVx5 3.3.1-ose. Is there any progress on a fix, or a work around?

[dmfs]

existing overrides should be handled correctly since 1.3.0. Please let me know if this is still an issue. I'm closing this for now.