notification about files not synced locally

[moscicki]

With our 2nd level support we discussed a looming operational problem with files which are not synced locally, for example conflict files or files which have been excluded from sync but a user forgot it. This is a problem because it is a potential file loss in case a user's laptop breaks.

Would it be possible that the user is notified about files not synced, or better could there be a notification the service operators be notified about the users who have not synced files? That would allow service operators to choose to notify a user by email automatically of this fact. Such a notification to service operators could be done in a easily-parsable format which would contain some additional information should be attached such as the reason why the file is not synced, mtime etc so that notification policy could be flexibly adjusted. For implementation, a hidden file uploaded to the server would be enough, as we have foreseen it in this case: #3086

BTW1. I understand that files with forbidden characters in the name should not be a problem anymore since these restrictions would be lifted.

BTW2. Is there a plan to sync conflict files by default in the future?

[guruz]

Delegating this discussion to @MTRichards @dragotin @cmonteroluque @jancborchardt since we have multiple related issues already.. (trying to find them now)

[MTRichards]

My current client just synced a conflict file...I think. I need to run a real test, but I thought it does this now.

[MTRichards]

How would you want a user to be notified? Like a list of files that are not syncing somewhere in the desktop client?

[MTRichards]

How about an option to sync conflict files?

[moscicki]

@MTRichards: is there a reason NOT to sync conflict files?

For the notification: it would be interesting that service managers (operators) can get access to this information somehow. For example, if .owncloudsync.log contained information also about ignored files it would be sufficient to upload it to the server once a day with a special prefix making this file hidden from the user and not syncing down by another client of the same user, and with some client-based uniquifier because we want to keep track of all clients of a given user separately. Based on this, one can imagine to use service notification mechanisms existing in an organization (for example not only via an email but also via other means and according to completely customizable policy defined somewhere centrally). That would be as easy as scanning for such files, parsing them on the server and fireing some action. That would also increase our understanding on what is going in the system in general and anticipate potential problems.

[moscicki]

BTW. For us a plain file on a server would be sufficient, but I can also imagine a server-side owncloud app that could be useful for other customers out-of-a-box.

[danimo]

@moscicki btw: The amount of non-synched files will massively drop with 2.1 because we no longer limit ourselves to the lowest common denominator (i.e. Windows), but the limit is purely what the server is able store.

[dragotin]

I don't think that simply syncing conflict files improves the usability of our product.

But maybe we can build this enhancement: If the client of user A has a conflict, it uploads it to the server. The server hides it from the normal file system, so it is not going to be synced to all other clients. But the other clients get the information that there is a diverged document version of user A available, and can either notify the user, or let it download to a different name etc.

That way it is maybe a bit more useful for users: We do not clutter the file system with unexpected files, but announce the availability of a diverged version. Would be an extension for the versions app?

[jancborchardt]

I don't think that simply syncing conflict files improves the usability of our product.

2 reasons:

• when there is a conflict and you don’t notice, your local stuff actually gets backed up. As you would expect from a product which does sync
• you can resolve the conflict from any other machine, for example from mobile as well or your other computer.

We do have that problem right now – the mobile app often says something like »conflicts found« but there is nothing you can do about since the app doesn’t really tell you which files are the ones in question (cc @davivel @rperezb), and maybe you can’t even view the specific files from the device. Same could be the other way around. I’d just expect for conflict files to be synced.

[dragotin]

@jancborchardt I think you wanted to write "You can not resolve...." (second bullet point) otherwise I fail to understand...

[danimo]

I agree with @dragotin: We need to move the problem of conflict file from a filesystem to a application level problem. If conflict files were synched to the server and provided similarly to versioning, and if there was a client API to retrieve these states, a workflow could be like this:

1. User A and B modify a file
2. User A is first to save
3. User B receives a warning upon saving the file
4. User B can then decide (on whatever device he is currently using), to
   • Restore state from user A
   • Overwrite with his own state
   • Restore (either from then notification or from the context menu in the file browser) the conflict version produced by the machine he was working on, which gets restored to a temporary directory as a read-only file and opens it with the default application. This enforces some conflict resolving action.

[jancborchardt]

@dragotin yeah, I meant to describe the situation as it would be when the conflict files would be synced. So yes – currently you can not resolve conflicts on other devices.

[moscicki]

Let me throw my requirement first and then a bunch of thoughts.

In the first place I would like to avoid data loss. So the dirty contents (local modifications) should make its way to the server one way or another.

In the second instance, you discuss ability to resolve conflict files from any device. It's a functional improvement indeed. How do you resolve conflict files today? On the machine with the local conflict file one renames it to the original name. So why not using this logic for the server-side? Rename a conflict file into the original name (maybe with a help of some gui/app that presents this as a logical operation).

The problem is that if your local conflict file is still open by the application it would be removed by the sync client when the conflict is resolved on the server. I don't know if we really want this or not. Maybe you should revisit your policy of how you create conflict files locally: do not rename dirty file as conflict but rename the one downloaded from the server as conflict (in which case you would not need to sync it as it came from the server and would be available as a version).

So here is the ultimate alternative: implement conflicts as versions on the server (maybe adding some metadata indicating that they are conflicts so they could have a different retention policy on the server).

[phil-davis]

And the design will need to allow some multi-device-unique conflict file naming scheme. If users/devices A, B and C save "x.txt" and B device gets conflict "x_conflict-date-time.txt" then in the general case you could get unlucky and C saves and a conflict is generated "at the same time" and then there is a conflict of the conflict file names on the server!
The alternative mentions of using some server version system means that the web interface could manage that in the files app or other special app for conflict resolution. But the ownCloud client apps would need to also somehow expose the existence of server versions that were not plain files in the local client file system.

[jancborchardt]

@phil-davis in that case, simply a number starting with » (2)« should be appended after the conflict name.

[phil-davis]

@jancborchardt True, at the time the client detects a conflict it is by definition in contact with the server, so it can check on the server for conflicting names and choose a unique name.

[rperezb]

Per Jan's comment

We do have that problem right now – the mobile app often says something like »conflicts found« but there is nothing you can do about since the app doesn’t really tell you which files are the ones in question (cc @davivel @rperezb), and maybe you can’t even view the specific files from the device. Same could be the other way around. I’d just expect for conflict files to be synced.

how the Android app works, in the case that the same file has been modified from 2 clients at the same time:

1.- User1 is using the mobile app, download a file (file1) and then gets offline and modifies it
2.- User2 is using the desktop client modifies file1 and gets synced
3.- user1, connection is back and select to sync the file (or it´s a favorite..whatever) a conflict is detected while uploading/downloading the changes, the app offers between: overwritte / rename (so keeps both) / cancel (conflict persists)

Why it's important from my pov? while versioning it´s a great feature, you need to decide what is the latest version, so that the one that clients will be downloading by default.

On the other hand, about how we notify conflicts, within the next version we will update and improve it, so that conflict files will be set. owncloud/android#1218 (comment)

About iOS, it´s more complex to force a conflict, however, it may be forced, to fix conflicts we use a similar process, conflicts files are listed on the uploads view offering the same 3 options (overwrite, rename or cancel)

[davivel]

We do have that problem right now – the mobile app often says something like »conflicts found« but there is nothing you can do about since the app doesn’t really tell you which files are the ones in question (cc @davivel @rperezb), and maybe you can’t even view the specific files from the device. Same could be the other way around. I’d just expect for conflict files to be synced.

@jancborchardt et al, I'm happy to say that the Android OC app does better than that from the release of folders synchronization (autumn?).

True, the notification "conflicts found" doesn't lead to the file(s) in conflict in any way, and doesn't allow any other smart action.

But when the user enters the app, any folder containing any file in conflict in any depth below will be marked with a conflict icon. Red circle. This: https://github.com/owncloud/android/blob/master/res/drawable/conflict_file_indicator.png

That way the user may easily browse through the folders tree and find the file in conflict, that will be marked with the same icon. With a manual sync on the file, the user may choose to keep both versions, and then may download the server version to see both in the mobile device.

Many improvements may be added to this, of course. Let's do them!

[dragotin]

The conflict file notification is in 2.2.0 now.