Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Version 2.5: Client does not sync/The server file discovery reply is missing data #819

Closed
klada opened this issue Nov 13, 2018 · 90 comments
Closed

Comments

@klada
Copy link

klada commented Nov 13, 2018

I have installed version 2.5.0 of the Desktop Client (Windows, completely fresh install) and the client does not sync at all. I am getting the following sync error right after going through the configuration assistant:

A HTTP transmission error happened. The server file discovery reply is missing data.

The requests on the server look fine, I am getting many PROPFIND requests there, all answered with HTTP 207.

⚠️ Version 2.3.3 of the client is working fine on the very same machine with the very same user. → regression

Expected behaviour

The sync process should start after configuration.

Actual behaviour

The sync fails immediately after configuration (fresh config, fresh installation, empty sync dir).

Steps to reproduce

  1. Install and configure client (clean/fresh install)
  2. Sync fails immediately

Client configuration

Client version: 2.5.0

Operating system: Windows 10 (x64)

OS language: English/German

Server configuration

Operating system: CentOS 7.5

Web server: Apache 2.4.6-80

Database: Postgres

PHP version:7.2.12

Nextcloud version: 14.0.3

Storage backend (external storage): -

Logs

  1. Client logfile: Output of nextcloud --logwindow or nextcloud --logfile log.txt
    (On Windows using cmd.exe, you might need to first cd into the Nextcloud directory)
    (See also https://docs.nextcloud.com/desktop/2.3/troubleshooting.html#log-files)

https://gist.github.com/klada/fd88210850c4f4ec783c9e837a622acb

  1. Web server error log:

(there are no errors)

192.168.12.16 - otto [13/Nov/2018:11:06:26 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 399 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:06:38 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 400 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:06:56 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 399 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:07:10 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 400 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:07:26 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 399 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:07:42 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 400 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:07:56 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 399 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:08:14 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 400 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:08:26 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 399 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:08:46 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 400 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:08:56 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 399 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:08:58 +0100] "GET /ocs/v2.php/apps/notifications/api/v2/notifications?format=json HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 0
192.168.12.16 - otto [13/Nov/2018:11:09:18 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 400 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:09:26 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 399 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:09:50 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 400 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:09:56 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 399 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
  1. Server logfile: nextcloud log (data/nextcloud.log):

Not applicable.

@klada
Copy link
Author

klada commented Nov 13, 2018

This seems to be the relevant part of the client log:

[OCC::LsColJob::finished 	LSCOL of QUrl("https://mycloud.local/remote.php/dav/files/otto/") FINISHED WITH STATUS "OK"
[OCC::DiscoverySingleDirectoryJob::directoryListingIteratedSlot 	Missing properties: "test" 2 0 1542103459 "DNVS" "" "-0000001ocrvl2k5jw23"
[csync_ftw 	opendir failed for  - errno 10011
[OCC::SyncEngine::handleSyncError 	ERROR during  csync_update :  "Es hat sich ein HTTP-Übertragungsfehler ereignet. Bei der Server-Dateierkennungsantwort fehlen Daten."

What is interesting is the Missing properties part. Apparently an ETAG is missing for the file or directory "test". The check which is failing has been introduced by 687b6f5, which explains why the client version 2.3.3 is working fine.

The strange part is that the user does not have a directory or file with the name "test" in his user folder. I have already tried running occ files:scan for the user, but this did not change anything.

BTW: In Client version 2.3.3.1 I am getting this warning:

11-13 13:49:20:680 8260 OCC::DiscoverySingleDirectoryJob::directoryListingIteratedSlot: WARNING: etag of test is   This must not happen.

In version 2.3.3 this was a warning and in version 2.5.0 this is now an error which stops the entire sync.

@klada klada changed the title Version 2.5: The server file discovery reply is missing data. [regression] Version 2.5: Client does not sync/The server file discovery reply is missing data. [regression] Nov 15, 2018
@merlinschumacher
Copy link

Same issue for me on Arch Linux

@rullzer
Copy link
Member

rullzer commented Nov 16, 2018

This seems more like a server issue. But even then it is weird.

There is of course a good reason that the client refuses to sync if it gets back things it can't make sense of from the server. Because it can't make sense of what is happening.

I assume it only happens for both of you on some accounts and not on all?

@klada
Copy link
Author

klada commented Nov 16, 2018

Yes, it's probably a server issue. It still triggers a regression in the client:

  • Version 2.3.3 is working fine
  • Version 2.5.0 is not working at all → regression

And you are right, I can only reproduce the issue with one user account. Other user accounts are working fine with version 2.5.0. The strange part is that I have tried a occ files:scan, but it does not solve the issue.

@merlinschumacher Do you have a line like this in your client log?

[OCC::DiscoverySingleDirectoryJob::directoryListingIteratedSlot 	Missing properties: "test" 2 0 1542103459 "DNVS" "" "-0000001ocrvl2k5jw23"

If yes, what does the "Missing properties" message say? Does it mention a filename you recognize?

@nliautaud
Copy link

nliautaud commented Nov 16, 2018

Same here, caused by a bug in Google Drive external storage extension. Removing or updating the extension eliminated the client error.

OCC::DiscoverySingleDirectoryJob::directoryListingIteratedSlot 	Missing properties: "GoogleDrive"

Note that the 2.3.3 client version was working even with this server extension issue.

@merlinschumacher
Copy link

The Gdrive-Plugin is indeed the issue. After installing a newer version from the app-store sync works fine.

@tofuSCHNITZEL
Copy link

tofuSCHNITZEL commented Nov 21, 2018

I have a similar issue. But I don't have the Gdrive Plugin installed and all other Plugins are up to date.

[OCC::DiscoverySingleDirectoryJob::directoryListingIteratedSlot Missing properties: "Fotos Tauschrausch" 2 0 1542792323 "DNVS" "" "-0000001oc90bef3ea20"
[csync_ftw opendir failed for - errno 10011

the above mentioned folder is not visible in the webclient - but it appears in the "select files/folders to sync dialog" and even if it is not checked the above error appears.

in the oc_mounts and oc_shares table an entry exists for this folder. I removed the entry and now the client syncs again.

@messimuc
Copy link

I had a similar issue. As I am just a user I cannot post logs. Some other user shared a folder with me, at some point this other user got deleted but her shared folder stayed in my files view. Client 2.3.3 had no issues, Client 2.5 stopped working: "A HTTP transmission error happened". Deleting the shared folder solved the issue for me.

@r2evans
Copy link
Contributor

r2evans commented Dec 11, 2018

Same symptom, for me it was an external share that could not connect (the remote end is shut down for a bit). I don't know that it's always the problem (nor if SFTP is the only culprit), but when I disabled the external share the error went away and normal syncing resumed.

@klada
Copy link
Author

klada commented Dec 18, 2018

In my case it's also a stale share causing the issue. Pretty much like @tofuSCHNITZEL described it:

  • In my case is a share named "test" (see log in post 2)
  • I see this share in the WebUI under "Shares", but not in "All Files".
  • I see this folder in the sync client properties
  • When I exclude this folder from syncing the sync client is working

@seanbaker74
Copy link

I have the same issue. I had set up a CIFS/SMB external storage that was "broken" due to a configuration issue on my part. Until I fixed the external storage, my NextCloud client sync was failing with the "server file discovery reply is missing data" error message. Seems pretty easy to reproduce.

@mlanner
Copy link

mlanner commented Jan 2, 2019

It seems that the "external storages" server component that was changed in the 14.x series caused this bug, as I reported here:

nextcloud/server#11264

@r2evans
Copy link
Contributor

r2evans commented Jan 2, 2019

@mlanner, we might be talking about something different. My server is still on 13 (not stated earlier, but it now appears to be more of a server-bug than a client-bug).

@nathanael-h
Copy link

I had the same problem on ArchLinux, I tried to delete my old non working external storages (SFTP I did not use anymore) and it worked.
Server is 14 on Debian

@mioiox
Copy link

mioiox commented Jan 4, 2019

Server NC13, client NC 2.51. I am using a bunch of SMB folders (the SMB server is Windows Server 2019, but is shouldn't matter). All folders are accessible except one (that is physically missing on the server at the moment). However, when I start syncing any of the fine folders, I get the same error. I managed to clear it by removing the bad folder from the NC configuration and the sync started without error.

However, I would expect to be able to sync the folders that are OK, as issues with one folder should not impact the sync capability of any other folder.

@ghost
Copy link

ghost commented Jan 4, 2019

Same here I guess.

#908

@bjoernv
Copy link

bjoernv commented Jan 12, 2019

Now I see this bug too:

nextcloud 14.04 (with PHP 7.2 on Ubuntu 14.04)
nextcloud-client 2.5.1git (openSUSE Tumbleweed)

I do not have external storage except another Nextcloud 15.0.0 server which is connected with this Nextcloud server.

@bjoernv
Copy link

bjoernv commented Jan 13, 2019

I was able to analyze my "The server file discovery reply is missing data." error message which stopped client synchronization because the error is thrown in the discovery phase before the synchronization phase.

I had two problematic files (one file, one directory). Unfortunately I have only partial logs. I used the --logfile nc.log --logdebug options.

I changed the file and directory names for privacy reasons. The server send this message for the file: Please note the trailing slash for the file "20150727_myexcelfile KVP20150820.pptx/". This is a normal Powerpoint file and should not have a trailing slash.

  <d:response>
    <d:href>/nextcloud/remote.php/dav/files/myusername/20150727_myexcelfile KVP20150820.pptx/</d:href>
    <d:propstat>
      <d:prop>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <oc:size>0</oc:size>
        <oc:permissions>SGDNV</oc:permissions>
        <oc:fileid>-1</oc:fileid>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>

The NC client complained:

[OCC::DiscoverySingleDirectoryJob::directoryListingIteratedSlot 	Missing properties: "20150727_myexcelfile KVP20150820.pptx" 2 0 1547298781 "DNVS" "" "-0000001oc2khb5drj3s"
[OCC::DiscoverySingleDirectoryJob::directoryListingIteratedSlot 	Missing properties: "200 MyDirectory" 2 0 1547298781 "DNVS" "" "-0000001oc2khb5drj3s"
[csync_ftw 	opendir failed for  - errno 10011
[OCC::SyncEngine::handleSyncError 	ERROR during  csync_update :  "Es hat sich ein HTTP-Übertragungsfehler ereignet. Bei der Server-Dateierkennungsantwort fehlen Daten."
[OCC::ActivityWidget::addError 	Item  "/var/home/myusername/nextcloud"  retrieved resulted in  "Es hat sich ein HTTP-Übertragungsfehler ereignet. Bei der Server-Dateierkennungsantwort fehlen Daten."

The directory "200 MyDirectory" exists in the filesystem, but only in files_versions and files_trashbin. The file 20150727_myexcelfile KVP20150820.pptx exists as a normal file.

I checked the file and directory in the NC database and removed the entries in oc_filecache. After that I started the command "./occ files:scan --all".

Now NC client 2.5.1 synchronizes again. I have to wait for the next problem to analyze the problem further. I also did not checked, if NC client 2.3.3 would also have stopped on the problematic files. I would guess, that NC client 2.3.3 was more fault tolerant here so that this problem comes up with NC 2.5.1. But also with NC client 2.3.3 I had problems with files, which were in oc_filescache but not in the filesystem.

@harcesz
Copy link

harcesz commented Jan 22, 2019

same problem, tested o 2-3 stations, good thing, we havent upgraded all our clients yet...

a simple workaround for ubuntu/linux is to use the 3.3 appimage; https://download.nextcloud.com/desktop/releases/Linux/Nextcloud-2.3.3-x86_64.AppImage

@rrrnld
Copy link

rrrnld commented Jan 22, 2019

My problem was a folder shared with me by a person that no longer existed. I removed the folder and sync is working again.

@aaronSkar
Copy link

aaronSkar commented Mar 7, 2019

Had a similar issue with a file that showed up with as examplefile.ext/ rather than examplefile.ext it also was mounted in incorrect spots. I deleted all entries for the file in oc_mounts and oc_filecache and sync works again.

Note this issue came about after a migration from a snap deployment to a docker deployment in my case.

@gerroon
Copy link

gerroon commented Mar 7, 2019

I had the same issue, which was caused by a missing path to a share. However it would have been nice if it throws the error and syncs the initial folders

@thoro
Copy link

thoro commented Mar 13, 2019

Agree, the error message could at least contain the path to the folder which is the issue, for non-IT users absolutely impossible to figure out ...

@pachevalier
Copy link

Same error on Ubuntu Mate. What is the trick ?

@thoro
Copy link

thoro commented Mar 15, 2019

Start the client with --logwindow then check what folder is there, and then try to figure out where the folder comes from.

For me it was a unshared / shared folder, only visible in the "Shared" section of the website.

@wouterVE
Copy link

wouterVE commented Mar 4, 2020

@misch7
I have the same problem as @brunt82 :

The admin user shared a (external) folder called Algemeen Sales with the users belonging to the dept of sales. I later renamed the external folder to Algemeen SALES. Now I see in the client the two folders. The old one (Algemeen Sales) with a size of 0 bytes. When I try to expand the folder, the desktop client crashes.
I see the same error as the OP (A HTTP transmission error happened. The server file discovery reply is missing data) and the synchronisation stops.
I tried to update the client to 2.6.4, but it makes no difference. I downgraded tot 2.3.3 and now I still see the ghost folder Algemeen sales, but the client does sync again and does not crash when I expand the folder. It shows an error:
an error occurred while loading the list of folders from the server. Error transfering https://cloud.example.com/remote.php/dav/files/user/Algemeen Sales/ - server replied: fo..

Here is the relevant log entry for client 2.6.4

[OCC::DiscoverySingleDirectoryJob::directoryListingIteratedSlot 	Missing path to a share : "Algemeen Sales" "Algemeen Sales" 2 0 1583312380 "DNVS" "" "-0000001oct3tct3f1ya"
[csync_ftw 	opendir failed for  - errno 10011
[OCC::SyncEngine::handleSyncError 	ERROR during  csync_update :  "Une erreur de transmission HTTP s'est produite. Donnees manquantes dans la reponse a decouverte du fichier sur le serveur "
[OCC::ActivityWidget::addError 	Item  "Nextcloud"  retrieved resulted in  "Une erreur de transmission HTTP s'est produite. Donnees manquantes dans la reponse a decouverte du fichier sur le serveur "
[OCC::ActivityListModel::addErrorToActivityList 	Error successfully added to the notification list:  "Une erreur de transmission HTTP s'est produite. Donn饳 manquantes dans la reponse a decouverte du fichier sur le serveur "

In client version 2.3.3 it is only mentioned in one line:

``03-04 10:03:52:094 0x6d1bb04 csync_walker: directory: Algemeen Sales [file_id=-0000001oct3tct3f1ya]`

I didn't test beta version 2.7 but as I understand the fix is backported to 2.6.x so it would not make an difference.

Thus, as far as I can tell, the http transmission error is not fixed yet. Hopefully you can do something with the listed logs.
(fyi: I installed the different clients on the same pc, but wipped the client config in %appdata%)

@misch7
Copy link
Member

misch7 commented Mar 5, 2020

@wouterVE
Thanks for the detailed report! 👍

This is really helpful for further investigation. I think I've to find a way to trigger the same error for debugging.
Perhaps @brunt82's hint with the deleted user may help me, I remember to have read this as a cause for the HTTP Transmission Error somewhere too.

@wouterVE
Copy link

wouterVE commented Mar 5, 2020

@misch7
You are welcome! If in need for further info please feel free to ask.
FYI: my Nextcloud is version 17.0.3

@Derkades
Copy link

Sync Client 2.6.4 / Nextcloud 18.0.3

Happened to me when one of my external storages was unavailable because of missing smb client. After installing the package it immediately started syncing again.

@claell claell changed the title Version 2.5: Client does not sync/The server file discovery reply is missing data. [regression] Version 2.5: Client does not sync/The server file discovery reply is missing data Apr 4, 2020
@Linutux
Copy link

Linutux commented Apr 12, 2020

I have had the same error. I had an old mount point pointing to a Openstack storage, which did not work. It showed as a red folder in the web file interface and could be deleted at the moint point config of my user. After removing it, the sync client synced again.

@Fuseteam
Copy link

i can confirm removing an unreachable external storage resolves this issue in my case

@InfamousUser
Copy link

Great! Please fix the message to say so in case of external storage issue.

@leotulipan
Copy link

leotulipan commented Nov 15, 2020

Just wanted to leave my comments here. Had this exact problem with client 3.0.3 (Win10) on a Nextcloud installation with 20.0.1.1

With the tips here I started the client on the command line with

"C:\Program Files\Nextcloud\nextcloud.exe" --logwindow --logdebug

which brought me to these two lines:

2020-11-15 09:33:38:311 [ warning nextcloud.sync.discovery ]: Missing properties: "plenty_ftp" 2 0 1605429216 "M" "" "-0000001ocfvc8lpdbsi"
2020-11-15 09:33:38:312 [ debug nextcloud.sync.discovery ] [ OCC::DiscoveryMainThread::singleDirectoryJobFinishedWithErrorSlot ]: 10011 "In der Antwort der Server-Dateierkennung fehlen Daten."

That ftp/sftp external storage had problems with the username/password. What is frustrating, I actually de-selected this in the folder-synchronisation selection but it still stops ANY sync from happening.

A better fall-back would be to skip (external) folders with missing properties so the sync can still continue for all the rest.
Also it would be really helpful to actually output the warning in the standard GUI and not the message following it about missing data, as that is really not useful to find the error

@much-doge
Copy link

Just wanted to leave my comments here. Had this exact problem with client 3.0.3 (Win10) on a Nextcloud installation with 20.0.1.1

With the tips here I started the client on the command line with

"C:\Program Files\Nextcloud\nextcloud.exe" --logwindow --logdebug

which brought me to these two lines:

2020-11-15 09:33:38:311 [ warning nextcloud.sync.discovery ]: Missing properties: "plenty_ftp" 2 0 1605429216 "M" "" "-0000001ocfvc8lpdbsi"
2020-11-15 09:33:38:312 [ debug nextcloud.sync.discovery ] [ OCC::DiscoveryMainThread::singleDirectoryJobFinishedWithErrorSlot ]: 10011 "In der Antwort der Server-Dateierkennung fehlen Daten."

That ftp/sftp external storage had problems with the username/password. What is frustrating, I actually de-selected this in the folder-synchronisation selection but it still stops ANY sync from happening.

A better fall-back would be to skip (external) folders with missing properties so the sync can still continue for all the rest.
Also it would be really helpful to actually output the warning in the standard GUI and not the message following it about missing data, as that is really not useful to find the error

got the same issue, i had to delete the sync connection in the admin panel(external storage section) for the error to go away. the error message should be clearer

@github-actions
Copy link

github-actions bot commented May 6, 2021

This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you!

@github-actions github-actions bot added the stale label May 6, 2021
@InfamousUser
Copy link

Stop botting around.

@github-actions github-actions bot removed the stale label May 8, 2021
@github-actions
Copy link

github-actions bot commented Jun 5, 2021

This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you!

@github-actions github-actions bot added the stale label Jun 5, 2021
@Derkades
Copy link

Derkades commented Jun 5, 2021

do not close

@github-actions github-actions bot removed the stale label Jun 5, 2021
@github-actions
Copy link

github-actions bot commented Jul 3, 2021

This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you!

@github-actions github-actions bot added the stale label Jul 3, 2021
@github-actions
Copy link

This bug report is getting automatically closed due to no answer since the issue has been staled. Thank you!

@Fuseteam
Copy link

Welp

@much-doge
Copy link

Don't close

@Fuseteam
Copy link

Too late :x

@FlexW
Copy link

FlexW commented Jul 19, 2021

@Fuseteam @much-doge Does the issue still exist? With which version are you experiencing errors? Does it happen with the latest release 3.2.4 too?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests