Moving from one shared folder to a different shared folder fails (database gets out of sync) #4169

fgrsnau · 2017-03-30T21:20:40Z

Steps to reproduce

create 3 different users (a, b, c)
let user "a" share a directory "da" with "c"
let user "b" share a directory "db" with "c"
login as "c" and move a subdirectory from "da" to "db"

Expected behaviour

The directory should visible in db.

Actual behaviour

The move fails with an generic error. The underlying files are moved to directory "db". The database query fails and the database is not updated. All files are "visible" in "da" but are inaccessible. An occ files:scan --all fixes this.

Server configuration

Operating system: Debian Stable

Web server: ngnix

Database: Postgres 9.4

PHP version: PHP 5.6.30-0+deb8u1

Nextcloud version: 11.0.2 (stable)

Updated from an older Nextcloud/ownCloud or fresh install: Any (tested 3 different servers, one of them was freshly installed)

Where did you install Nextcloud from: Website

List of activated apps:
Only stock NextCloud.

Are you using encryption: no

Are you using an external user-backend, if yes which one: LDAP, but reproducible without any external user-backend

NextCloud Logs

{"reqId":"NE81Fi3k0jojqFyF0HT3","remoteAddr":"xxx","app":"webdav","message":"Exception: {\"Message\":\"An exception occurred while executing 'SELECT \\\"path\\\", \\\"fileid\\\" FROM \\\"oc_filecache\\\" WHERE \\\"storage\\\" = ? AND \\\"path\\\" LIKE ?' with params [false, \\\"Photos\\\\\\\/%\\\"]:\\n\\nSQLSTATE[22P02]: Invalid text representation: 7 ERROR:  invalid input syntax for integer: \\\"\\\"\",\"Exception\":\"Doctrine\\\\DBAL\\\\Exception\\\\DriverException\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/DBALException.php(128): Doctrine\\\\DBAL\\\\Driver\\\\AbstractPostgreSQLDriver->convertException('An exception oc...', Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException))\\n#1 \\\/var\\\/www\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Connection.php(836): Doctrine\\\\DBAL\\\\DBALException::driverExceptionDuringQuery(Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOPgSql\\\\Driver), Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException), 'SELECT \\\"path\\\", ...', Array)\\n#2 \\\/var\\\/www\\\/lib\\\/private\\\/DB\\\/Connection.php(194): Doctrine\\\\DBAL\\\\Connection->executeQuery('SELECT \\\"path\\\", ...', Array, Array, NULL)\\n#3 \\\/var\\\/www\\\/lib\\\/private\\\/Files\\\/Cache\\\/Cache.php(526): OC\\\\DB\\\\Connection->executeQuery('SELECT `path`, ...', Array)\\n#4 \\\/var\\\/www\\\/lib\\\/private\\\/Files\\\/Cache\\\/Wrapper\\\/CacheJail.php(313): OC\\\\Files\\\\Cache\\\\Cache->moveFromCache(Object(OCA\\\\Files_Sharing\\\\Cache), 'Photos', 'files\\\/Test2\\\/Pho...')\\n#5 \\\/var\\\/www\\\/apps\\\/files_sharing\\\/lib\\\/Cache.php(106): OC\\\\Files\\\\Cache\\\\Wrapper\\\\CacheJail->moveFromCache(Object(OCA\\\\Files_Sharing\\\\Cache), 'Photos', 'Photos')\\n#6 \\\/var\\\/www\\\/lib\\\/private\\\/Files\\\/Cache\\\/Updater.php(198): OCA\\\\Files_Sharing\\\\Cache->moveFromCache(Object(OCA\\\\Files_Sharing\\\\Cache), 'Photos', 'Photos')\\n#7 \\\/var\\\/www\\\/lib\\\/private\\\/Files\\\/View.php(330): OC\\\\Files\\\\Cache\\\\Updater->renameFromStorage(Object(OCA\\\\Files_Trashbin\\\\Storage), 'Photos', 'Photos')\\n#8 \\\/var\\\/www\\\/lib\\\/private\\\/Files\\\/View.php(804): OC\\\\Files\\\\View->renameUpdate(Object(OCA\\\\Files_Trashbin\\\\Storage), Object(OCA\\\\Files_Trashbin\\\\Storage), 'Photos', 'Photos')\\n#9 \\\/var\\\/www\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/ObjectTree.php(268): OC\\\\Files\\\\View->rename('Test\\\/Photos', 'Test2\\\/Photos')\\n#10 \\\/var\\\/www\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(642): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\ObjectTree->move('Test\\\/Photos', 'Test2\\\/Photos')\\n#11 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpMove(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#12 \\\/var\\\/www\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#13 \\\/var\\\/www\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(479): Sabre\\\\Event\\\\EventEmitter->emit('method:MOVE', Array)\\n#14 \\\/var\\\/www\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(254): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#15 \\\/var\\\/www\\\/apps\\\/dav\\\/appinfo\\\/v1\\\/webdav.php(60): Sabre\\\\DAV\\\\Server->exec()\\n#16 \\\/var\\\/www\\\/remote.php(165): require_once('\\\/var\\\/www\\\/apps\\\/d...')\\n#17 {main}\",\"File\":\"\\\/var\\\/www\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Driver\\\/AbstractPostgreSQLDriver.php\",\"Line\":91,\"User\":\"stefan\"}","level":4,"time":"2017-03-30T21:09:29+00:00","method":"MOVE","url":"\/remote.php\/webdav\/Test\/Photos","user":"fgrsnau","version":"11.0.2.7"}

#0 /var/www/cloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php(128): Doctrine\DBAL\Driver\AbstractPostgreSQLDriver->convertException('An exception oc...', Object(Doctrine\DBAL\Driver\PDOException))
#1 /var/www/cloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(836): Doctrine\DBAL\DBALException::driverExceptionDuringQuery(Object(Doctrine\DBAL\Driver\PDOPgSql\Driver), Object(Doctrine\DBAL\Driver\PDOException), 'SELECT "path", ...', Array)
#2 /var/www/cloud/lib/private/DB/Connection.php(194): Doctrine\DBAL\Connection->executeQuery('SELECT "path", ...', Array, Array, NULL)
#3 /var/www/cloud/lib/private/Files/Cache/Cache.php(526): OC\DB\Connection->executeQuery('SELECT `path`, ...', Array)
#4 /var/www/cloud/lib/private/Files/Cache/Wrapper/CacheJail.php(313): OC\Files\Cache\Cache->moveFromCache(Object(OCA\Files_Sharing\Cache), 'manuals/UPS_Giz...', 'files/Server/UP...')
#5 /var/www/cloud/apps/files_sharing/lib/Cache.php(106): OC\Files\Cache\Wrapper\CacheJail->moveFromCache(Object(OCA\Files_Sharing\Cache), 'manuals/UPS_Giz...', 'UPS_Gizeh')
#6 /var/www/cloud/lib/private/Files/Cache/Updater.php(198): OCA\Files_Sharing\Cache->moveFromCache(Object(OCA\Files_Sharing\Cache), 'manuals/UPS_Giz...', 'UPS_Gizeh')
#7 /var/www/cloud/lib/private/Files/View.php(330): OC\Files\Cache\Updater->renameFromStorage(Object(OCA\Files_Trashbin\Storage), 'manuals/UPS_Giz...', 'UPS_Gizeh')
#8 /var/www/cloud/lib/private/Files/View.php(804): OC\Files\View->renameUpdate(Object(OCA\Files_Trashbin\Storage), Object(OCA\Files_Trashbin\Storage), 'manuals/UPS_Giz...', 'UPS_Gizeh')
#9 /var/www/cloud/apps/dav/lib/Connector/Sabre/ObjectTree.php(268): OC\Files\View->rename('AG DSN/manuals/...', 'Server/UPS_Gize...')
#10 /var/www/cloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php(642): OCA\DAV\Connector\Sabre\ObjectTree->move('AG DSN/manuals/...', 'Server/UPS_Gize...')
#11 [internal function]: Sabre\DAV\CorePlugin->httpMove(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
#12 /var/www/cloud/3rdparty/sabre/event/lib/EventEmitterTrait.php(105): call_user_func_array(Array, Array)
#13 /var/www/cloud/3rdparty/sabre/dav/lib/DAV/Server.php(479): Sabre\Event\EventEmitter->emit('method:MOVE', Array)
#14 /var/www/cloud/3rdparty/sabre/dav/lib/DAV/Server.php(254): Sabre\DAV\Server->invokeMethod(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
#15 /var/www/cloud/apps/dav/appinfo/v1/webdav.php(60): Sabre\DAV\Server->exec()
#16 /var/www/cloud/remote.php(165): require_once('/var/www/cloud/...')
#17 {main}

I was digging a little bit deeper today. The failure is visible in

server/lib/private/Files/Cache/Cache.php

Line 515 in 4821c00

list($sourceStorageId, $sourcePath) = $sourceCache->getMoveInfo($sourcePath);

as sourceStorageId is false. This is, of course, not a valid integer in PostgresSQL and we get this nice error: ERROR: invalid input syntax for integer: \"\"

I tracked this value back to

server/apps/files_sharing/lib/Cache.php

Line 81 in 4821c00

return false;

.

Somehow the $sourceCache is not populated correctly (the numericId is not set). Currently I do not now how to debug this further.

The text was updated successfully, but these errors were encountered:

blizzz · 2017-04-03T12:06:42Z

@schiessle @rullzer

rullzer · 2017-04-03T13:28:14Z

Let me summon @icewind1991

MorrisJobke · 2017-04-05T03:58:51Z

I had the same issue :/

schiessle · 2017-04-05T09:01:35Z

In the past we blocked moving one mount point into another one because this create a lot of problems for re-shares, allow to create share-circles, etc. I think that's still true, right? So what we need to fix here is the check for moving one mount point into another one and block it.

Or did I miss something, @rullzer ?

rullzer · 2017-04-05T12:18:42Z

@schiessle it is not moving the mount point it is moving a subfolder of a mount point.

schiessle · 2017-04-05T18:48:20Z

ah, I see... maybe a permission issue? user 'c' needs to have delete permissions of "da" and create permissions on "db".... I read nothing about the share permissions in this thread that's why I'm asking.

fgrsnau · 2017-04-05T18:57:23Z

I did not move the mount point itself (“shared folder“ symbol in the root directory) but a subdirectory from one shared folder to another one (as @rullzer said). Interestingly, moving a single file from one share to a different share seems to work.

Permissions are not the problem. Full access was granted for all shares.

I was also able to reproduce this using only two users. User "b" shares two directories "d1" and "d2". User "a" is not able to successfully move a subdirectory of "d1" to a location anywhere in "d2".

rullzer · 2017-04-06T06:34:47Z

You move it via webdav I assume? Or in the webinterface?

rullzer · 2017-04-06T06:50:29Z

Mmmm I'm getting a permission denied (not ideal but at least no data is lost)

Steps I took

create user0
user0 creates a
user0 creates b
user0 creates b\c
create user1
user0 shares a with user1
user0 shares b with user1
user1 logs in see a and b
via webdav user1 tries to move b\c to a\c
Permission denied

fgrsnau · 2017-04-09T09:21:41Z

No, I move the files with the web ui (which seems to issue a MOVE webdav call).

I perform exactly the same steps as @rullzer described. I don’t get a Permission denied failure (but the very generic error message "Could not move $DIRECTORY") and the files are moved physically (user1 has full r/w permissions on both shares a and b).

Additionally, I think that this operation should not lead to any error and has to be implemented correctly as moving folders between shares is a very common workflow. A workaround is to let user0 move b/c to d/c and afterwards d/c to b/c where d/ is a "normal" folder owned by user0.

rullzer · 2017-04-12T13:21:51Z

@fgrsnau I think I was actually hitting #4327 I'll retest later.

MorrisJobke · 2017-04-12T16:45:37Z

Should be fixed with #4329

rullzer · 2017-04-12T17:27:11Z

I'm not so sure @MorrisJobke. Sice hte bug that is fixes by #4329 was not really present in 11 yet.

icewind1991 · 2017-04-13T14:16:42Z

Some quick testing suggests that this on is fixed now

fgrsnau · 2017-04-14T20:35:04Z

I tried with current master branch and moving directories between shares is working! (Note that 983210d was not working.)

Unfortunately, applying #4329 to the current stable release does not help. It is still not working.

rullzer · 2017-04-21T20:19:46Z

@fgrsnau mmm good to know it is fixed. But not so good that we don't know why.
Are you familiar with git? And if so could you do a bisect to find out when we fixed it?

fgrsnau mentioned this issue Mar 30, 2017

Database and filesystem out of sync (files visible in web interface but never hit filesystem) #3071

Closed

blizzz added the feature: sharing label Apr 3, 2017

MorrisJobke added the bug label Apr 5, 2017

MorrisJobke closed this as completed Apr 12, 2017

rullzer reopened this Apr 12, 2017

MorrisJobke closed this as completed Aug 15, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Moving from one shared folder to a different shared folder fails (database gets out of sync) #4169

Moving from one shared folder to a different shared folder fails (database gets out of sync) #4169

fgrsnau commented Mar 30, 2017 •

edited

Loading

blizzz commented Apr 3, 2017

rullzer commented Apr 3, 2017

MorrisJobke commented Apr 5, 2017

schiessle commented Apr 5, 2017

rullzer commented Apr 5, 2017

schiessle commented Apr 5, 2017

fgrsnau commented Apr 5, 2017

rullzer commented Apr 6, 2017

rullzer commented Apr 6, 2017

fgrsnau commented Apr 9, 2017

rullzer commented Apr 12, 2017

MorrisJobke commented Apr 12, 2017

rullzer commented Apr 12, 2017

icewind1991 commented Apr 13, 2017

fgrsnau commented Apr 14, 2017

rullzer commented Apr 21, 2017

Moving from one shared folder to a different shared folder fails (database gets out of sync) #4169

Moving from one shared folder to a different shared folder fails (database gets out of sync) #4169

Comments

fgrsnau commented Mar 30, 2017 • edited Loading

Steps to reproduce

Expected behaviour

Actual behaviour

Server configuration

NextCloud Logs

blizzz commented Apr 3, 2017

rullzer commented Apr 3, 2017

MorrisJobke commented Apr 5, 2017

schiessle commented Apr 5, 2017

rullzer commented Apr 5, 2017

schiessle commented Apr 5, 2017

fgrsnau commented Apr 5, 2017

rullzer commented Apr 6, 2017

rullzer commented Apr 6, 2017

fgrsnau commented Apr 9, 2017

rullzer commented Apr 12, 2017

MorrisJobke commented Apr 12, 2017

rullzer commented Apr 12, 2017

icewind1991 commented Apr 13, 2017

fgrsnau commented Apr 14, 2017

rullzer commented Apr 21, 2017

fgrsnau commented Mar 30, 2017 •

edited

Loading