Update to 20.0.2 fails at integrity check

[dh3wr]

Steps to reproduce

1. Use updater to 20.0.2

Expected behaviour

• update

Actual behaviour

• integrity check fails
• no response or reason given in message

Environment data

Nextcloud server version: 20.0.1

[AndyScherzinger]

@MorrisJobke transfered from the Android repo since this is not related to the Android App.

[blizzz]

@MorrisJobke we saw it today on a demo instance, alas there was nothing logged. I restarted by removing data/updater-$INSTANCEID/.step and running the updater again, this time it went through without issue.

[MorrisJobke]

Without logs there isn't much we can do. Best would be the updater.log inside the nextcloud data directory.

[blizzz]

Without logs there isn't much we can do. Best would be the updater.log inside the nextcloud data directory.

I don't see anything useful there. It just stops after 2020-11-30T21:29:12+0100 qPTIgT7kkT [info] storage location: /mnt/demos/mdp/data/updater-ocqvzan9bdza/downloads/. Following lines are from poking it via php updater.phar.

2020-11-30T21:29:12+0100 qPTIgT7kkT [info] POST request for step "5"
2020-11-30T21:29:12+0100 qPTIgT7kkT [info] startStep("5")
2020-11-30T21:29:12+0100 qPTIgT7kkT [info] verifyIntegrity()
2020-11-30T21:29:12+0100 qPTIgT7kkT [info] getUpdateServerResponse()
2020-11-30T21:29:12+0100 qPTIgT7kkT [info] updaterServer: https://updates.nextcloud.com/updater_server/
2020-11-30T21:29:12+0100 qPTIgT7kkT [info] releaseChannel: beta
2020-11-30T21:29:12+0100 qPTIgT7kkT [info] internal version: 20.0.2.0
2020-11-30T21:29:12+0100 qPTIgT7kkT [info] updateURL: https://updates.nextcloud.com/updater_server/?version=20x0x2x0xxxbetaxx2020-11-12T20%3A16%3A16%2B00%3A00+105fc60eb15a78d22ef34ac8a1f49b5842a58b06x7x4x3
2020-11-30T21:29:12+0100 qPTIgT7kkT [info] getUpdateServerResponse response: Array
(
   [version] => 20.0.2.2
   [versionstring] => Nextcloud 20.0.2
   [url] => https://download.nextcloud.com/server/releases/nextcloud-20.0.2.zip
   [web] => https://docs.nextcloud.com/server/20/admin_manual/maintenance/upgrade.html
   [changes] => https://updates.nextcloud.com/changelog_server/?version=20.0.2
   [autoupdater] => 1
   [eol] => 0
   [signature] => xgmYqDDmUkSdkP3+01rdn7FKMZ6/3fYD+lx9Ldrbv/Lg+CmM1SPpzO8xl+XvZi0z
C4JMB8BgPYMaJZZ2jZCJGuio9BZTqxtVBi738D3ZD2Iqk5M1RqNF8EYnmCiptTj6
Y/b8Y7IOElPURLLjelBjT+wWyaMdBEifGmWZznhbaiOq7vNB8u31yFlTKBEn3qB+
zd2ymDMn9r0aSrFmuJWvqbp8VMRFg166UBuHoeQ1fJlA2oA3HWDtsNxGQCPISD+M
Kpuqj2XLRa72vi12QgnMMYRoFNQhZ8Zmv/EdpTlb5ubdMQtNxAiYGN0nCR37+F3T
aiywvIt69DttL/rlTZPTFw==
)

2020-11-30T21:29:12+0100 qPTIgT7kkT [info] storage location: /mnt/demos/mdp/data/updater-ocqvzan9bdza/downloads/
2020-11-30T21:29:48+0100 WuEG72K2Zb [info] request to updater
2020-11-30T21:29:48+0100 WuEG72K2Zb [info] currentStep()
2020-11-30T21:29:48+0100 WuEG72K2Zb [info] Step 5 is in state "start".
2020-11-30T21:29:50+0100 wUzn0GaN04 [info] request to updater
2020-11-30T21:29:50+0100 wUzn0GaN04 [info] currentStep()
2020-11-30T21:29:50+0100 wUzn0GaN04 [info] Step 5 is in state "start".
2020-11-30T21:31:51+0100 nkdUmn8rN9 [info] request to updater
2020-11-30T21:31:51+0100 nkdUmn8rN9 [info] currentStep()

Ha, but now looking into the web server error logs, it seems we died for RAM

[Mon Nov 30 21:29:12.744458 2020] [proxy_fcgi:error] [pid 83610:tid 139787981788928] [client 1.2.3.4:5] AH01071: Got error 'PHP message: PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 141997160 bytes) in /path/to/updater/index.php on line 754'

And looking at the server conf, the memory limit is too low, 128M only.

So, PEBCAK.

[blizzz]

@dh3wr you may want to check whether this is what you ran into as well ^

[dh3wr]

@dh3wr you may want to check whether this is what you ran into as well

root@12345:/srv/nextcloud/updater# sudo -u www-data php updater.phar
Nextcloud Updater - version: v20.0.0beta4-11-g68fa0d4

Step 5 is currently in process. Please call this command later.

How can I stop the updating process?

[blizzz]

@dh3wr you may want to check whether this is what you ran into as well

root@12345:/srv/nextcloud/updater# sudo -u www-data php updater.phar
Nextcloud Updater - version: v20.0.0beta4-11-g68fa0d4

Step 5 is currently in process. Please call this command later.

How can I stop the updating process?

I restarted by removing data/updater-$INSTANCEID/.step and running the updater again

[dh3wr]

Deleting data/updater-$INSTANCEID/.step and running the updater again resulted in update without errors. Still interesting it failed the first time. This ticket can be closed form my point-of-view, if you don't want to investigate more or mitigate the cause of failure from the first attemp.

[blizzz]

@dh3wr did you check your web server's error log? And/or are there any warnings in your settings overview?

[dh3wr]

[Mon Nov 30 19:47:43.284108 2020] [php7:error] [pid 21793] [client 192.168.17.250:44712] PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 141997160 bytes) in /srv/nextcloud/updater/index.php on line 754

I did not change anything but calling the updated from the bash instead of the webinterface.

[blizzz]

You're memory_limit in PHP settings is set to 128MB, which is below the recommendation and this is where the updates trips on. IIRC the recommended value is 512 MB. But you should also see a warning about it in the admin settings.

[b0ssi]

Hi all,

I've had the same issue (mentioned in the forums here. Raising PHP's memory_limit to >=512M yielded success in my case as well.

Some observations gathered from debugging seems to agree as well - the updater crashes somewhere in this call (The log entry [info] storage location: [...] from this call: $this->getDownloadedFilePath() still prints, but the openssl_verify call doesn’t return any more).

[blizzz]

it's not a bug in the updater when the necessary resources are not provided. The solution is to have the PHP configured accordingly and pay attention to the warnings on the admin page.

[b0ssi]

I would agree with your opinion. Could it be a good idea to raise the notification level for min. memory limit on the admin page from info (I believe it is) to something that would indicate a harder necessity? Or alternatively rephrase that the risk of unpredictable behavior becomes more apparent? As it currently stands, I'm mostly interpreting a soft degradation of performance into the notice, but not necessarily a risk of hard crashes.

[blizzz]

I would agree with your opinion. Could it be a good idea to raise the notification level for min. memory limit on the admin page from info (I believe it is) to something that would indicate a harder necessity? Or alternatively rephrase that the risk of unpredictable behavior becomes more apparent? As it currently stands, I'm mostly interpreting a soft degradation of performance into the notice, but not necessarily a risk of hard crashes.

Well, if it would not matter, it would not have been listed there at all. So, yes, it can have effects if it is ignored:

Nevertheless we can make it more red, if it helps…

[kesselb]

if it helps…

Or remove the ability to update via web ;)

[blizzz]

if it helps…

Or remove the ability to update via web ;)

something else might break

[b0ssi]

Good idea I think. Having the notice communicate a stronger degree of necessity and clearly point out that dismissal could have side-effects would help draw a more consistent picture.

[horrokotoj]

I just tried to update from 19.0.13 to 20.0.14 and ran in to this problem.
I however can not find a file data/updater-$INSTANCEID/.step to remove. So I am stuck with "Step 5 is currently in process. Please reload this page later.". Anyone have an idea on how to stop the updater?

[genesisfactor]

@horrokotoj , i know this is over 4 months old but for anyone else in the future, the $INSTANCEID part of updater-$INSTANCEID is a wildcard for an alphanumeric string associated with your instance. Just look for updater-<bunch of letters and numbers> using ls -al. You may have to use sudo to do the remove.

also data is a folder in yor NextCloud install directory